FDA should require device makers to include more diverse populations in their analysis of clinical trials

September 19, 2016


Comments of Patient, Consumer, and Public Health Coalition

 on “Evaluation and Reporting of Age, Race, and Ethnicity Data in Medical Device Clinical Studies: Guidance for Industry and Food and Drug Administration Staff”

Docket No. FDA-2016-D-0734


Members of the Patient, Consumer, and Public Health Coalition have substantial concerns about the weaknesses of the draft guidance “Evaluation and Reporting of Age, Race, and Ethnicity Data in Medical Device Clinical Studies.”  It does not go far enough to improve the proportion of medical devices that are as safe and effective as possible for all subgroups included in an indication for approval.

The purpose of the guidance is to outline FDA’s expectations for the evaluation and “reporting of age, race and ethnicity [subgroups] data in medical device clinical studies.”  FDA evaluated the subgroup “sex” in a previous guidance.[1] 

We agree with the guidance’s recommendation that “clinical trials include diverse populations that reflect the intended populations,” but strongly disagree with the caveat “especially when clinically meaningful differences in safety, effectiveness…or benefit-risk profile are expected across these groups.”   We oppose that caveat because in many cases when the risk-benefit ratio differs, those differences were not expected.

We agree that sponsors (device makers) “should develop a strategy to enroll diverse populations.”  We also agree that these data are needed because “historically many medical device clinical studies” have not included subgroups in proportions that reflect “disease distribution in the affected [subgroup] population.”[2]  In fact, members of our coalition have worked on this issue for many years. For example, at a 2014 FDA AC meeting for a weight-loss device, National Center for Health Research (NCHR) noted that even though four out of five Black women in the U.S. are overweight or obese and Hispanic women are more likely to be overweight compared to White women as well, the clinical trial was 93% White. Only 11 patients were Black and only 2 were Hispanic.  Although the Advisory Committee members voiced their concerns about the mostly white sample, they voted for approval of this device anyway.  This shows two problems: 1) the FDA is not requiring diversity in clinical trials and 2) when companies fail to include sufficient numbers of minority groups to conduct subgroup analyses, the FDA is not providing instructions to their Advisory Committee members that such diversity is essential for FDA approval and for public health.

Several of our members also spoke at the April 2014 Section 907 hearing on this issue.  We noted the importance of including women, minorities and the elderly in clinical studies of safety and effectiveness for drugs and devices. We recommended that subgroup analyses be used as a basis for approval, in labeling decisions, and that the information be made available to patients and providers.

This draft guidance addresses many of the concerns raised previously by coalition members, and we are concerned that the FDA has continued to approve devices that were not analyzed in subgroup analysis proving that benefits outweigh the risks for women, people of color, and people over 65.  If FDA continues to approve/clear devices without subgroup data, then many medical device companies will ignore this guidance, as they often have in the past.  While guidances are not enforceable, FDA has the authority to refuse to approve medical products that have not been adequately tested on representatives of the population that will rely on them.  For example, if the FDA clearly states that a sponsors’ applications will be rejected — or perhaps approved only for the population that was adequately tested in subgroup analyses of safety and effectiveness, we are confident that companies will find a way to comply.


Below are our specific comments on the guidance:


Analyses should focus on each subgroup independently

Analyses should focus on whether a device is safe and effective for each subgroup independently, rather than whether there is a difference between groups.

Differences in safety and effectiveness between subgroups are much less important than whether benefits outweigh the risks for each subgroup.  For example, in a study of 20,000 patients, it is possible that a small difference between racial groups would be statistically significant for a device that is very effective for both all races.  But, for patients, that small difference would not matter.  What matters is not that they differ, but whether or not the device is useful for all racial groups.

Another reason why comparisons between or among groups is not sufficient is that smaller sample sizes result in wider confidence intervals. Therefore, the subgroups with fewer patients have a greater chance of appearing not significantly different from the other groups.  This would mask much of the meaningful information about safety and effectiveness in specific subgroups.  The guidance recommends device makers engage in “consultation with FDA” when sample sizes are not large enough.  This unfortunately will result in subjective case-by-case reviews, which can result in bias as well as being labor and resource-intensive for CDRH, which is under-funded. If the FDA believes such case-by-case reviews are needed, industry should pay for them through generous user fees.  Such “consultations” should not be funded by appropriations that are already inadequate for CDRH.


Analysis by age

The guidance notes that different age populations are “often underrepresented” in clinical trials for medical devices, and it cited the 2013 FDASIA 907 report that found “only 40%” of approved PMAs “reported an age based analysis of outcome data” and that age information was inconsistent and not detailed enough.  Medical device companies must provide more and better data. The guidance states that “older patients may have age-related covariates” (less bone density, slower metabolisms, and digestion issues) “that could affect the performance of medical devices.”   It also notes that pediatric subgroups are affected differently by medical devices “due to the size of the implant,” and that they are more “radiosensitive than adults.”

We agree with the guidance’s recommendation that studies should use more discrete age categories in performing subgroup analyses.  Even though FDA does not define a specific age for the geriatric population, we strongly urge that devices intended to be used by older patients should include an analysis of patients who are over 65 separately from younger adults.  FDA has expressed an interest in harmonizing with the Centers for Medicare & Medicaid Services (CMS), and since most Medicare patients are 65 and older, research evidence that a device is safe and effective in this age group would benefit patients, companies, and CMS.


Analysis by race, and ethnicity

The guidance notes that the U.S. population is becoming more diverse, yet subgroup representation in clinical trials “remains a challenge and inconsistent analysis and reporting contributes to persistent lack of publicly available data on device performance in diverse ethnic and racial groups.” The FDASIA 907 Report showed that “only 27% of the studies reviewed contained a race or ethnicity subgroup analysis and only 16% had public statements” regarding the subgroups.  Recent analyses by the National Center for Health Research found that this continued to be a problem for devices FDA reviewed in 2015.  It is critical that racial and ethnic subpopulations be adequately represented in clinical trials in order to ensure that the tested product is safe and effective for all the subpopulations that are likely to use that product.


Postmarket submissions

The guidance states that “consideration should be given to whether market approval/clearance is supported for the general population, with postmarket studies to gain further information regarding any observed…subgroup differences.” We urge FDA to require subpopulation data be analyzed before the device is marketed and that the focus be on benefits outweighing risks for each subgroup, rather than comparing subgroup differences.

Once a device maker has its product on the market, there is no incentive for them to do postmarket studies in a timely manner (or to complete them at all) since their device has already been approved/cleared by the FDA.  Additionally, the guidance notes there are “concerns about disproportionate dropout and loss to follow-up [which] are potential barriers to diverse study representation of minorities and older patients.”  For that reason, companies need to create incentives to keep patients in their studies.  If those studies are not completed appropriately, the FDA should rescind approval.


Unplanned subgroups analyses

We agree with the guidance that “unplanned subgroup analyses or those with inadequate sample size are generally not considered to be adequate to support statements in the labeling regarding the safety or effectiveness of the device.”  We suggest deleting the word “generally” from that sentence.


Device labeling

We agree with the guidance that “when clinically relevant differences in treatment effect are anticipated across age, race, or ethnic groups, these effects should be considered in the study design and…device labeling.”  However, as stated above, it is not sufficient to only consider such issues when subgroup differences are anticipated – they should always be considered.  In addition, also as stated above, subgroup differences are not the key issue; what matters to patients and providers is how safe and how effective devices are for patients in each subgroup.

In addition, we are concerned that patients will never see the label for the devices implanted in their bodies (hips and knees, for example).  Even more disturbing, surgeons have told us that they do not see the medical device labels either. FDA should investigate whether or not surgeons actually see and read the device labels, or if the labels are removed before surgeons receive the devices in the operating room.


“Snapshots” for medical device trials

Although it is beyond the scope of this guidance, the FDA should consider creating an equivalent to the FDA Drug Trials Snapshots for medical devices. This would make demographic subgroup data more available and transparent to the public.



To ensure that medical devices intended for use by the entire U.S. population are safe and effective for all relevant subgroups, FDA needs to require device makers to include more diverse populations in their analysis of clinical trials, and needs to require that subgroups be large enough to analyze to determine how safe and how effective a device is for each major subgroup population.   If a device is not safe and effective for all target groups, then labels should say that and approval indications should be restricted to those populations for whom benefits outweigh the risks.  Otherwise, doctors and patients will not have the information needed to make decisions about whether to use the device for treatment.

We are disappointed that this guidance, like other FDA documents, continues to miss the essential issues regarding subgroup analyses: to answer the question of how safe and how effective is this device for each major subgroup.   We strongly urge the FDA to require that safety and effectiveness be analyzed separately for each subgroup instead of just comparing safety or effectiveness between or among subgroups.  Companies need to collect sufficient information to evaluate the benefit/risk profile for each subgroup independently.  That will make it possible to determine if medical devices are sufficiently safe and effective for anyone who uses them.

Without these changes, the guidance will not help ensure that patients and providers have the information they need to determine if a device is likely to be safe or effective for a wide range of patients in terms of age, race, and ethnicity.  Moreover, it is crucial that FDA hold device makers accountable by not approving devices for all Americans if it has not been proven to have benefits that outweigh the risks for all major subgroups that will use those devices.


American Medical Student Association

American Medical Women’s Association

Breast Cancer Action

Breast Cancer Consortium

Connecticut Center for Patient Safety

MRSA Survivors Network

National Center for Health Research

National Organization for Women

Our Bodies Ourselves

The TMJ Association

Washington Advocates for Patient Safety



The Patient, Consumer, and Public Health Coalition can be reached through Paul Brown at (202) 223-4000 or at pb@center4research.org

[1] Food and Drug Administration (2014).  Guidance: Evaluation of Sex Specific Data in Medical Device Clinical Studies. http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm283707.pdf

[2] Food and Drug Administration (2016).  Draft guidance: Evaluation and Reporting of Age, Race, and Ethnicity Data in Medical Device Clinical Studies. http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM507278.pdf