Kenneth A. Getz

The world is anxiously watching and rooting for the global drug development enterprise in anticipation of COVID-19 treatments and vaccines. Pharmaceutical and biotechnology companies, in collaboration with regulatory and government agencies, have taken many steps to try to compress drug development timelines. To name but a few: they have rapidly mobilized scientific and operating teams; entered into an unprecedented number and variety of co-development arrangements; scaled-up manufacturing capabilities; and leveraged technology solutions and advanced analytics to access and analyze data from animal and early human studies and from commercially available therapies treating other diseases.

Regulatory agencies have issued emergency use authorizations and offered research sponsors guidance to assist preparedness, ensure recruitment continuity, and embrace more virtual and remote approaches to conducting clinical trials.

Of the 400+ COVID-19 treatments and vaccines now in R&D, a rapidly increasing number are in human testing with several promising therapies in late-stage clinical trials. The successful development of safe and effective COVID-19 therapies within a forecasted 12-to-24-month time horizon is truly groundbreaking. Indeed, our research at the Tufts Center for the Study of Drug Development (Tufts CSDD) shows that such an acceleration represents a 400% speed improvement: historically, experimental immunological and anti-infectives take approximately eight years to develop from first-in-human clinical trials to FDA approval. Late-stage Phase III vaccine trials alone usually take around three years.

Will R&D strategies and tactics designed to dramatically shorten timelines during the pandemic translate across development portfolios post-pandemic? This is a question that many sponsor companies are exploring and it couldn’t come at a better time.

The most recent Tufts CSDD benchmarks—gathered at the end of 2019 just prior to the pandemic—show that while mean regulatory review times for U.S.-approved drugs and biologics (referred to throughout this column as “drugs”) have declined by nearly two months during 2014 and 2018, average clinical phase times have increased by nearly seven months during the same period. Clinical phase times for drugs targeting rare diseases are one of the few positive news items coming out of our recent analysis: orphan drugs now take more than nine months less time, on average, to complete clinical trials and receive regulatory approval compared with non-orphan drugs.

The results of this recent study present useful baseline measures gathered just before the COVID-19 pandemic from which to assess post-pandemic strategies and tactics to accelerate development and approval timelines.

Tufts CSDD analyzed clinical development and regulatory approval phase times for all 377 new drugs and biologics approved by the FDA’s Center for Drug Evaluation and Research (CDER) during 2008 to 2018. Clinical trial durations and scope metrics were obtained from summaries of drug approvals available on the FDA website, from medical reviews, and from the ClinicalTrials.Gov website. Benchmarks were derived from several subgroup analyses, including by therapeutic area, whether an approved drug was originally with an orphan drug designation, and whether the drug was developed and approved by one of four expedited FDA review programs (e.g, accelerated approval, fast track, breakthrough designation, and priority review).

This study was funded in part by an unrestricted grant from the Biotechnology Innovation Organization (BIO).

The average cycle time for all clinical testing (clinical phase duration) from IND filing to FDA submission (shown in Figure 1 below), for drugs approved during 2014 -2018 is 89.8 months, an increase of 6.7 months, or 8.1%, compared to the earlier 2008-2013 time period. Mean regulatory phase duration—from submission to FDA approval—for all drugs approved between 2014 and 2018 was 11.7 months, a decrease of 1.9 months, or 14%, shorter than the earlier 2008-2013 timeframe. Average total time to bring a successful drug through clinical testing and into the market (i.e., from IND filing to regulatory approval) was 101.5 months—approximately eight-and-a-half years. This represents an increase of 4.8 months, or 5%, between the two time periods, 2008-2013 and 2014-2018.

Increases in the overall clinical phase duration are due to growth in cycle times for individual clinical trials in each phase of development. Substantial growth in protocol design complexity has been a major contributor to phase-specific cycle time increases. Phase I clinical trials have increased more than 7% in duration between the two time periods—from 13.8 months on average to 14.8 months. Phase II clinical trials saw the largest relative increase in duration from an average of 27.1 months to 30 months—or nearly 11%—between the 2008-2013 and 2014–2018 observation periods. And Phase III clinical trial durations have increased by more than 6% between the two time periods, from an average of 26.8 months in the 2008-2013 period to an average of 28.5 months per individual trial in the 2014-2018 timeframe.

Scope increases in clinical trials by phase is another factor driving cycle time increases. With the exception of Phase IV studies, the average number of participants and investigative sites per clinical trial, in all pre-approval phases, have increased.

The highest increases have been observed in Phase II: the mean number of participants, for example, increased 9.2%, from 152 in 2008-2013 to 166 in 2014–2018.The average number of participants in Phase I and III clinical trials increased 5%–7% between the two time periods.

The average number of participants in Phase IV clinical trials has declined sharply—by 26.1%—in part due to the growing proportion of orphan drug approvals now being marketed.

Very high variation was observed around the mean number of participants per clinical trial and this variation has increased over time.

In this most recent study, Tufts CSDD also observed wide variation in overall duration (clinical plus approval phase) by therapeutic area. Looking at the most active therapeutic areas—those with the largest number of drug approvals during the full 11-year (2008-2018) period—anti-infectives had the shortest average overall duration of 84 months. CNS had the longest average overall duration of about 108 months. The average clinical phase duration for anti-infectives was 17.6 months. This is 19.2% shorter than that of CNS treatments.

Oncology approvals between 2008 and 2018 averaged almost 95 months overall per drug—87.2 months in the clinical phase and 7.7 months in the regulatory phase. Some of the variation in review durations can be explained by differences in the proportion of drugs that received a priority review rating from FDA. Whereas 74% of drugs approved in oncology had a priority rating, only 43% of CNS drugs were designated for priority review.

Overall durations (clinical plus regulatory phases) for orphan drug approvals decreased from 109.3 months, on average, in 2008-2013 to 96.6 months in 2014-2018, an acceleration of more than a year of time.Mean clinical duration for orphan drug approvals during 2014-2018 was 5.3 months faster than for non-orphan drug approvals.The difference between orphan and non-orphan drug mean regulatory review durations has increased with orphan drugs now seeing a four-month speed advantage during the later 2014-2018 time period. Mean clinical duration for non-orphan drug approvals increased substantially (16.2 months) in the 2014–2018 period, whereas mean regulatory review duration for non-orphan drugs decreased by one month.

As expected, approvals under one of four expedited or facilitated regulatory pathways (FRPs) were considerably faster than those receiving non-expedited review. The overall duration for approvals of drugs under expedited review was 11% shorter, on average, than non-expedited review approvals during the full 11-year (2008–2018) observation period.

Compared to approvals under non-expedited review, the mean clinical phase duration for drugs approved under an FRP was four months shorter in the 2008-2013 time period (4.7%) and 4.2 months (4.5%) shorter in the 2014-2018 time period. The average regulatory review phase duration for expedited review drugs was approximately 45% faster than than for non-expedited drugs during both 2008-2013 and 2014–2018 time periods.

For approvals under expedited and non-expedited review, mean clinical phase durations increased by 7.4 months (9.1%) and 7.6 months (8.9%), respectively. Meanwhile, the mean regulatory review phase duration decreased by 0.3 months (3.1%) for expedited approvals and 0.7 months (4.0%) for non-expedited approvals.

Tufts CSDD also evaluated the impact of sponsor company size on development and regulatory approval speed. This subgroup was assessed in response to recent claims that smaller companies (i.e., organizations spending less than $1 billion on R&D annually) enjoy a speed advantage over the largest pharmaceutical and biotechnology companies. Looking at all approved drugs during 2008 and 2018, the claim is not supported. Approvals of drugs sponsored by the largest companies enjoy a nine-month average clinical duration advantage and a 1.2-month average regulatory review advantage.

We look forward to assessing the speed records broken by COVID-19 treatments and vaccines when they complete development. Contrasting them against our most recent pre-pandemic clinical and regulatory review phase durations will provide valuable insights into time-compression strategies and practices that can be applied across development portfolios long after the pandemic is behind us.

, MBA, is Deputy Director and Research Professor, 

, both based in Boston, MA. Email: kenneth.getz@tufts.edu

Articles

Study offers clues for installing rapid R&D tactics post-pandemic.

Applying Pre-Pandemic Benchmarks to Assess Future Speed Strategies

	Inadvertent and intentional dose non-adherence in clinical trials has been a major drug development challenge for decades. Studies in the literature have demonstrated that a high percentage of study volunteers-as high as 75% -conceal or fabricate information and provided inaccurate self-reports of adherence.

	Participant non-adherence reduces statistical power, and leads sponsors to underestimate the true dose response

 and toxicity from investigational treatments and to overestimate the true dose level required. The anticipation of study participant non-adherence compels sponsor companies to expend resources and personnel to recruit an inflated number of study volunteers and to unethically expose larger numbers of patients to an investigational treatment. A common inflation factor used by biostatisticians is 15%-20%; for a typical Phase III pivotal trial, an additional 120-150 patients are recruited to account for non-adherent study volunteer behavior. Partial and full non-adherence may also cause sponsor companies to miss detecting treatment effects when they do occur, resulting in failure to identify a potentially effective drug.

	Several well-publicized examples provide compelling evidence that non-adherence directly contributes to clinical trial, and even development program, failure. In a recent Phase II clinical trial of a treatment for adult attention deficit hyperactivity disorder (vortioxetine), the clinical team found that non-adherence compromised study validity. The team reported that high levels of non-adherence in the clinical trial lowered the signal effects of the investigational treatment and contributed to the failure to demonstrate differences in the primary and key secondary endpoints for placebo and active study drug cohorts. A similar conclusion was drawn from a recent Phase I clinical trial evaluating the effects of vilazidone on sexual function in healthy adults.

	A large Phase III pivotal trial conducted in the late 1980s on a cholesterol-lowering investigational drug failed to account properly for high non-adherence rates overall and at protocol-specific time intervals. As a result, the sponsor and FDA underestimated the dose response and overestimated the dose level causing patients to pay unnecessarily higher costs to receive the treatment.  

	Although there is strong and broad consensus that the accurate estimation of non-adherence rates in clinical trials is essential to determining the dose-response relationship and to allowing for a valid analysis of treatment safety and efficacy, accurate estimates have yet to be produced. Reported non-adherence rates in the literature are largely anecdotal, varying widely from 15% to as much as 90%.

	There is virtually no robust data characterizing patient motivation to be non-adherent. Some speculate that inadvertent and intentional dose non-adherence is associated with patient perceptions that their condition has improved and an investigational treatment is no longer needed; that the study drug is not working; or the study drug is causing intolerable side effects.  

	There are also no hard benchmark data characterizing non-adherence rates; this is largely a function of assessment and measurement approaches that are highly limited. Many approaches cannot distinguish between a patient simply missing a dose or deliberately concealing one. Pill counting-one of the most commonly used approaches-is unreliable and tends to underestimate non-adherence rates. Patient questionnaires and diaries rely on subjective assessments that can easily be misstated. Ingestible sensors are more precise but many study volunteers are uncomfortable with this approach.  

	Medication packaging with imbedded memory chips (e.g., MEMS) has serious limitations since this approach cannot detect what the patient is doing once the cap has been removed. And evaluations of drug levels in biologic fluids and biologic markers are impractical because they must be performed at distinct time periods when blood samples are collected and within a finite time frame before the study drug has been metabolized. These approaches are also subject to patient-specific pharmacokinetic variation.  

	The most reliable and accurate method is direct patient observation. But this approach has historically been cost-prohibitive and difficult to carry out in outpatient settings. In recent years, however, artificial intelligence (AI) has improved the feasibility of direct patient observation and presented a unique opportunity for the Tufts Center for the Study of Drug Development (Tufts CSDD) to conduct an academic study in late 2019 to assess the magnitude of intentional dose non-adherence and inform statistical analysis planning.

	Tufts CSDD conducted this study using an AI platform developed by AiCure. The platform relies on computer vision and machine learning technologies using a front-facing camera to capture dosing data followed by human review if deliberate non-adherence cannot be determined. Study volunteers use the AiCure software on their smartphone during all dosing administrations. In all, the adherence behaviors of 2,796 study volunteers and 257,672 doses from 23 recently conducted, IRB-approved clinical trials primarily for CNS and neuromuscular diseases were analyzed.  

	Of the total 257,672 doses observed, 4% were confirmed as intentionally non-adherent. Nearly half of all study volunteers (48%) had at least one intentionally non-adherent dose; Nearly one in 10 volunteers had more than 10% of their total doses deliberately non-adherent over the course of their clinical trial; And one-out-of-20 volunteers (5%) were intentionally non-adherent for more than one-third of their total doses required by the protocol. 

	A number of factors were associated with intentional dose non-adherence. As the duration of the clinical trial increased, the likelihood of intentional non-adherence increased significantly. Less-experienced, low-annual-clinical-trial-volume investigative sites had a significantly higher incidence of study participants who were intentionally dose non-adherent.

	Study volunteers whose first dose within the clinical trial was intentionally non-adherent had a significantly higher mean intentional non-adherence rate throughout the clinical trial. The mean intentional non-adherence rate was 26% for study volunteers whose first dose was intentionally non-adherent-nearly six times the mean for participants whose first dose in the clinical trial was adherent (4%). Study volunteers with at least one intentionally non-adherent dose in the first week of the clinical trial had a significantly higher rate of non-adherence throughout the trial.

	The results of this study offer insights into a number of potential new practices and policies. Raising awareness, training, and additional support focusing on patient adherence specifically for less-experienced, lower-volume investigative sites may help address the higher incidence of intentional non-adherence there.

	Implementing practices and solutions designed to detect and act on intentional non-adherence within the first week of a clinical trial will likely have a major impact. Remedial patient education, for example, and targeting non-adherent study volunteers at the outset of a clinical trial may help increase overall participant adherence. Clinical trial enrichment strategies designed to identify non-adherence during a placebo run-in period might be strengthened by including a prespecified threshold of intentional non-adherence as part of the eligibility criteria for randomization. 

	Tufts CSDD hopes to conduct future research that will look to gather adherence benchmarks across a larger and broader mix of disease conditions-particularly the most active, including oncology, infectious diseases, and endocrinology. This study also only included U.S. clinical trials. Future research will expand our scope to include international clinical trials to derive global baseline measures. 

	The historical lack of accurate measures of dose adherence behaviors has contributed to industry-wide inability to detect product non-use during clinical trials. The results of this study present a useful first baseline measure of intentional non-adherence, and inform practices and policies to address it. At the present time, as pharmaceutical and biotechnology companies face intense pressure to accurately measure dose response and true treatment effects, reducing and preventing intentional dose non-adherence takes on even greater importance.

 MBA, is Deputy Director and Research Professor, Tufts CSDD and Chairman of CISCRP, both based in Boston, MA. email: 

AI presents rare opportunity to assess the scope of this longtime challenge. 

Taking Stock of Intentional Dose Non-Adherence

	It is widely known that diversity in clinical trials is an important part of developing new medications that are safe and effective for all potential patients in need of treatment. While researchers have spent more than 20 years investigating this aspect of clinical trials, the majority of studies are anecdotal; focus narrowly on a single therapeutic area, or a short list of therapeutic areas; or are drawn from a convenience sample.

	There have been a few studies, however, which attempt to look at the issue at a more macro level. In 2009, the Tufts Center for the Study of Drug Development (Tufts CSDD) used FDA’s New Drug Approval Database (drugs@FDA) to collect data on patient demographic data for studies on 27 drugs approved in 2007. These demographic data were then compared to U.S. census data and found that participants identifying as Black or of African descent, and Hispanic/Latinx participants were significantly underrepresented.

	In an effort to help address the need for diversity in clinical trials, Congress enacted FDASIA (the FDA Safety and Innovation Act) in 2012, which requires the agency to publish publicly available reports on trial participant demographics. Additionally, in 2014 the FDA began publishing the Drug Trial Snapshot, an annual 12–16-page report which highlights the distribution of clinical trial participant for the drugs approved during that year. Unfortunately, the Snapshot falls short in a few areas. Most importantly are the facts that it is a static report which makes evaluating trends difficult, and the report does not assess disparities in demographic diversity. The report presents what distribution looks like but gives no indication of where it should be.

	Improving our understanding of the demographic disparities in clinical trials will provide several benefits to the field. The information can be used to identify the areas of greatest need-which demographic subgroup disparities are the greatest, both overall and within specific therapeutic areas or disease conditions. The information can be used to assess whether participant diversity has changed over time and how it has done so. It can also be used to assess whether new diversity programs and policies are having an impact, allowing the effective programs to continue and the resources dedicated to ineffective programs to be reallocated to other programs that may have a greater impact.

	In 2018, the Tufts CSDD was awarded a grant from Merck to investigate, measure, and quantify the level of clinical trial participant diversity and the disparities therein. This research had three goals. The first was to assess the availability of participant demographic data. The second goal was to determine a baseline assessment of participant diversity and disparities among demographic subgroups. The third was to develop an approach that could be used by FDA and others to improve the clarity and value of the Drug Trial Snapshot and other publications reporting on clinical trial participant diversity. While data was collected for all trials when available, Tufts CSDD’s analysis focused mainly on disparity in pivotal trials since these are the trials FDA uses when considering the approval of a new drug application.

	In 2018, Tufts CSDD compiled data on all new drugs and biologics approved by the FDA from publicly available data on the agency 

. Data were gathered on a total of 341 NDAs and BLAs including Trade Name, Generic Name, Approval Date, Application Number, Classification (NDA or BLA), Sponsor, Indication, Therapeutic Area, FDA Designations (First-in-Class, Orphan, Fast Track, Breakthrough, Priority, Accelerated Approval, or 505b2). Participant demographic data including sex, race, ethnicity,  and age were also gathered for all new drug and biologic approvals. Supplemental data were collected from the Medical Reviews and Printed Labeling for each 

, and additional peer-reviewed publications.If demographic data were not found on these sources, the data were considered “not available.”

	To assess participant demographic subgroup disparities, clinical trial demographic data were compared with corresponding disease prevalence rates or U.S. census data. Tufts CSDD searched for and collected prevalence rates for each of the disease conditions for which a new drug or biologic was approved. Data were collected from publicly available sources such as government websites, national health organizations, and peer-reviewed literature. When disease prevalence data were not available, it was assumed that prevalence rates were equal between racial and demographic subgroups, and Tufts CSDD referred to US Census Data to impute participant demographic subgroup distributions.

	Tufts CSDD found disease prevalence rates for 193 drugs (56.6%) out of the total 341 approved, of which 114 (33.4%) target diseases where prevalence varied by race and ethnicity. U.S. Census data were collected for each year from 2007 to 2017. A 10-year census average was calculated for each demographic category by calculating the mean percentage across all years. 

	For each new drug and biologic approved between 2007 and 2017, Tufts CSDD calculated an expected or “Predicted” distribution of participants by demographic subgroup based on either the known disease prevalence or overall US population distributions. Tufts CSDD derived a “Disparity Percentage”-a summary statistic to characterize underrepresentation-for all approved drugs. This percentage is the difference between the “Actual” number of participants by demographic subgroup and the expected or “Predicted” level (i.e., determined by the identified prevalence rate or US census data in the approval year) divided by the “Predicted” level. An example of the Disparity percentage is given below. Means were calculated for this statistic for each participant subgroup by disease condition, therapeutic area, and year.

					Disease Condition for Approved Drug:

					“Actual” Distribution of Participants who are Black or of African Descent:

					Expected or “Predicted” Distribution of Participants who are Black or of African Descent:

	One of the more surprising findings of this study was the general lack of availability of demographic data. While disclosure of this data is an FDA requirement, only 83% (282) of the drugs approved during the 2007 to 2017 time period had data on participant race readily available for at least one trial referenced in the Medical Review, and only half (171) of the drugs provided ethnicity data on participants for at least one trial (Table 1). When looking at pivotal trials, the availability of race data drops to 73% (551 of 757 pivotal trials) and only 37% of pivotal trials (278) had ethnicity data available. Availability of age and sex data were relatively high for pivotal trials, with 83% and 90% (630 and 679) of pivotal trials having data available, respectively. The availability of data for all demographics plummets dramatically when looking at all trials. Forty-six percent (2,148) of all trials had sex data readily available, and 38% (1,781) had age data available. Availability of data regarding participant race drops to 27% (1,296) of trials, and the availability of data regarding participant ethnicity was even lower-13% (614) of trials (Table 1).

	The extreme lack of ethnicity data speaks to a great need for increased diligence in the collection of these data, while observations made during the data collection process speak to a need for a standardization of the process. Ethnicity data were rarely collected, yet in the instances they were collected, the process frequently varied by study. In many cases race and ethnicity were conflated and treated as the same variable. Even within the CSDD’s study, keeping these variables separate became a challenge as a result of this conflation within the data. When data on participant ethnicity were collected, the ethnicities available were rarely the same. For many studies, the only ethnicities listed were Hispanic or Latinx, and non-Hispanic or Latinx. In rare studies a more extensive list was made available, and this list varied between studies. While an entirely comprehensive list of ethnicities may not be a realistic aspiration, a standardized list of ethnicities that industry researchers can default to may help increase collection of this data, and in turn contribute to an improvement in diversity inclusion in clinical trials.

	Table 2. Disparities by race & ethnicity (pivotal trials only)

	Of the racial and ethnic demographics examined in this study, participants identifying as Black or of African descent were the most underrepresented. Pivotal studies for drugs approved between 2007 and 2017 under enrolled these participants by more than 46,000-an under-enrollment disparity of -65.4% (Table 2). Further, pivotal trials averaged an underrepresentation of greater than 20% for this demographic for 80% (198) of the drugs in the sample, and 86% (19) of the therapeutic areas where demographic data were available had an average underrepresentation of greater than 20% (Table 3). 

	Table 3. Race and ethnicity under-representation in pivotal trials

	Participants identifying as other racial identities (e.g. Native American, Native Alaskan, Native Hawaiian,  or Pacific Islander, etc.) were also significantly underrepresented. Participants with other racial identities were under enrolled in pivotal trials by nearly 12,000 (-46.1%) (Table 2). They were underrepresented in pivotal trials for 76% (157) of drugs approved in the time period, and 76% (16) of therapeutic areas (Table 3). Hispanic or Latinx participants showed a less dramatic, but still significant, underrepresentation, being under enrolled in pivotal trials by nearly 5,000 participants, an under-enrollment disparity of -12.4% (Table 2). Although the overall under-enrollment of Hispanic or Latinx participants was less dramatic, pivotal trials for 62% (87) of the approved drugs which collected ethnicity data under-enrolled participant of this demographic group by greater than 20%, and they were underrepresented in 62% (13) of therapeutic areas (Table 3). When examining all trials together, Asian participants were over enrolled by more than 23,000 participants in pivotal trials, a disparity of +148.9% (Table 2). However, when examining the pivotal trials for individual drugs, Asian participants were underrepresented in pivotal trials for 59% (115) of drugs approved during the time period, and in 46% (10) of therapeutic areas (Table 3). This apparent contradiction stems from the fact that certain drugs recruited large numbers of Asian participants in their pivotal trials. These trials resulted in very high overrepresentation disparities and raised both the total number of Asian participants and the average disparity when examining all trials together, while the majority of individual trials continued to show an underrepresentation for this demographic. Further complicating the analysis of this demographic is the fact that researchers in some countries included participants of Indian descent as Asian, while researchers in other countries did not. This inconsistency may have also contributed to the inflation of the demographic subgroup when looking at all trials together. For comparison, White participants were over-enrolled in pivotal trials by more than 41,000 (+13.6%) (Table 2) and underrepresented in pivotal trials for 9% (22) of drugs, and in 5% (1) of therapeutic areas (Table 3).

	It is clear that the issue of underrepresentation of racial and ethnic minorities is a consistent problem in clinical trials, however there are some areas in particular that show a dramatic lack of representation. 

	Table 4. Underrepresentation in pivotal trials by therapeutic area

	Table 4 lists the therapeutic areas with the highest percentage of drugs that underrepresented demographics by at least 20% within their pivotal trials, as well as the average disparity percentage when examining all of the drugs within the TA (to ensure that therapeutic areas were fairly represented, only those with 10 or more drugs were included). One TA from this list that stands out is Neurology as it was among the top five for each of the racial or ethnic subgroups. Among Neurology drugs, 89% underrepresented participants with other racial identities, 88% underrepresented participants that are Black or of African descent in pivotal trials, 85% underrepresented Hispanic or Latinx participants, and 71% underrepresented Asian participants. Similarly, dermatology was among the top five therapeutic areas for three of the demographic subgroups. Eighty-one percent of the drugs in this TA underrepresented participants who are Black or of African descent, and Asian and participants with other racial identities were both underrepresented by 69% of the drugs in this TA. Cardiology/vascular diseases, gastroenterology, immunology, infections and infectious diseases, oncology, and pulmonary/respiratory diseases were each among the top five for two demographic subgroups.

	Literature from the past two decades has attempted to identify the causes of underrepresentation of demographic subgroups, as well as offer potential solutions to these problems.

The results of this Tufts CSDD study indicate that continued efforts to resolve these barriers is required; however, there are indications that at least some of these efforts are being made. One source of these efforts is the patient engagement movement, which has inspired clinical researchers to improve their efforts to increase diversity in clinical trials.

As a result, many large and mid-sized pharmaceutical and biotechnology companies have publicly committed themselves to improving diversity in clinical trials in recent years.

	While this study goes a long way toward quantifying the problem of racial and ethnic underrepresentation in clinical trials, it is not without limitations, leaving opportunities for further research. First, the availability of demographic data for clinical trials. Although quantifying this problem was one of the goals of this research, the low availability of data indicates that if a more complete sample were collected, the findings may vary. Additionally, even though using the Medical Reviews for each drug as a basis for trials can reasonably be assumed to capture all pivotal trials, it is known that not all trials conducted for a drug are included in the Medical Review. Our conclusions can only be applied to the trials reported and it is likely that many of the drugs on our list have unreported trials. Further, no data were collected on trials conducted for drugs that failed to receive FDA approval, another sizeable set of clinical trials, which may have different disparities. Finally, the use of U.S. Census data in instances where disease prevalence rates were unavailable means that disparity rates cannot necessarily be generalized outside of the US. Demographic makeup varies by country, as do incidence and prevalence rates for many diseases, so future research will require applying regional population census and prevalence data to ensure that results are applicable in countries other than the US. Additional research into these areas, and demographic disparity in general, is required.

, MA*, email: zachary.smith605922@tufts.edu; 

, BA, all with Tufts Center for the Study of Drug Development, Tufts University School of Medicine,

		Pinsky PF, Ford M, Gamito E, Higgins D, Jenkins V, Lamerato L, Tenorio S, Marcus P, Gohegan J. Enrollment of racial and ethnic minorities in the prostate, lung, colorectal and ovarian cancer screening trial. 

		Gadegbeku C, Stillman K, Huffman P, Jackson M, Kusek J, Jamerson K.  Factors associated with the enrollment of African Americans into a clinical trial: results from a study of kidney disease and hypertension. 

		Gifford A, Cunningham W, Heslin K, Andersen R, Nakazono T, Lieu D, Shapiro M, Bozzette S. Participation in research and access to experimental treatments by HIV-infected patients.  

		Zuckerman R, Getz K, Kaitin K.  A new mechanism for tracking publicly available study volunteer demographics.  

		Salmon A, Nguyen C, Lee Y, Cooksey-James T.  A review of barriers to minorities participation in cancer clinical trials: implications future cancer research.  J Immigr. Min. Health. 2016; 18: 447-453.	

		Ford M, Siminoff L, Pickelsimer E, Mainous A, Smith D, Diaz V, Soderstrom L, Jefferson M, Tilley B.  Unequal burden of disease, unequal participation in clinical trials: Solutions from African American and Latino community members. 

		Kaitin K, ed. New benchmarks on participant diversity show high level of demographic under-representation. Tufts CSDD Impact Report. May/June 2020.

Diversity in clinical trials is an important part of developing new medications that are safe and effective for all potential patients, understanding the demographic disparities within them will provide several benefits to the field. 

Demographic Disparities in Patient Samples for Drugs and Biologics Approved by FDA Between 2007-2017

	In 2018, global drug development companies spent approximately $375 million to perform 25,000 qualification

 assessments of contract research organizations (CROs) and other service and technology solutions providers. These estimates are based on the results of a new study conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) in collaboration with The Avoca Group. The results highlight a demanding area that has often been discounted as simply a cost of doing business but for which there are process improvements and efficiencies to be gained.

	The study benchmarked the vendor qualification assessment (VQA) process and found that the process-from a request for information (RFI) through the vendor assessment questionnaire and audit-is complex-, labor- and time-intensive, with wide variation observed within and between companies. A high percentage of assessments-most notably those conducted by larger sponsor companies-involve customized assessment areas, making it difficult to streamline the process and to execute consistently. 

	The assessment process today is elaborate. It covers a wide range of areas, including regulatory and ethical compliance; technology expertise; quality management practices; data privacy, confidentiality and security; and operational and financial health, controls, and oversight. As demand for vendor services to support drug development activity continues to grow, the assessment qualification process represents an untapped area to apply new practices and solutions-such as standardization and risk-based approaches-to drive speed and efficiency and lower cost. 

	Seventy-eight distinct pharmaceutical and biotechnology companies and 31 distinct CROs completed an in-depth online survey during the summer and fall of 2019. The survey looked at several primary areas: (1) the volume and types of VQAs conducted each year; (2) direct and indirect costs associated with the assessment process; and (3) the time required to perform each assessment. Two primary subgroups were evaluated: sponsor vs. CRO differences and company size 

differences. Small companies were defined as those with annual R&D budgets below $250 million; medium-sized companies had $250 million to $2 billion annual R&D budgets; and large companies had annual R&D budgets greater than $2 billion.

	The typical VQA consists of an average of 61 primary questions across 24 broad areas of inquiry (See Table 1). The most frequently assessed areas include the vendor’s data privacy protection provisions (94% of companies); quality management system (93%); document management and control (89%); and 21 CFR part 11 system compliance (87%). The least common areas assessed include product communications (25.5%); human resources management (23.6); and fair trade (23.6%).

	Most new vendor assessments are conducted in person, whereas most vendor requalifications are conducted remotely. Companies that responded to the survey indicate that nearly two-thirds (62%) of questions in the qualification process are routinely or commonly assessed. Approximately one-third of questions are customized to new areas or capabilities-of-interest (e.g., a new biomarker or diagnostic test; a new data analytic capability; a new patient-centric service). Large companies qualify and verify many more areas than do small and medium-sized companies. And large companies in particular report a very high level of customization, with only 50% of areas of inquiry commonly assessed.

	In 2018, large companies each completed an average of 86 vendor qualification assessments; medium-sized and small companies completed 28 and 12 qualification assessments, respectively. From the time the RFI is sent to the vendor to when the contract is signed, sponsor companies, overall, take 19 weeks-or nearly five months-on average to complete the vendor qualification process for a new single-service vendor. It takes sponsor companies nearly 27 weeks (almost seven months) to complete the vendor qualification process for a new vendor that is providing multiple services.

	CROs are significantly faster, cutting the overall cycle time by 35%-45%. No significant differences in cycle time or cost were observed between companies that use a centralized vs. a decentralized vendor qualification approach. Small companies have longer relative RFI completion times but shorter timelines assessing new vendor qualifications culminating in a contract signature. Large companies have relatively short RFI-initiation-to-VQA-completion cycle times but significantly longer time periods to execute the vendor contract.

	For all sponsor companies, vendor requalifications are two-to-four-weeks faster on average than qualifying new single- and multi-service vendors, respectively (see Table 2). Mid-sized companies report the most substantial time savings between new vendor qualifications and requalifications.

	Large companies report dedicating an average of 27.7 full-time equivalents (FTEs) to perform VQAs in 2018. Medium-sized and small companies dedicated an average of 11.1 and 3.5 FTEs, respectively. The typical sponsor company completes an average of 2.5 to 3.5 new vendor qualifications per FTE annually. CROs are significantly more productive. Compared to sponsor companies, CROs completed 2.5 times the number of new vendor qualifications per FTE.

	Large companies each spend on average $2.5 million annually in direct and indirect costs to complete vendor qualifications and requalifications. Mid-sized companies each spend about $600,000 and small companies spend approximately $180,000 annually to perform vendor qualifications and requalifications.

	More than 90% of assessments-including those requiring some remediation-result in a vendor being qualified to work with the sponsor. Approximately 20% of new vendor assessments identify a vendor deficiency and require remediation (e.g., modifying or correcting a capability or operating practice) before the vendor can be qualified.  Despite longer cycle times and more elaborate and demanding assessments, large sponsor companies have the highest rate of vendors failing to qualify (11.5%).

	Companies are investing substantial time, personnel, and resources to complete vendor qualifications and requalifications each year. Despite familiarity with existing vendors, requalification cycle times are only marginally faster-by only two to four weeks on average. Growing functional requirements and demanding procurement practices are likely contributing factors here.

	Wide variation around mean cycle times and workload were observed throughout the study. Some of the variation can be explained by the diversity of the vendor companies now requiring qualification. A growing community of technology and application solution providers, specialty labs, and service providers offering enhanced convenience to study volunteers (e.g., participant navigators and assistants; transportation support; decentralized clinical trial support) may be new to the clinical research industry and are unfamiliar to the sponsor company performing the assessment. These providers no doubt require more time and resources to qualify.

	Wide variation signals opportunity to isolate and uncover factors contributing to best and worst outcomes. Some of the variance can be explained by the absence of standardized qualification practices and the prevalence of custom areas of inquiry. More effective and robust internal data sharing and precompetitive information sharing on routinely-assessed and new vendor qualification areas may play a part in helping to reduce cycle times and their variability. Risk-based approaches to reduce and streamline select qualification areas of inquiry may also contribute to faster timelines and efficiency.

	Large companies have the longest overall vendor qualification cycle times. They generally enjoy a speed advantage over small and medium-sized companies in completing the RFI and VQA processes, but subsequently lose that advantage to significantly longer contract negotiation and execution. Large company contracting practices appear to be an important improvement area warranting attention.

	Although CROs report dedicating more hours to perform each RFI and VQA, process cycle times are significantly faster and more resource-efficient in qualifying vendors than are their sponsor company counterparts. This may in part be due to the urgency that CROs place on meeting clients’ tight project timelines and budget requirements and on achieving rigid corporate operating profit targets.

	The vendor qualification and requalification processes take far longer than expected at substantial personnel and resource expense. As sponsors and CROs look for ways to accelerate drug development timelines and lower development costs, these processes hold new and untapped opportunities to qualify and verify vendors faster and more effectively.

Study holds new benchmarks and improvement opportunities.

Author | Kenneth A. Getz