Project Manager and Data Scientist, Tufts CSDD

Zak Smith

Decades of aggressive timelines, along with demanding and complex drug development activity has facilitated the use of highly structured, sequential, repetitive, and standardized behaviors and practices meant to drive efficiency and speed. Many practices and procedures are intended to protect the safe and ethical treatment of study volunteers and ensure the integrity of the research. But sponsors, CROs, and investigative sites also point to the use of extensive conventional approaches and standardized operating practices to streamline activities, save time, and plan and predict executional outcomes. Throughout this article, we refer to these collectively as ‘linear practices’ in that they follow a sequential order or progression—determined by previous actions and conditions—that rarely deviates.

Anecdotal reports from industry insiders suggest that linear practices may be having the opposite effect of what is intended, resulting in more delays and inefficiencies. As one example, our research has consistently shown that operational planning practices—such as calculating enrollment rates, study timelines and study cost estimates—have been steadily losing their accuracy and effectiveness. Our studies, and others, have demonstrated that as clinical trials have become more complex over time, the difference between actual and planned timelines has widened and cycle times have gotten longer, with greater observed variation around the mean durations.

 In response, sponsors have implemented a larger number of corrective measures, including more amendments per protocol

At this time, given the unprecedented complexity of clinical trials, many sponsor companies are hoping to leverage practical, evidence-based insights to inform more flexible and agile approaches than historical—and ineffective—linear practices.

In this article and others to follow, Phesi and Tufts CSDD share the results of collaborative research focusing on select linear operating practices that can and should be improved with better planning and real-time data. Here we look at a common linear practice in investigative site management: planning the number of Phase III sites required based on Phase II patient enrollment experience. When a Phase II trial is successfully completed, Phase III trials are planned to confirm and build on the Phase II experience. To achieve the same enrollment rate and meet the study timelines, the number of investigative sites will be proportionally increased according to the enrollment experience in Phase II.

In this analysis, we have gathered a sample of clinical trial pairs: a primary Phase II trial with its companion Phase III pivotal or supportive trial. Trials were paired based on a combination of factors and elements including pivotal trial status, indication, primary endpoints, dates conducted, and other select trial design features. Table 1 below provides the NCT numbers, durations (trial start to primary completion date, in months), and therapeutic areas of the associated pairs of clinical trials. Trial start date and primary completion date were taken from clinicaltrials.gov.

Eleven trial pairings were selected from 9 drugs approved by the FDA between 2018 and 2020. Trials included in our sample had start dates between 2008 and 2016, and primary completion dates between 2010 and 2020. Three trial pairings (27%) were oncologics (anti-neoplastic), 2 trial pairings (18%) were immunologics, and the remaining pairs (55%) were from a variety of CNS, gastrointestinal, and cardiovascular disease conditions.

Gross Site Enrollment Rate (GSER) was selected as the key metric for this analysis. GSER is defined as the number of patients per site per month.

 Typically, the number of sites in the Phase III trial is directly and linearly scaled from the Phase II trial in order to increase the number of participants enrolled. Estimated timelines are based on the assumption that the GSER will be the same in Phase III as it was in Phase II. When this linear practice results in a lower-than-expected GSER, the estimated timelines cannot be met, the clinical trial is delayed and often unplanned and unbudgeted remedial steps are taken.

 Trial Durations, in Months (Trial Start – Primary Completion Date)

Table 2 below provides the number of sites engaged and patient enrollment experience for each clinical trial in our sample. Table 3, located below Table 2, provides the GSER—drawn from the Phesi Trial Accelerator™ platform—for each trial.

The results show that the proportional increase in the number of investigative sites does not yield a proportional increase in patients enrolled; indeed, it often results in a lower enrollment rate and longer enrollment cycle times. In these paired examples, when the Phase III trial is larger in scale as measured by the number of investigative sites and the number of patients recruited, the enrollment rates (GSER) are frequently lower.

Gross Site Enrollment Rates are impacted by many factors. For any given multi-site clinical trial, investigative sites are engaged and activated over a period of time. The more sites engaged and activated within a shorter period of time, the better the utilization of the enrollment capacity for allsites participating.

Phesi has measured the utilization of investigative site enrollment capacity and calculated a Site Effective Index (SEI).

 SEI essentially measures the percentage of sites activated, and when, during the enrollment period. When all sites are activated on Day 1, the SEI is 100%. Hypothetically, were all sites to be activated on the last day of the enrollment period, the SEI would be 0%. Phesi research shows that the industry SEI average is approximately 50%.

Whereas adding investigative sites to any given clinical trial may initially improve enrollment performance, there is a diminishing return on the number of sites engaged due to competition for patients, the declining availability of targeted patients per subsequent site activated, and lower relative site capabilities and effectiveness. Over time, the larger the size of the clinical trial as defined by the number of investigative sites, the lower the proportion of high performing investigative sites is engaged.

The ultimate consequence of linear practices associated with site identification and activation is the delay in achieving target enrollment. Looking at five trials in our sample where there is sufficient information, these delays range from 5 to 21 months.

Simulation to inform Phase III planning would go far in addressing the challenges introduced by linear site management practices. Based on actual data from a cohort of Phase II trials with similar design features and disease conditions, a baseline GSER scenario can be established. Next, various “what if” trial design and execution scenarios can be conducted to accurately simulate the impact of adding or subtracting investigative sites. Figure 1 below shows the application of a simulation exercise. Each sphere represents a clinical trial with the size of each sphere corresponding to each individual trial’s enrollment rate (e.g., the larger the sphere the higher the enrollment rates).

Based on this simulation, the Phase III trial in this model would have benefited from involving more investigative sites resulting in a shorter enrollment duration. The ability to objectively visualize scenarios helps to optimize site management and patient enrollment.

There are a number of other approaches that sponsors and CROs have historically implemented to address the effects of linear site management and patient recruitment practices: Some sponsors and CROs, for example, comprehensively identify and activate top performing investigative sites to mitigate diminishing GSER return. With the average estimated cost to activate and monitor an investigative site at $40,000 - $50,000, the marginal cost of activating a larger number of Phase III sites will be offset by the gains in the speed to achieving target enrollment. At the present time, given the availability and granularity of clinical trial performance data informing site and recruitment management, simulation is likely more cost-effective.

Another approach that select sponsors and CROs are implementing involves more active management and support of investigative sites to ensure higher GSER levels. This support may include providing recruitment assistance, funding and training for dedicated recruitment specialists. The deployment of virtual and remote clinical trial solutions (i.e., DCT)—improving study volunteer access and participation convenience—may also play a role in driving improvement in GSER. At this time, sufficient evidence to evaluate the impact of these approaches is not available.

Linear practices in the identification and activation of investigative sites have resulted in clinical trial delays and it is likely that these practices contribute to clinical team decisions to terminate the development of promising new treatments. Rich operating data and analytics can play a critical role in avoiding linear practices and in delivering better clinical trial performance outcomes.

, executive director and research professor, Tufts CSDD, 

, project manager and data scientist, Tufts CSDD, 

Getz K, Campos R. Trends in clinical trial design complexity. 

Getz K, Stergiopoulos S, Short M et al. The impact of protocol amendments on clinical trial performance and cost. 

Getz K, Smith Z, Jain A, Krauss R. Benchmarking protocol deviations and their variation by major disease categories. 

. 2022; doi.org/10.1007/s43441-022-00401-4.

Li G. Forecast Enrollment Rate in Clinical Trials. 

https://www.pharmexec.com/view/finding-sweet-spot

Spinner J. Sponsors must rethink ‘primitive’ clinical trial approaches: Phesi. 

https://www.outsourcing-pharma.com/Article/2021/05/04/Sponsors-must-rethink-primitive-trial-approach-Phesi#

Articles

Phesi and Tufts CSDD share results from research on operating practices that can be improved. 

Zak Smith

Articles

Leveraging Data Insights to Address the Perils of Linear Practices in Site Identification and Activation