Gen Li

	It is common, and many would assume logical: if the enrollment performance data from an investigator site is known, particularly if there is a lot of this data, one should be able to predict future enrollment performance by extrapolating from this data.  Phesi analysis of over 330,000 trial protocols held in our global database shows this approach is flawed.

	An investigator site’s ability to enroll patients is dependent on the protocol design. For instance, a sponsor designed 2 multiple sclerosis trials; one targeted female patients only and the other included both male and female patients. Note that 75% of multiple sclerosis patients are female, so we assume that the sponsor thought that with enough sites, there would be no issue enrolling the female-only trial. This trial encountered so many challenges, that the protocol eventually had to be amended to include both male and female patients. The damage was done. It took almost twice as long to enroll fewer patients for the female-only study. The Adjusted Site Enrollment Rate (ASER) was 0.33 patients/site/month, compared to the second trial, which included both men and women from the outset and enrolled 1.44 patients/site/month. 

	An investigator site’s ability to enroll patients also depends on how well sponsors and their chosen CRO activates them. Sites activated toward the end of the enrollment period, which is typical during a ‘rescue’ situation, are less likely to emerge as top enrollers. While trial rescues, including the unplanned addition of sites, creation of protocol amendments, and running recruitment campaigns, is painfully common in clinical research, it is very costly, and hurts the relationship between investigator sites, patients, and clinical development organizations globally. It is important to point out that some of the best performing sites consciously avoid being involved in such trial rescues.  The important point is that today there are structured predictive analytics approaches to minimize, or even eliminate disruptive, frustrating, and costly “rescue missions”.

	Another key consideration is that enrollment performance of investigator sites from different countries can be markedly different. An experienced and consistently high performing investigator site in the USA might only enroll 1/3 or less patients than their counterpart, say in Poland, in the same trial. The complication of country allocation and the level of sophistication needed to solve that problem, is worthy of discussion in a separate blog. Simply to say that it is risky to compare enrollment data between investigator sites from different countries. 

	It is also risky to think that historical site enrollment results are directly correlated with patient data quality levels. An investigator site can enroll a large number of patients, but the site’s ability to provide reliable data for the sponsor to evaluate the safety and efficacy profile of a clinical development candidate, are two different things.

	Make no mistake, historical investigator site level enrollment data are valuable, but only as one piece of a complicated puzzle. Investigator site enrollment performance is dynamic, and there are so many moving parts (variables) attached to it. What we have described here are only a few examples of these many variables. The efforts of AI and analytics companies over the past decade have not only begun to quantify these variables, but also have been revealing the complex relationships among this very large number of variables. Complex algorithms and increasingly large sources and amount of data allow us to better understand the interrelationships and more accurately model the design of the development plan and the protocols and the selection of the investigator sites and make accurate predictions of site performance. Thus, we can now avoid the error prone approach of extrapolation. 

	Gen Li, Lauri Sirabella, 2010. "Planning the Right Number of Investigative Sites for a Clinical Trial." 

Articles

Analysis of over 330,000 Phesi trial protocols shows predicting future enrollment performance by extrapolating data is flawed.

To Extrapolate Site Enrollment or Not To Extrapolate? That is the Question.

	Investigator site enrollment performance is probably the most misunderstood concept in clinical trial planning and execution. A trial’s success depends on enrolling the planned number of patients in the shortest possible time frame. That imperative puts pressure on each participating site to recruit more patients, focusing attention on the number of patients enrolled per investigator site as a critical factor of success.

	Nevertheless, some investigator sites will recruit more patients than other sites. That tendency can prompt trial planners to set enrollment targets based on linear extrapolation - which is still the dominant thinking in our industry - as a means to identify and deploy the sites capable of enrolling the largest numbers of patients. Such thinking can spur a narrow focus on obtaining and analyzing all investigator site enrollment data, as if that would solve all problems pertaining to site enrollment performance.

	Despite the dubious allure of enrollment data, many large pharmaceutical companies conduct a large number of trials, and naturally accumulate data from more than 100,000 investigator sites. Not to be outdone, the largest CROs have also accumulated a very large amount of enrollment data points from investigator sites, but do not necessarily share operational details with their sponsors. One would imagine these organizations would have leveraged the data to improve their investigator site enrollment performance. Phesi analyses show that is not the case.

	Simply put, the number of patients enrolled per site is a complicated metric. Investigator site enrollment performance is, mathematically, only a component of this metric.

	To understand the limited utility of individual site enrollment rates as a performance metric, it is helpful to consider the concept of “trial rescue,” or its more crude synonym, “disaster recovery.“ A common practice in clinical development, trial rescue is usually required when the originally deployed investigator sites were not able to deliver the required number of patients within the planned time frame.

	Let us look at a trial with a 15-month enrollment cycle time, with two sites of equal quality. One site was activated in Month One and the other was activated as a part of rescue effort in Month Ten. Obviously, we cannot expect the second investigator site to enroll the same number of patients as the first. That difference is not caused by the gap in enrollment performance potential between the two sites, but by the process by which the two sites were deployed/activated.

	Trial design, as every veteran of clinical trials understands, is one of the most important factors in the ability of an investigator site to enroll qualified patients.

	Let’s use a Phase 2 non-small cell lung cancer (NSCLC) trial as an example. NSCLC trials typically incorporate ECOG Performance Status (PS) as an eligibility criterion. The modal value, a statistical term used in Phesi’s proprietary protocol design method, is an ECOG PS value of 0 or 1, indicative of good performance with little or no restriction. When a Phase 2 NSCLC trial is designed to only include patients with ECOG PS 2 – a grade assigned to individuals who are ambulatory and capable of self-care but unable to carry out work activities -- an investigator site will be able to enroll only a small fraction of the required number of patients, given that this grade applies to only about 15% of patients with NSCLC. By contrast, a trial designed with an inclusion modal value of ECOG PS 0 or 1 will have a better chance of meeting the enrollment target.

	Competition within investigator sites is another factor that frequently can impact the number of patients enrolled per site. For two equal-quality investigator sites, the one with multiple trials targeting the same patient population (or a similar population) will likely enroll fewer patients than the site that is not competing with itself for participants.

	To faithfully predict investigator site enrollment performance, we not only need to understand the factors described above, but to build and deploy quantitative models that enable the extraction and processing of multiple parameters from various sources (i.e., “big data”). Moreover, the models must incorporate a self- enhancement mechanism that improves their predictive power as fresh data come in (i.e., machine learning or artificial intelligence), thereby providing constantly updated predictive results and improving our confidence in the data. Phesi has developed such a powerful platform, one that has consistently provided our clients with superior enrollment results in hundreds of clinical trials over the past decade.

	Our experience has taught us that enrollment data for individual investigator sites have only limited utility. One should therefore be suspicious of site selection vendors who claim to have the largest cache of individual site enrollment data, as they do not necessarily deliver better site selection results.

A trial’s success depends on enrolling the planned number of patients in the shortest possible time frame. Such thinking can insight a limited focus on obtaining and analyzing all investigator site enrollment data - as if that would solve all problems pertaining to site enrollment performance, writes Gen Li, President of Phesi.

Gen Li

Articles

To Extrapolate Site Enrollment or Not To Extrapolate? That is the Question.

Predicting Investigator Site Enrollment Performance: Separating Myth from Truth