Vadim Paluy, MD

Site selection is one of the most important and at the same time challenging problems in clinical trials planning. Poor site selection may cause enrollment delays, resource waste on low or zero enrollment, and even potentially compromise trial results.

	Site selection is a complex process, which includes sites identification, sites assessment, sites validation. and selection of a set of sites aligned with study goals and limited resources.

	Many factors affect the site selection process. Some of them are generic, others are trial specific (Figure 1). Some factors are qualitative and subjective, such as “Experience and qualifications of Principal Investigator” or “Staff turnover.” The value of each factor can be evaluated qualitatively (e.g., excellent – 10, the worst – 1). Or it can be quantitative, such as “Planned patients’ enrollment” or “Projected enrollment rate.” The factors could be extracted from historical databases, questionnaires, etc. Some companies use data mining techniques for site identification. Examples of clinical trial-related factors are “Trial budget,” “Enrollment target,” etc. A variety of business rules are also related to the entire trial, e.g., “If a country is selected, then minimum two sites per country should be selected,” or “Priority site”-forced site prioritization.  

	Traditionally, site selection efforts are focused on a selection of “best” clinical sites based on:

		Scoring of multiple site attributes (e.g., experience of principal investigator (PI) and staff, historical patient enrollment rates, overall facility quality, etc.) and ranking index calculation as a sum of weighted scores.

		Handpicked approach to sites selection based on prior relations with sites and sites’ qualitative evaluation.  

	Site rankings according to a criteria has limitations. For example, site selection based on sorting of predicted enrollment rates does not take into account parameters like cost per patient, site capacity, or “soft” attributes such as experience of PI, facility quality, etc. The value of site selection based on extrapolation of historical performance may be limited due to high site turnover [Getz, 2018] or a limited or lack of historical data for new sites. Site selection based on sorting of ranking index also does not take into account budgeting and other constraints, as well as study goals and business rules.

	Often, the site selection process mistakenly identifies with only site identification and feasibility assessment. After identifying the “best” sites and evaluating them, it is assumed that the best sites will be selected somehow from a feasible set of “best” sites.  

	Often, a feasible set of sites is identified based on an informal process which may include, but not be limited to:

	More advanced techniques (AI, data mining) could be applied for large sites’ databases but not transferrable to all sites.

	Usually, all feasible sites are divided into three major tiers. Their advantages and disadvantages are presented in Table 1.

	Sites could “migrate” from tier to tier depending on most recent performance assessments.

	A high-level overview of the traditional site selection process is presented in Figure 2 and described below.

		From sites database (A) (it could be proprietary or commercial), a feasible portfolio of sites is defined based on certain attributes (e.g. “A list of doctors treating multiple sclerosis in USA”).

		Then, all feasible sites are assessed based on scoring

 of each site attribute and its weight (B). A list of attributes may include: (1) Experience of PI and site employees; (2) geographic location; (3) transportation to the site; (4) enrollment risk; and many others. Each attribute is scored (e.g., from 1 to 10) and weighted (in %). A list of attributes can be trial-specific. Scores and weights could be obtained from questionnaires, previous experience, historical data, etc. Small companies usually rely on questionnaires. Then, site value is calculated as follows:

	After site evaluation, all feasible sites are ranked according to their value

		As mentioned, ranked sites are divided into three tiers. First, sites from Tier 1 are selected, then from Tier 2, and if there are not enough patients or other goals have not been met, from Tier 3 - (C). Then, preselected portfolio of sites needs to be aligned manually with available budget, enrollment target, sites capacity, number of patients, and other goals and rules. If site selection meets study goals, site selection is finalized (F). If not, but other feasible sites are available (D), they need to be included in the selection process (E) with subsequent alignment to study goals and available resources (C), etc.

		If number of feasible sites is not sufficient (D), a new search in sites’ databases needs to be performed (A).

		It is assumed that after multiple iterations, site selection is finalized (F).

	Let’s consider an example illustrating the site selection approach based on sites’ rankings.

Case study: Contingency planning for Phase III CNS global clinical trial

	For a Phase III trial, an initial set of sites was already selected. However, some sites canceled their participation due to various reasons. Therefore, in order to meet study goals (enrollment rate, number of patients, and limited budget), there was a need to select additional sites. Twenty five new feasible sites were identified.

	Site selection needed to meet the following goals:

		Sites evaluation. Multiple site attributes were identified, as presented in Table 2. Some of them may be unique for a trial. Total site value is calculated according to the formula (1).

		Sites are ranked according to their value (or other attributes such as predicted enrollment rate could be used as well). Data used in the ranking and the ranking results are presented in Table 3.

	Sites parameters, such as number of randomized patients, projected enrollment rate, and patient-related costs, are accumulating until their values meet or exceed study goals. For example, cumulative number of patients, cumulative enrollment rate, and cumulative site costs are obtained by adding site #2 data to the site #6 data (Table 3). It means that cumulative number of patients = 10 (site #6) + 10 (site #2) = 20. The same applies to projected enrollment rate for sites #2 and #6 = 0.52 (site #6) + 0.24 (site #2) = 0.76, etc.

	Unfortunately, often study goals cannot be met simultaneously. For example, sites-related budget was met by adding site #4, but enrollment target goal was not met. Therefore, according to the ranking algorithm, sites #10 and #11 have to be added to meet the enrollment target goal. It means that budget was exceeded by $147,500, or by 12%.

	If sites are ranked according to their enrollment rate, budget has to be increased by 8%.

	This process is time- and labor-consuming and does not guarantee optimal site selection. Adding more sites does not solve the problem, because it is associated with increased cost of a clinical trial beyond the budget and inclusion of riskier sites into a pool of feasible sites.

	Is there a better site selection solution aligned with study goals and within the budget?

	This paper presents a portfolio approach to site selection similar to selection of financial portfolios or portfolio of projects. For site selection, this approach was formulated in [1].

	Portfolio approach to site selection means that instead of selecting individual sites, clinical trial planners need to select a portfolio of sites based on advanced analytical models, where the goal of site selection is to maximize the overall value of a portfolio of sites, and to align it with clinical trial goals and limited resources. As shown in [2], the most effective approach to portfolio selection is based on the mathematical optimization model. The model replaces the loop, including steps B (except sites evaluation), C, D, E, with advanced modeling algorithms, automating site selection aligned with study goals and resources.

	In the context of decision-making, optimization means determining the most favorable solution, outcome, or course of action from a set of alternatives that satisfies all constraints and dependencies based on the mathematical optimization model.

	In order to optimize site selection, an optimization model was developed. The model was formulated as a mixed integer programming (MIP) model. The MIP models are a subset of the linear programming (LP) [3] models, where some variables are binary (0, 1). LP models find the globally optimal value (e.g., total value of selected sites) of a linear function of a certain number of variables

 given a set of linear constraints on these variables (equalities or inequalities).

	The model for optimal site selection includes four components:

 = (0 or 1). Their value will be defined automatically. If X


	Estimates for each site, such as cost/patient, projected enrollment rate, and others.

		Business rules (forced site prioritization, minimum number of sites per country (if a country is selected), requirements for patients’ allocation across regions, and others).

		Study goals-number of patients to be enrolled (trial power), enrollment target, etc.

		Other rules relevant for particular clinical trial.


	Single (e.g., maximum value of sites’ portfolios), or multiple criteria (maximum enrollment target, minimum budget, etc.) can be used.

	The model (Site Selection Optimizer) uses the same data as presented in Tables 2 and 3. The optimization algorithm found a better solution than the one based on the ranking algorithm. It automatically selects a portfolio of sites aligned with study goals and budget (Table 4). At the same time, in order to reach study goals, ranking algorithm requires ~12% bigger budget and more sites (17 - ranking vs. 16 - optimization).

	Selected portfolios of sites using ranking vs. optimization (baseline scenario) are presented in Table 5.

	Optimization results may not look intuitive. For example, site #2, despite its high score/value, was not selected due to high costs/patient, high number of patients, and total costs/site.

 Also, site #2 was not selected because of a “knapsack effect.” That means that it’s harder “to pack” a large item (site capacity = 10pt) vs. several small “items” (most sites capacity =5 pts). Sites #9 and #16 were selected in the optimization model despite their relatively low score, because cost/patient is low and enrollment is high enough.

Articles

Optimization model evaluates the benefits of selecting a portfolio of investigative sites based on advanced analytical models.

Portfolio Approach to Optimize Site Selection

	The model validation table is presented in Table 6.