Molly Blake-Michaels

It's well recognized that lengthy and expensive clinical trials, high failure rates, and increased patient safety concerns collectively make the economics of drug development more challenging than ever before. Part of the remedy is likely to be a heterogeneous group of protocol designs, collectively termed "adaptive clinical trials." The common thread between all adaptive trial designs is the use of accumulating data to make modifications while the trial is still underway, increasing the likelihood of selecting the right dose (or endpoint) for the right patient population earlier in a development program.   

Adaptive trials are adaptive by design, and they require a wholesale change in the way drug development has traditionally been carried out.

 The entire development of a therapeutic candidate, not just the next trial phase, needs to be mapped out in advance. More, and earlier, operational planning is a prerequisite to conducting adaptive trials, which may require a major culture shift at some companies. 

Just as the technology enabling the adaptive trials themselves has come of age, so too will organizations need to move beyond traditional tools and approaches for the operational planning, forecasting, and budgeting of adaptive trials. The planning spreadsheet, a tool commonly used to estimate budget needs for clinical trials, does not provide the flexibility or granularity to quickly and accurately model and compare the many potential scenarios for an adaptive study. New, more sophisticated approaches are needed. 

The rationale for conducting adaptive trials is that they can increase the probability of treatment success, identify ineffective drugs sooner, and require fewer patients, or fewer patients at less informative doses—all without compromising the trials' validity and integrity. They utilize learn-and-confirm concepts, whereby results get substantiated in an ongoing manner rather than waiting for a phase to complete. Examples of adaptations that might be made midstudy are changes in eligibility criteria, sample size, randomization, treatment regimens, and even the primary endpoint.

Adaptive trials require the use of prospective statistical simulation to optimize the trial design, and the criteria for making adaptations (modifications) during the study must be defined prior to the study start. 

In order to control for potential bias once an adaptive study is underway, an independent Data Monitoring Committee (DMC) will usually be established to review planned or pending trial adaptations, including verifying their face validity and making recommendations concerning further conduct of the trial, in addition to its traditional duties.  

A separate independent body, with expertise in statistics, also needs to be assembled to prepare data, conduct interim analyses of unblinded data, and prepare briefing documents for discussion with the DMC. The current randomization algorithm "in play" must be known only to unblinded personnel, and any sponsor involvement on the DMC requires careful consideration due to the increased potential for bias. 

Adaptive trials will often be more expensive than similarly sized conventional studies, but the long-term savings over the course of a development program can be substantial. Savings can come from avoidance of future studies that are no longer necessary, reduced lag time between phases, and the need for fewer subjects. Ultimately, an adaptive trial may result in a product getting registered sooner. 

Importantly, an adaptive clinical trial is not always necessary, and may not be viable. In some cases, enough is known about the patient population and the investigational drug to make an adaptive design unnecessary. In others, outcomes that can trigger an adaptive response won't be observable early enough in treatment in order to "adapt."

From adaptive trials that have been successfully completed to date, a general guideline has become apparent: The recruitment duration needs to be at least four times longer than the observational period in order to preserve the opportunity to make adaptations and prevent the need to pull back on enrollment. If the observable endpoint is too far out, it's not worth trying to do an adaptive trial unless a biomarker or other predictor of the outcome is available that shortens the time frame.  

Upfront planning and strong project management are critical to the effective execution of an adaptive trial because, relative to a traditional trial, data needs to be more quickly captured and processed. New players need to get pulled into the planning process, and input from the functional areas usually involved, such as biometrics, clinical research, data management, drug supply, and clinical operations, is required earlier. 

A substantial amount of time and effort must be devoted to performing simulations of patient response and preparing the simulation report that summarizes the study operational characteristics, in addition to the complex logistics involved in setting up the appropriate IT infrastructure (e.g., EDC, IVRS, algorithm host), which is critical to ensuring that data are available and adaptations made as efficiently as possible. 

Additional upfront time is also needed for coordinating randomization and drug inventory. Creative dosing and packaging strategies are often necessary to ensure that the available drug supply is sufficient for the study. For example, for studies involving a large range of multiple doses, supplies may be used more efficiently by adding pills in lieu of creating more pill strengths.

Planning should begin at the drug development plan stage, long before the study actually occurs. Regulatory authorities are receptive to adaptive trial designs, but must be consulted early in the development of a product to ensure that the methods utilized are appropriate.  

Figure 1. High-level comparison of timelines for 12 different potential study scenarios. 

For sponsors, obtaining internal buy-in may be the tougher task. Adaptive studies require significantly more effort in the planning stage, and will generally cost more than a similarly sized conventional study. In addition, successful implementation also requires specialized knowledge that many companies do not yet have in-house, which means that outside consultants and vendors must be identified and hired. Finally, although the need for innovative approaches to increase drug development efficiency is recognized by all, companies have successfully obtained marketing approval for drugs for decades using conventional study designs. 

Until official regulatory guidance is available, and the companies at the forefront of using this approach begin to reap its benefits, many sponsors may choose to stick with the tried and true methods of drug development, in spite of the advantages that adaptive designs may offer. 

For conventional studies, single point estimates are usually developed for the budget, headcount, and timeline—for example, the study will cost $15.2 million, require "X" number of personnel, and the final report will be signed off on December 15, 2011. However, adaptive trials require a flexible, adaptive planning approach to forecasting and budgeting. Because of their very nature, single point estimates that represent the most likely budget, headcount, and timeline must always be associated with a range of possibilities when prepared and submitted for review and approval. Of primary importance for planning, then, is the ability to easily model and compare multiple operational scenarios based on the predefined adaptation "checkpoints" built into the trial design, as well as the numerous potential branches resulting from the adaptations made at those checkpoints.  

Figure 2. CRF generation by month for study scenarios 1 and 11. 

In order to adequately plan, clinical operations will need to simulate various scenarios to fully characterize the potential outcomes of the study in terms of budget, headcount, and timing. This is similar in principle to the simulations of patient responses conducted by biostatistics and the simulation of potential scenarios for drug needs across study sites performed by the drug supply department, although the tools used are different. 

The scenarios evaluated by clinical operations aren't developed in a vacuum; they require close collaboration with the rest of the project team. Based on modeling of patient responses, a minimum and maximum number of potential patients to be enrolled can be identified, as well as potential enrollment periods and treatment periods. It isn't feasible (or necessary) to evaluate every potential scenario for the study, but the scenarios considered should cover the range of possibilities, and particular attention should be paid to their impact on drug supply requirements. 

This kind of planning represents a paradigm shift for many in clinical operations. As stated previously, the most commonly used planning tool, the planning spreadsheet, does not provide the flexibility needed to easily model and compare many different adaptive scenarios based on differing planning assumptions. Planning spreadsheets usually provide high-level cost estimates based on benchmark data from previously conducted trials (this technique is also called top-down benchmarking). However, because adaptive trials differ significantly from one trial to another, benchmark data are often not available and, if available, are prone to error due to undetected differences in underlying assumptions between studies. These errors apply to traditional studies as well, however, the impact is even more pronounced with adaptive studies. 

In addition, most benchmark data are based on outsourced studies and do not take into account internal resource requirements at all, which means that estimates for internal headcount needs for activities that aren't outsourced can't be generated.  

Figure 3. CRA resource needs by month for study scenarios 1 and 11. 

 Activity-based costing (ABC) is a planning methodology widely used in other industries with highly complex projects, such as construction and manufacturing, and which is gaining currency in drug development due to its benefits in planning adaptive, as well as traditional, trials. Although a full discussion of this technique is outside the scope of this article, a brief overview helps shed light on its applicability to the operational planning of adaptive trials. 

ABC provides estimates for the required budget and headcount based on the work breakdown structure (WBS) for the project, in other words, all of the work to be performed on the study.

 The work to be performed, the types of resources who will perform it (and when), and the cost of those resources are determined, and the budget and resource estimates are built from the ground up. 

Creating the WBS is the most time-consuming part of the process, and requires a thorough understanding of the work to be performed; however, once it is created, this WBS can be modified for other studies and used again. Software systems also exist that have embedded clinical development WBS and can automate much of the above effort. 

 Planning starts with the study assumptions and is built from the ground up. The undetected inconsistencies in underlying assumptions that plague the top-down benchmarking process do not occur with this technique. 

Accurate costs for outsourced work as well as internal resource needs are generated. Having the task-level detail that ABC provides helps ensure appropriate internal staffing, in addition to helping ensure that CRO partners have allocated adequate resources. 

Creating multiple adaptive planning scenarios and reforecasting based on the findings at the adaptive "checkpoints" once the trial is underway, are straightforward and can be performed quickly and accurately by modifying the underlying assumptions.      

Following is a look at a 10-arm, placebo and positive-controlled Phase II dose-finding study of four doses of investigational drug 1, and four doses of investigational drug 2, in patients with Condition X. 

Anticipated enrollment rate: two patients/site/month 

Prior to the study, the project manager met with the statistician and medical director to determine which scenarios should be evaluated. A set of 12 were selected, which were considered to more than adequately cover the range of potential outcomes. Budgets, resource needs, and timelines were developed using the ABC method. Because this study could have been stopped abruptly, it was important to look at scenarios where patients were allowed to complete the treatment period (full treatment period) as well as scenarios where the study was stopped abruptly (see Figure 1). 

There are no differences in the duration of study start-up activities between the 12 scenarios. While there are relatively small differences in the duration of close-out activities due to the decreasing number of patients/CRF pages, because of the large differences in the active treatment part of the study the timing of the close-out activities shifts markedly to the left, which will have a significant impact on when resources are needed for these activities. 

Budget and resource estimates for the 12 scenarios were prepared. Two scenarios were selected to illustrate how CRA resource needs are determined using the ABC method (see Figure 3). Scenario 1 assumed that all 500 planned patients were enrolled and the study was completed. Scenario 11 assumed that the study was stopped abruptly for safety reasons after enrolling 300 patients, and subjects currently in treatment did not complete the treatment period. 

Efficient monitoring is based on data availability at the study sites, which is based on the treatment schedule and patient enrollment rates. Figure 2 shows the CRF generation curve for each of these two scenarios. 

This simple example highlights both the importance of simulating a range of outcomes for an adaptive trial to understand the budget and resource implications, as well as the flexibility and granularity of the ABC planning method. 

Articles

A concise look at this nontraditional approach to clinical studies that explains the how and why of it.

Molly Blake-Michaels

Articles

Operation Adaptive