Shining a Light on the Inefficiencies in Amendment Implementation

Ken Getz, MBA, Executive Director and Research Professor, Tufts Center for the Study of Drug Development, Tufts University School of Medicine

Despite decades of heightened awareness and recognition of the delays and cost associated with implementing protocol amendments, the prevalence of protocols having at least one amendment and the mean number of amendments per protocol have increased across all clinical phases since 2015. A recent study by the Tufts Center for the Study of Drug Development (Tufts CSDD) found that the average number of amendments per protocol has increased nearly 60% during the past seven years with 80% of all late-stage Phase III protocols averaging 3.5 substantial amendments (see table below).

Table 1

Source: Tufts CSDD

Tufts CSDD research has consistently shown that high and increasing growth in protocol complexity is driving the need to implement a larger average number of protocol amendments.And although their implementation is often necessary to protect study volunteer safety, improve patient recruitment and retention, and revise clinical trial strategies, protocol amendments are highly disruptive, representing the largest single cause of unplanned delays and unbudgeted expense faced by clinical teams and their collaborators.

The recent 2023 Tufts CSDD study also provides a more detailed look at the amendment implementation process—and the results are sobering. The total average time to implement an amendment has nearly tripled during the past decade, with the time from identifying the need to implement an amendment to the last ethical review board approval of that amendment now taking an average of 260 days. The average duration during which a global community of investigative sites operate with different versions of a given clinical trial protocol spans 215 total days.

This highly inefficient process is receiving considerable attention among sponsors and contract research organizations (CROs) looking to compress and optimize the protocol amendment process.

Study methods

In the spring of 2022, 16 pharmaceutical companies and CROs provided de-identified data on nearly 1,000 protocols and 2,200 amendments to Tufts CSDD. Participating companies were asked to provide data on a representative sample of randomly selected protocols—with and without amendments—having primary completion dates between 2016 and 2021. CROs provided data on protocols for companies not already participating in the study.

All participating companies collected and coded their own data. The data collected included substantial and country-specific amendments. Substantial amendments were defined by consensus among participating companies as any change to a protocol on a global level requiring internal approval followed by approval by a regulatory authority and oversight body (e.g., ethical review committee). Country-specific amendments included any change to the protocol that did not apply to all global locations. Few protocols in our sample were conducted during COVID-19. The protocols in our sample were conducted among an average of 45 investigative sites across seven countries in Phase II; and 120 investigative sites across 14 countries in Phase III.

Overall, three-out-of-four protocols required at least one amendment, with a mean of 3.3 substantial amendments implemented per protocol. Nearly 90% of Phase II protocols and 82% of Phase III protocols had at least one substantial amendment. Phase II and III clinical trials also had a high percentage of country-specific amendments at 44.8% and 60.1% of all protocols, respectively.

Changes in clinical trial strategy and regulatory agency requests were among the most common primary reasons reported for implementing substantial amendments in the 2023 study. Protocol design flaws and recruitment difficulties were among the top reported reasons for amending protocols in the earlier 2015 Tufts CSDD study.

A higher prevalence of protocols with at least one amendment and a higher mean number of amendments per protocol were observed in protocols for large molecules compared to small molecules or vaccines. And a higher prevalence of protocols with at least one amendment and a higher mean number of amendments per protocol were observed in oncology compared to other therapeutic areas.

The Amendment implementation process

For protocols that had at least one substantial amendment implemented, companies reported that two-thirds of the total number of actively participating study volunteers were reconsented. Less than 2% of total amendments, however, paused recruitment or halted the clinical trial during implementation.

A relatively high percentage of substantial amendments—approximately one in four— were implemented before the first patient had presented for their first visit (FPFV). Industry insiders have suggested that this figure reflects, in part, the haste in which protocols are completed prior to placing them in research centers.

Among Phase I protocols evaluated in the 2015 study compared to those in the 2023 study, we observed a decrease from 40% to 25% of substantial amendments implemented before FPFV. Across this seven-year period, however, we observed an increase in Phase II protocol amendments implemented before FPFV from 18% to 26%, and an increase from 15% to 22% in Phase III protocols. The total implementation process, from the date when final internal approval is granted to implement a substantial amendment to the last required approval by an ethical review committee (or oversight body) took, on average, about 260 days. The duration from final internal approval to first patient reconsented is 89 days in the most recent Tufts CSDD study—more than 2.5 times longer than the duration observed when we last gathered this data point in 2010.

This finding suggests that during amendment implementation, the length of time in which sites are operating with different versions of a given clinical trial protocol (i.e., time between the first required approval by an ethical review committee to the last required approval by an ethical review committee) was just over 7 months (215 days).

Inefficiencies in the implementation process contribute to uncertainty, confusion, and delays as some sites continue to recruit patients under a previous version of the protocol while others may postpone or even suspend enrollment. In this recent study, Tufts CSDD found that for protocols with at least one amendment, average study enrollment timelines—from FPFV to last patient last visit (LPLV)—were nearly three times longer than those without a substantial amendment and the difference between plan and actual timelines were much wider.

Driving efficiency

Protocols with at least one substantial amendment are highly associated with larger and more complex studies, having nearly 25% more endpoints, 16% more eligibility criteria, and significantly larger average numbers of countries and sites.

Sponsors have resigned to the fact that complex protocol designs are common place. Sponsors also acknowledge that rising levels of complexity are inevitable—as are the associated number of protocol amendments—given the growing number of drugs in active clinical trials targeting more narrowly defined patient populations; the difficulty finding and recruiting patients; and the more logistically demanding and complicated clinical trial execution models involving more countries, sites, convenience-enhancing solutions, and primary data collection sources deployed.

The delays and disruptions caused by protocol amendments are garnering notable attention at this time. Sponsors and others are hoping to optimize protocol amendment experience through reducing the need to amend and managing and minimizing process inefficiencies. Initial solutions under consideration include improvements in communication and coordination between internal teams and collaborators.

Automation and artificial intelligence (AI)-enabled approaches are also aspirational solutions to streamline global changes made in support of implementing substantial protocol amendment. These approaches may ultimately help to rapidly identify and predict when the implementation of substantial amendments is necessary.

Ken Getz, MBA, executive director and research professor, Tufts Center for the Study of Drug Development, Tufts University School of Medicine