The lack of diversity in clinical trials is arguably one of healthcare’s most pressing health equity barriers. Certain populations, as defined by demographic factors such as race, ethnicity, sex, gender identity, or age, have been underrepresented in research for decades. That lack in equitable access to breakthrough treatments has significant consequences, as, reportedly, about 20% of drugs have a variation in responses across ethnic group.

Still, among participants in clinical trials for new molecular entities and therapeutic biologics approved in 2020, 75% percent were white, while only 11% were Hispanic, and less than 10% were Black or Asian, according to FDA data. Additionally, exclusions of lesbians and gay men in trials in the US are not uncommon.

In a first, the fiscal year 2023 Omnibus Appropriations bill included clinical trial diversity provisions, making what had previously been suggestions on how developers and researchers might increase diversity, now mandatory. In support, the “clinical trial diversity and modernization” language in the spending package ensures the FDA can carry out its commitment to inclusion.

This bill is a catalyst for widespread change as it requires sponsors to develop and implement diversity action plans for Phase III trials, including sponsors’ goals for enrollment, rationale for such goals, and an explanation of how to meet them. The bill also establishes public workshops to enhance trial diversity and annual summary reports on progress.

I believe community pharmacies have a clear role in this evolution of clinical trials toward a more patient-centric, equitable research enterprise. We can do this through insights-driven patient acquisition at the local level, which can result in real-world insights applied across all trial designs.

One barrier that will undoubtedly continue to arise in reports is enrollment challenges. Community pharmacies, a key and trusted access point to healthcare for all Americans, are optimally positioned to help industry and regulators reach an increasingly diverse pool of participants to be invited and learn about the impact clinical trials can have on their lives.

During the COVID-19 pandemic, which magnified healthcare inequities, we saw first-hand the value of local pharmacies in providing trusted information and high-quality care for patients who needed it most and real-world insights and evidence that helped inform national and local health decisions.

The trusting relationship pharmacists enjoy with their patients, combined with the access to an extensive foundation of patient-authorized clinical data, provides an avenue for proactively matching patient populations of all backgrounds to trials across a range of disease areas.

We must—and will—continue to work hard to create the right highway to bring trials to communities. It is not a one-size-fits-all approach. We continue to face challenges with health, medical, and digital literacy, as well as trust and fear, among all underrepresented groups. However, a key doorway to improving access for patients who have traditionally been left out of research is active community education and engagement, whether directly at home, virtually, or in-person.

In addition to their physical footprint that unlocks grassroots patient education, local pharmacies can be outfitted to be fully compliant to protect research integrity and patient safety and directly engage patients using validated digital platforms. Educating patients at a community level will then hopefully reduce the cycle time from invitation to enrollment in a clinical trial.

Retail pharmacies can also serve as clinical trials hubs, improving access and the patient experience by removing system-level barriers with convenient locations, such as the local pharmacy or at home. Community pharmacies can reach patients in socially vulnerable areas, fostering the ability to provide personalization and last-mile enablement at every step in the trial journey.

Clinical trials are not only a research path, but also a way to get patients in underserved communities into the overall healthcare system, helping to advance treatment options for diverse communities. The more we can enable, engage, and empower, the more likely patients will routinely ask their healthcare providers, whether a pharmacist or doctor, if a clinical trial may be a suitable care option for them.

Articles

A Closing Thought

Through grassroots patient education, local pharmacies serve as a pivotal doorway to improving access to trials.

Community Pharmacies Can Help Bridge the Diversity Gap

There are times when problems arise, and a sponsor needs to determine if its adjudication committee needs to be rescued. This article examines early indicators of risk for adjudication committees, common reasons why adjudications are rescued, mitigation steps, and indications when it is time to seek additional expertise. It shares insights regarding additional measures that can be taken to set a committee up for success and put a trial back on track.

Endpoint adjudication committees (EACs) are independent bodies of experts that provide a consistent, objective, and unbiased assessment of safety and efficacy events during a clinical trial. As the EAC needs to be independent from sites, participants, and the sponsor, it can prove challenging to determine whether the EAC is on track or at risk, and thus placing the trial at risk.

Adjudication processes often add a level of complexity into trial conduct and without expert management, they may impact sites, vendors, and the study analysis. Many companies do not have experts in adjudication on staff, since not enough trials need adjudication, and as a result, sponsors often look to outside experts to manage their EACs.

Challenges arise if an EAC is not managed properly, or the workflows required to provide complete data to an EAC are incorrectly managed. Uncovering those challenges is critically important. Sometimes for instance, a charter submission deadline may be missed, and adjudication never begins, resulting in a growing backlog of events that will require adjudication. The EAC provider may experience ongoing and significant changes to the charter, high rates of member discordance, and missed deliverables to the sponsor or vendors that require access to the adjudication results. If delays continue, timelines will shift, and costs will rise. It is essential that an EAC provider has the requisite expertise, access to an expert network, and the tools and infrastructure necessary to deliver adjudication documents and technologies simultaneously.

The most common reason why an EAC needs to be rescued is improper management of the committee members, adjudicated data, and overall EAC process and workflow. This may result from a vendor managing multiple trial activities, including the EAC, without adequate staff or experience or more complex projects, which require expertise in that space, taking precedent. Regardless of the reason, improper management of the EAC may impact multiple workflows, vendors, sites, and the sponsor. Common signs and symptoms of poor EAC management include:

Lack of EAC-specific standard operating procedures, templates, and communication tools

Inability to drive EAC activities, data, and communication across multiple vendors

Loss of knowledge between EAC support staff

Delays – not receiving responses within 24 hours

EAC member surveys report unsatisfactory ratings

Lack of ownership and oversight of core EAC processes

Absence of EAC templates, procedures, or tools for sponsor review

Inadequate record keeping for EAC activities (i.e., manuals, charter, system training, site guidance, meeting minutes, spend reporting)

High volume of backlog events that require adjudication

Average adjudication time to reach decision exceeds 10 working days

Average time for EAC vendor to process EAC records exceeds 2.5 working days

No automation of EAC workflow or lack of EAC technology

Sponsor lacks access to EAC performance dashboard

High rate of discordance among EAC members

High rate of variability between EAC members uncovered during quality control re-read workflow

Inadequate event identification workflows

Inability to programmatically detect events or detect events that require re-adjudication

Lack of clinical and medical oversight of event identification workflow

The adjudication system plays a critical role in the entire adjudication management process and workflow and should be assessed from the perspective of each user on the platform. The technology should:

Utilize video training modules and produce training certificates

Track progress of all events for adjudication

Have real-time user-generated reports and a real-time performance dashboard

Allow users to access other EAC relevant documents within the platform

Include built-in redaction tools for site or clinical research associate (CRA) users

Include seamless translation workstreams for local language source record reporting

Be easy to navigate, with global users in mind

Be built for adjudication with EAC members in mind; it should be easy to navigate the case packet, and the platform should offer a split-screen voting option with DICOM viewer.

Receive high customer satisfaction ratings from key users including the EAC, sites, and the CRA and sponsor teams

Easy to use systems result in faster turnaround times for the EAC and higher rates of on-time database locks. In one EAC rescue study, the adjudication turnaround time (time from when the event was reported to reach a decision) was reduced by an average of 23 working days.

Checklist to determine if the technology is meeting the EAC’s needs:

Review the rate of adjudication – how many events are outstanding for adjudication and what is the average time to reach a decision?

Review the member agreement metrics – do more than 20% of events require full EAC panel review?

Connect with the group performing the statistical analysis to ensure they are receiving regular access to adjudication results in accordance with the transfer agreement.

Request system access – login to the EAC technology, access reports, or seek a demonstration from the EAC view.

Request information from the EAC vendor regarding how it seeks input from users of its EAC technology. System feedback or user pain points should be made available to the sponsor.

Planning for the end – review adjudication results and metrics on a regular basis with the vendor (being mindful of any blinding considerations).

Obtain access to submission ready archive case packets throughout the adjudication either directly in the technology or by requesting them from the EAC vendor

Many sponsors face challenges with their EAC but are unable to pinpoint their exact cause or struggle to determine whether they should try to salvage the existing committee and vendor relationship or move their EAC to a new provider. There are many factors to consider, which usually require outside expertise, to determine the right steps for the trial.

It is recommended that sponsors seek additional expertise to assess the state of their EAC, identify risks, and form a plan to ameliorate the pain points. This may be achieved by requesting that a separate team from the current vendor performs a risk assessment, or more commonly, seeking a different provider with adjudication expertise to help uncover risks and form a plan to put the EAC and trial back on track.

Feature Articles

Determining if an EAC is at risk and key actions to mitigate trial impact.

Helping Rescued Adjudication Committees Get Back on Track

As with another oft-mentioned path, the road leading to regulatory approval is paved with good intentions. Clinical trials testing molecules once deemed to be of financial and medical value can, metaphorically at least, be found stacked up outside the regulator’s door.

The many factors that halt a trial’s successful progress are a rote recitation at this point: safety, efficacy, management, and enrollment.

But do all trials dubbed failures deserve that fate? Can some be rescued, turned around? The short answer is yes. But the more efficient way to reach a regulator’s open door is to minimize the risk of that trial going belly up, before it starts.

It takes time to do that, says Heike Schön, managing director, Lumis Life Science Consulting GmbH. If the right things are done before the trial starts, it is easier to manage, she adds.

For some researchers, those right things have to happen way before the trial starts, such as during drug discovery and preclinical research. For other experts, like Schön, the right things include reconsidering the trial design; assembling a heterogenous group of key opinion leaders (KOLs) to review the protocol; keeping answers sought in a trial to a minimum; and incorporating real-world data (RWD) where possible.

“Everyone understands that there is a lot better we can do,” Frank David, MD, PhD, managing director, Pharmagellan, tells 

But while drug discovery is mostly about science, drug development is rich with, and risked by, human drama.

Unless shareholders understand the amount of time and money involved in reaching approval, dead trials can pile up.

Change must come gradually, says Duxin Sun, PhD, associate dean for research, Charles R. Walgreen Jr. professor of pharmacy and professor of pharmaceutical sciences, College of Pharmacy, University of Michigan.

For the last decade or so, the Sun lab has been studying whether validation of a new molecule should also include its effect on the disease target tissue vs. normal tissue. His team published last fall.

“Scientists are very hard to convince,” says Sun. “If the trial is at Phase II, no one wants to listen. It’s their baby … Every expert has their own vision.”

What follows is a discussion on how to advance industry’s good intentions.

David’s clients seek him out generally at the Phase II “live-or-die” juncture. As he put it, there is failure and then there is not enough success. A drug can get to the target, but not the right spot. Or vice versa. “If I get to the right place but I can’t hit the target, do I not have the right drug?” asks David.

Sun has had the same thoughts for some time. Just looking at a molecule’s structure, its biological activity, target selectivity, and potential for toxicity is not enough to pinpoint the right drug candidate. Disease vs. normal tissue, he says, is an important consideration to add, because learning about the toxic levels of a drug in healthy organs often comes too late in the clinical trial process.

Investigators looking for molecules targeting the central nervous system do this now. Those drugs must travel beyond the blood brain barrier to get to the appropriate brain tissue.

“It is well accepted that if a drug candidate has difficulty penetrating [the blood brain barrier] to reach brain tissue, it will not achieve adequate efficacy, which shall be terminated early in drug development,” Sun and his colleagues wrote last year in a published report.

In animal studies, he says, if a drug isn’t working then the dose is increased. “Animals can’t tell you they are dizzy,” says Sun.

In the Sun lab paper,1 the team looked at drug approval and failures in particular classes. More than 600 trials have been conducted for selective estrogen receptor modulators; 11 have been approved. “Some have very similar structures with only very slight modifications, yet they have shown distinct efficacy in [treating various diseases and disorders,]” explains Sun. “Their structure-activity relationship and drug-like properties cannot fully explain the difference among their clinical dose, efficacy, and toxicity.”

How money is spent on lead optimization should change, he adds. “We put 90% of the cost [to structure‒activity-relationship]. Dropping that percentage to 60%, and then the rest to study the cell type, makes sense,” he adds.

Sun says he wanted to begin a discussion with his proposal; Christina M. Brennan, MD, is senior vice president, clinical research at Northwell Health and the Feinstein Institutes for Medical Research, was sent a link of the Sun lab’s work. She says the hypothesis was innovative. “As it relates to drug discovery, it makes sense.” Among the 2,000 or so trials being run at Northwell, Brennan says that “we are seeing more [sponsors] reusing molecules, or reusing drugs for a different indication rather than starting a new drug.”

David says pharma is spending “a ton of effort” on organ-on-a-chip technology, the Lego-looking piece of microengineering that incorporates the characteristics and functions of bodily organs. It is meant to replace the often-misleading animal experimentation. “Safety signals, when they kill programs, they kill them very late,” he notes. But with the chip, “If you have a worrisome signal from that assay, then you might think twice before putting it into a trial.”

Sun is on board with organ-on-a-chip technology if it is validated, as various models can show how some tissue prevents drug penetration or how other tissue, like a tumor, is organized—the organization affects the drug’s binding capabilities.

The right translation tools help companies make the right decisions,” says David. “If you run a skimpy early clinical plan in a vacuum, it can be harder to decide whether to give it up or not.”

As for artificial intelligence (AI), its usefulness depends on the quantity, quality, and heterogeneity of the available data. According to Brennan, it can help with matching molecules to disease. AI could help at some point to create a protocol “that has global reach,” says Schön.

The use of RWD and real-world evidence (RWE) in the trial must be included, notes Brennan. The protocol of a global trial, adds Schön, must reflect the different characteristics of its participants, such as those with different metabolisms. The value of RWD is “always underestimated,” she believes.

A driver of this perception may be that RWD is not always accessible. It will take partnering with technology companies in the AI space when protocols are being developed, says Brennan. Sun was more pessimistic, believing there isn’t enough AI-generated data for his work. “Nobody is going to share,” he says.

A few years ago, Philip Pallmann, PhD, senior research fellow (statistics), deputy director RDCS South East Wales, Center for Trials Research, College of Biomedical and Life Sciences, Cardiff University, and a few colleagues wrote in 

that, essentially, adaptive design is a good idea. So far, the paper has been accessed 106,000 times. It cites case studies in which adaptive design saved the day.

“The defining characteristic of all [adaptive designs] is that results from interim data analyses are used to modify the ongoing trial, without undermining its integrity or validity,” they wrote.

Case in point: researchers re-estimated the number of patients needed for a trial on severe chronic angina after the standard deviation of the main endpoint turned out higher than originally estimated. By including more patients, the study reached its endpoint. In other words, the trial was saved. Other work has shown that the duration of trial phases—adaptive design to randomized controlled trials (RCTs)—are shorter. Of interest is that the sample size of Phase II trials, again adaptive design to RCTs, was larger.

“These options to make changes are built into the initial design of the trial from the very beginning, and ‘trigger points’ for making changes are pre-defined,” Pallmann wrote in an email. “You don’t just go off-road at some point, but you get to a fork in the road, which allows you to make an informed decision to take the fastest, or safest, way home.”

Some adaptive designs, he said, allow sponsors to stop a trial or a treatment arm early on if the drug shows little or no benefit. “This means trial sponsors can stop throwing money at something that doesn’t work, but it also means fewer participants are exposed to a treatment that is unlikely to help them, and research efforts can be focused on something else,” wrote Pallmann.

But from a practicality point of view, says Schön, who “totally agrees to it,” adaptive design is more expensive and can stretch resources because the data are needed sooner than it would be in an RCT.

That said, it is valuable in oncology trials for dose changing and ranging, the experts note. They point out that not every trial should be adaptive, because sometimes the additional complexity will outweigh the potential benefits. Regardless, Pallmann believes adaptive design benefits patients, as it allows “better and sometimes faster decision-making” in a trial, and the pros and cons of including adaptive elements should be considered early on in trial design.

Schön’s métier is righting the wrongs of a trial’s protocol, which generally reflects the politics and financial goals of the shareholders. For example, sponsors include too many endpoints in a trial because “they want to find [more] target groups, so they ask more questions,” says Schön. The upshot: underpowered trials. They have to stick to their marketing authorization goal. The questions-overload problem is no secret. “They all know that this is the case,” continues Schön. “I was at a roundtable, and we discussed these topics. They all agree there should be fewer questions.”

Brennan notes that of the 2,000 or so interventional trials run at Northwell, few follow adaptive design, whether industry- or Northwell-sponsored. “When you ask sponsors, they say, ‘What a great idea, but not with my trials,’” she explains. “We are not there yet and it’s where we should be.”

 study found that “even with a conservative estimate of trial costs, they suggest that competitor learning accounts for more than $2 billion in reallocated R&D funds.”

 The authors separated trials into various groups. Those that shared the same technology had a 23% more likelihood of stopping their trial after a competitor’s trial failed.

Publishing results of failed trials would help, says Schön. She adds that she tried to bring up the topic with colleagues, but to no avail. “They take the lessons learned internally,” she contends.

A trial’s potential pitfalls are often fixable, notes Schön, but “sometimes [sponsors] wait too long, because they think they have [designed] a good protocol.” One way to avoid trouble is to include different experts in the protocol design. Doing so, Schön says, allows design of a protocol that suits the drug and has the potential to reach the finish line. “It takes time to do the right thing at the beginning, [but ultimately the trial is] easier to manage.”

Schön recalls one situation in which the Phase II results didn’t support the primary endpoint, but it was clear the drug had potential. Instead of retesting the molecule, the developer started, and ultimately ended, a Phase III program. The primary shareholder was only interested in this one application, notes Schön. So the company went bankrupt.

Protocol adjustments aside, stakeholders, of course, shouldn’t overlook the human element involved in trial intervention—people’s foils, politics, ambitions, etc. can all impact the drug development process from start to finish. “These are human beings who are working on it, so you will have human behavior,” says Schön. “It should not be underestimated.” 

is a freelance writer for medical, clinical trials, and pharma information

Sun, D.; Gao, W; Hu, H; Simon, Z. Why 90% of Clinical Drug Development Fails and How to Improve it? 

https://pubmed.ncbi.nlm.nih.gov/35865092/

Pallmann, P.; Bedding, A.; Choodari-Oskooei, B.; et al. Adaptive Designs in Clinical Trials: Why Use Them, and How to Run and Report Them. 

https://doi.org/10.1186/s12916-018-1017-7

Bothwell, L.E.; Avorn, J; Khan, N.F.; Kesselheim, A.S. Adaptive Design Clinical Trials: A Review of the Literature and ClinicalTrials.gov. 

https://pubmed.ncbi.nlm.nih.gov/29440155/

Krieger, J.L. Trials and Terminations: Learning from Competitors’ R&D Failures. Harvard Business School. June 25, 2020, 

https://www.hbs.edu/ris/Publication%20Files/18-043_6eb7092d-9c40-417c-b8ba-5f0fba1876d0.pdf

Strategies for intervention—or, ideally, averting potential pitfalls from the onset.

How to Rescue ‘Failed’ Clinical Trials and Minimize Risks

Although most clinical trial data quality benefits from centralized statistical monitoring, critical data, processes, and associated risks identified via risk assessment following the Clinical Trials Transformation Initiative (CTTI) framework

 should drive the overall monitoring approach. Inclusion of scientific surveillance techniques as part of centralized monitoring

 enhances our ability to monitor critical and non-critical data, which are particularly relevant and recommended on trials at higher risk for measurement error, such as those with patient-reported outcomes (PROs) and clinician-reported outcomes (ClinROs).

Methods employed by scientific surveillance that enhance our ability to monitor and mitigate risks are well aligned with regulatory guidance document recommendations on including statistical surveillance as a key component of centralized monitoring. Those include FDA guidance on risk-based monitoring

; a European Medicines Agency reflection paper from 2013

; International Conference on Harmonization E6 (R2)

; and guidance from the National Medical Products Administration Center for Drug Evaluation (CDE). Some guidance documents call out specific statistical monitoring methods (e.g., the 2022 FDA Q9 (R1) and CDE guidance documents), many of which are included in scientific surveillance.

 in clinical trials and result in scientifically incompatible data. This may include:

Heterogeneity in assessments arising from the subjectivity of PROs

Rater changes, rater’s “drift” (changes in rater behavior across different test administrations), and fatigue

Expectation biases among trial investigators and subjects

Uses of prohibited medications and changes in background medications that affect study endpoints, directly or through interactions with study treatment

Increasingly complex protocols, such as adaptive designs or treatment regimens specific to certain geographic regions or to subjects with specific demographic characteristics

These factors occur due to myriad reasons such as sloppiness, insufficient training or lack of engagement and manifest as too much or too little variability, inconsistencies between related scales, “too-good-to-be-true” values or implausible temporal shifts in outcomes, causing systematic irregularities and increasing measurement error. In placebo-controlled studies, variability also can be introduced by the subject, caregiver or investigator and placebo response rates can influence outcomes profoundly. Additionally, predicting key study level risks such as premature treatment discontinuation or lost to follow up can facilitate taking early actions to minimize these risks.

Scientific surveillance includes methods that fall into three main categories:

Detecting scientific inconsistencies (or incompatibility)

Detection of inflated placebo response on blinded data

For many types of endpoints, no systematic differences are expected between sites. Hence, statistical validity is enhanced if sites’ measurements are similar in various respects. For instance, within a given scale (such as a depression rating scale), item scores are expected to exhibit similar correlations for all sites. Thus, statistical validity is enhanced if such correlations are shown to be similar between sites.

Scientific incompatibility begins with the calculation of site-level correlations (among item scores within a single scale or among scores from different scales) and corresponding overall study correlations. The presence of only random variation between outcomes of interest is tested. For a site to be declared unusual, it must lie atypically far from a central point calculated using data from all sites. Distance metrics and statistical tests using simulations are applied and the false discovery rate is controlled, after which sites of concern are identified.

Subject-level correlations among item scores within a single scale or among scores from different scales and the corresponding overall study correlations also are calculated. Distances between each subject-level correlation and the overall study correlation are calculated and standardized by dividing by the number of non-missing elements in the subject’s correlation matrix.

Figure 1 below shows two sites flagged (dark blue) based on site-level correlations for absolute values for all items within a PRO questionnaire (e.g., EQ-5D-5L) and four sites flagged for change from baseline values for the same questionnaire compared to all other sites after controlling the false discovery rate.

Subject-level multivariate mean (calculated using all relevant outcomes of interest) and correlation inliers and outliers are identified by using distance measures between subject and overall study by using standard rules associated with boxplot outliers. Observations lying at least 1.5 times the interquartile rate above the upper quartile or below the lower quartile are classed as unusual.

Figure 2 below depicts three subjects (shown as bubbles below the lower dotted control line) with a multivariate mean (using two PRO scales) below the lower quartile compared to all other subjects in the study.

Control charts recommended as statistical monitoring tools in the FDA Quality Risk Management Q9(R1) 2021 guidance are used to compare a site-specific summary measure to the study reference. Outlying sites, identified as having differences in means or variances two or three standard deviations from the study reference, are flagged for further investigation (subject to limitations as discussed later in this article).

Figure 3 below shows two sites that are at least two or three standard deviations below the control limit (shown as bubbles below the lower dotted control line) comparing variance for an outcome of interest pointing to unusually low variability at these sites. The size of the bubble is indicative of the number of subjects in that site, with larger bubbles corresponding to larger number of subjects relative to other sites.

Statistical summaries for outcomes of interest (such as those also mentioned later in this article) are visually inspected for total score/result inconsistencies, especially for sites and subjects that were flagged using multivariate inliers, outliers, and correlations.

 A visual inspection of subject-level outcomes within flagged sites is then assessed for the source of inconsistencies. For example, item scores within a PRO questionnaire, such as EQ-5D-5L, might be seen as repeated (i.e., response propagation) for multiple subjects across multiple visits at the flagged site based on site-level correlations.

Control charts also are used to compare site-specific responder rates using count type data (e.g., reduction in monthly migraine days by a defined timepoint by 30%) to the study reference value. Poisson regression with an offset is used.

Indication-specific consistency checks are implemented as applicable. For example, the primary endpoint for schizophrenia protocols is often the PANSS (positive and negative syndrome) total score. The International Society for CNS Clinical Trials and Methodology convened an expert working group that established consistency/inconsistency flags for the PANSS in 2017. The general strategy was to define irregular (e.g., excessively variable or insufficiently variable) scoring patterns, as well as incompatibility in scoring among items within the scale (i.e., cross-sectionally) and between assessments (i.e., longitudinally). The working group identified and classified 24 flags based on the extent to which they suggest error (possibly, probably, very probably, or definitely) within PANSS items or across repeated assessments. Scientific surveillance includes analyzing these flags for consistency to drive targeted actions.

The statistical monitoring methods summarized previously are applicable to a broad array of trials, spanning different therapeutic areas and clinical indications such as neuroscience, dermatology, immunology, oncology, cardiovascular outcomes, and others. Table 1 below provides examples of outcomes monitored through scientific surveillance across various therapeutic areas and indications. Key findings and recommended actions are then summarized at the study and site levels, based on these analyses.

Ideally, risks are predicted early in studies before they materialize. Early prediction facilitates adjustments to processes and retraining of involved clinical personnel so that problems do not worsen. If, for example, a misleading statement in a case report form completion guideline will lead study sites to conduct an assessment at the wrong time in subjects’ visits, then early identification of the wrong assessment times, using early on-study data, could facilitate identifying and adjusting the misleading statement before many visits are affected.

Risks are predicted using Bayesian statistical analysis which, unlike many traditional analyses, can generate probabilities about future (or otherwise unknown) events.

For instance, Bayesian prediction can generate the probability that, once the last subject completes the last visit, the to-be-observed mean deviations per subject will exceed a pre-specified threshold corresponding to some minimally acceptable level of trial quality.

In scientific surveillance, Bayesian predictions about variables of interest consist of probabilities that future, to-be-observed proportions of subjects, or means over subjects, will meet given criteria, assuming a specific number of future subjects will be observed. The criteria usually consist of upper and/or lower thresholds.

While the choices of thresholds may vary, they often correspond to one and two times the 95% upper one-sided asymptotic limits based on the trial sample size and the prior expected value selected.

In efficacy analyses of clinical trials of investigational drugs, placebo response refers to a tendency of placebo-treated subjects’ efficacy endpoints to improve due to the psychological effects of trial participation rather than any pharmacological effects.

While trial sponsors usually account for expected placebo response levels when designing trials, the actual level in any trial may exceed those expected levels, jeopardizing assay sensitivity, i.e., the ability to distinguish an effective treatment from a less effective or ineffective treatment (as defined in ICH E10 §1.5). Therefore, methods following Hartley (2012) for continuous endpoints

, have been developed and implemented for measuring, without unblinding treatment codes, the Bayesian probabilities that the placebo response exceeds expectations, and that it exceeds expectations by some clinically important amount or more.

The site correlation method described earlier only includes sites with a minimum of three subjects. It cannot incorporate smaller sites.

Similarly, site performance on efficacy measures only can be effectively assessed using the methods described in this article once a minimum number of subjects have completed assessments for the relevant post-baseline visit. Once this is the case, potential actions can be put forward.

The subject-level methods apply only to subjects with data from at least two post-baseline visits. Due to the limitations of blinded data, results are interpreted with caution. Overinterpretation of subject-level responses is avoided.

The Bayesian placebo response assessments apply only to binary and continuous (normally distributed) data in two-group blinded parallel-group trials.

Scientific surveillance is well integrated into the layered approach to protecting data integrity within centralized monitoring, as shown in Figure 4 below.

As with all risks, the process includes documenting risks that are detected via scientific surveillance in the Risk Assessment and Categorization Tool (RACT).

Each analysis is documented in the centralized monitoring plan, along with a brief description, and is applied with the appropriate cadence. This is typically in alignment with centralized monitoring reviews after an adequate number of randomized subjects have primary endpoint data available, as per the limitations noted, such that:

All relevant data supporting critical endpoints are used in scientific surveillance

Scientific surveillance is performed after sufficient accumulation of data on the primary endpoint

Results or findings from scientific surveillance are included as part of the full centralized monitoring report

The findings from scientific surveillance are discussed in a cross-functional centralized monitoring meeting, including review of all centralized monitoring findings and actions

Trends identified as risks are shared and discussed as a part of ongoing risk assessment meetings, which may warrant adding new risks to the RACT

Findings and actions from scientific surveillance related to ratings (PROs, ClinROs) generally are passed on to those responsible for quality reviews of that data, which can drive decisions to implement remediation protocols or put certain raters on a watch list

The clinical trial manager coordinates most site-level actions with the clinical research associate, especially if findings call for the investigation of site processes

Study-level risks, such as training gaps or the need for a protocol amendment or protocol clarification letter for all sites, may be identified to drive actions at the sponsor or program levels

Actions might target trial conduct, such as evaluating site process or improving subject-caregiver engagement, or might target the analyses plan, such as updating the analysis set definitions to exclude subjects from sites with serious concerns or adding sensitivity analyses to stress test impact on the analyses of primary and key secondary endpoints. Data correction may be performed where appropriate.

Incorporating advanced statistical monitoring methods in centralized monitoring significantly improves scientific integrity in clinical trials especially those with PROs or ClinROs. Implementation of scientific surveillance within the layered framework of centralized monitoring facilitates risk identification from multiple angles, ultimately contributing to a holistic risk detection and mitigation process resulting in tighter quality control. Trials of the future need to be more resilient to environmental disruptions and evolve with technological advancements.

Implementing scientific surveillance can be quite effective in detecting data errors that carry the highest potential to jeopardize study integrity. These advanced statistical monitoring methods are particularly useful as the clinical trial landscape shifts toward decentralization, coupled with continuous technological advancements in how we collect subject data.

The authors would like to acknowledge John Hamlet, Amy Kroeplin, Dorota Nieciecka, Christopher Perkins, and Timothy Peters-Strickland for the assistance they provided in developing this article.

Quality By Design - Overview. Clinical Trials Transformation Initiative. 

https://ctti-clinicaltrials.org/our-work/quality/quality-by-design/

Agrafiotis, D.K.; Lobanov, V.S.; Farnum, M.A., et al. Risk-Based Monitoring of Clinical Trials: An Integrative Approach. 

https://pubmed.ncbi.nlm.nih.gov/30100201/

Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring

https://www.fda.gov/media/116754/download

Reflection Paper on Risk-Based Quality Management in Clinical Trials 

http://www.ema.europa.eu/docs/en_GB/document_library/Scientific guideline/2013/11/WC500155491.pdf

https://www.fda.gov/media/159218/download

 International Committee for Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) E6 (R2) Guideline for good clinical practice 

https://www.ema.europa.eu/en/documents/scientific-guideline/ich-guideline-good-clinical-practice-e6r2-step-5_en.pdf

International Committee for Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) E8 (R1) General Considerations for Clinical Studies

https://www.ema.europa.eu/en/documents/scientific-guideline/draft-ich-guideline-e8-r1-general-considerations-clinical-studies-step-2b_en.pdf

Baigent, C.; Harrell, F.E.; Buyse M.; et al. Ensuring Trial Validity by Data Quality Assurance and Diversification of Monitoring methods. 

https://pubmed.ncbi.nlm.nih.gov/18283080/

Venet, D.; Doffagne, E.; Burzykowski, T.; et al. A Statistical Approach to Central Monitoring of Data Quality in Clinical Trials. 

https://pubmed.ncbi.nlm.nih.gov/22684241/

George, S.L.; Buyse, M. Data Fraud in Clinical Trials. 

https://pubmed.ncbi.nlm.nih.gov/25729561/

Herson, J. Strategies for Dealing with Fraud in Clinical Trials. 

https://pubmed.ncbi.nlm.nih.gov/26194810/

Kirkwood, A.A.; Cox, T; Hackshaw, A. Application of Methods for Central Statistical Monitoring in Clinical Trials. 

https://pubmed.ncbi.nlm.nih.gov/24130202/

Taylor, R.N.; McEntergart, D.J.; Stillman, E.C. Statistical Techniques to Detect Fraud and Other Data Irregularities in Clinical Questionnaire Data. 

https://link.springer.com/article/10.1177/009286150203600115

Kelly M.O. Using Statistical Techniques to Detect Fraud: A Test Case. 

Knepper, D.; Lindblad, A.S.; Sharma, G. Statistical Monitoring in Clinical Trials: Best Practices for Detecting Data Anomalies Suggestive of Fabrication or Misconduct. 

https://pubmed.ncbi.nlm.nih.gov/30227005/

Statistical methods used via this technique in centralized monitoring.

Applied Clinical Trials-04-01-2023

Community Pharmacies Can Help Bridge the Diversity Gap

Helping Rescued Adjudication Committees Get Back on Track

How to Rescue ‘Failed’ Clinical Trials and Minimize Risks

Harnessing the Power of Scientific Surveillance

Data Standards and ePRO Systems: A Road Less Travelled?

Raising its Profile: The Changing Practice of Pharmacovigilance

Pharmacovigilance vs. Drug Safety

Filling the Gaps in European Clinical Trial Reporting

Kudos and Hurdles in Tackling Rare Diseases