	Even at the risk of being accused of heresy, we think that moving beyond traditional statistics is overdue. And here is why: Sir Roland Fisher, who has been described as “a genius who almost single-handedly created the foundations for modern statistical science” was born in 1890. Randomized controlled trials, which many consider to be the gold standard of clinical research, were developed in the 1940s.

	This was all way before anyone had the slightest idea about big data, machine learning, neural networks, deep learning, artificial intelligence, etc. Those old methods were created for relatively small and simple datasets and before we really understood the complexity of biological systems, where interrelated and interdependent parameters always play together to generate a certain physiological output.

	But these methods still form the basis for modern medical research. We believe that a drug “works” if the difference of the means of the effect size of a variable in a large treatment group compared to a large control group is statistically significant. But, unfortunately, the “mean patient” rarely exists. Therefore, individual patients in the real world react often very differently to a specific drug than what has been predicted by the “mean” of a clinical trial. We must acknowledge that these century-old scientific methods have significant limitations, which, in our view, hamper the progress of modern medicine. A mean derived from n=1000 patients has little meaning for personalized medicine where n=1.

	The need for new, better ways for substantial evidence generation has become painfully obvious in the current COVID-19 pandemic. While there are many investigational drugs against the coronavirus, thousands of patients are still dying because these drugs are not approved for broader use due to the lack of traditional clinical evidence. This evidence is currently derived from randomized controlled trials that take months to years to complete.

We think nothing illustrates the failure of the old methodologies more than the fact that a large number of people lose their lives because the evidence generation takes simply too long to save them. 

	We urgently need statisticians, mathematicians, and computer and data scientists to come together and develop, with the tools of modern digital sciences, new 21st century methodologies for “substantial evidence” generation in a global health crisis. The signs of how powerful and game-changing these new methodologies can be are already here:

	• AI is beginning to surpass human radiologists’ in its ability to diagnose disease.

	• AI and advanced machine learning methods are starting to show promise in their ability to accelerate the discovery of novel therapeutics. Many biopharmaceutical companies and AI startups are betting that with enough data, these methods will work so well that they will help to accelerate the discovery of new therapies for the novel corona virus, 2019-nCoV.

	• Causal AI methods might be uniquely positioned to discover underlying causes of disease and clinical response to treatment on an individual level, making personalized medicine real. This approach leverages the richness of multimodal patient data from genomic, molecular, imaging across cells and tissues, deep and digital phenotyping, labs, and clinical measure across many individuals to train models that predict causal drivers of disease and response to treatment.

	Several results have been published demonstrating ability to find causal molecular drivers that emerge as a result of using AI to learning complex networks, underlying the disease with the goal of using these insights to better target treatments to patients in clinical trials and eventually at the point of care.

	We envision a future where these new tools and methods, developed under solid mathematical grounding, will enable us to go beyond the restrictions of traditional statistics, which are limited by sample sizes, the inadequacy of p-values as the metric for statistical significance,

 and the limitations of multiple hypothesis testing. 

	Looking at the big picture, they will reduce the time and cost of drug development substantially, and, more importantly, they will help to save the lives of patients desperately waiting for new, effective treatments.

, MD, PhD, Senior Vice President, Digital Sciences, Allergan; 

, PhD, Co-Founder and Chief Commercial Officer, GNS Healthcare

Articles

The need for statisticians, mathematicians, and computer and data scientists to collaborate on modern methods
for “substantial evidence” in drug development is critical.

Pandemic Highlights Urgency for New Methodologies for Evidence Generation

When it comes to implementing corrective and preventive action (CAPA) plans in clinical trials, we consistently hear similar frustrations from members of the research community year after year. They typically identify and implement corrective actions, but identifying and implementing preventive actions, which aim to minimize risks to other studies, is still somehow getting lost in the shuffle.

	Even with all the regulations, related guidelines, trainings, company policies, procedures, and quality systems in place, the same issues keep reoccurring in different trials. This happens for several reasons, namely because, when it comes to root cause analyses (RCAs), the correct root causes often aren’t identified beyond the individual study being reviewed. More often than not, the focus is solely on the tactical issue or issues at the study level, rather than also at the system level, or across studies. Consequently, issues are frequently only resolved within the individual study where they were identified, in a tactical way, and not further addressed with preventive measures for other, ongoing, and future clinical trials. A key reason RCAs are frequently not incorporated at the system level is the lack of available data on issues across trials. This makes pinpointing the systemic root cause(s) very difficult and frustrating for organizations that see the same issues and challenges -believed to have been previously addressed-pop up across their portfolio in other trials.

	In 2018, the International Council for Harmonization (ICH) of Technical Requirements for Pharmaceuticals for Human Use issued an addendum to update its guideline for good clinical practice (GCP). Taking the evolution of new technology and use of e-clinical tools into consideration, ICH E6 (R2), as the addendum is commonly known, encourages life sciences organizations to pursue innovative approaches for conducting clinical trials, specifically quality by design (QbD) and risk-based quality management (RBQM). ICH E6 (R2) also set the expectation that sponsors and contract research organizations (CROs) have systems in place to prevent noncompliance and efficiently manage significant deviations with appropriate CAPAs. However, regulatory findings worldwide consistently reveal that the CAPAs that sponsors, CROs, and investigators have in place are insufficient because there isn’t enough focus on preventive measures at the system level, across their studies. 

	In clinical research, quality issues that come up must be addressed and resolved in a timely, effective, and regulatory compliant manner. When a problem is minor and can be immediately addressed with a sufficient solution, then the quality issue can be easily resolved. However, sometimes issues are more significant and complex, and require a CAPA plan. When developing a CAPA plan, the quality issues that need to be addressed may be isolated or more broadly represented across a clinical trial or entire R&D programs. Either way, these issues can create risks to clinical trial data integrity and/or patient safety. 

	To assist with systemic RCA, we should ensure data and metrics on issues across trials are readily available.  

	Among the most significant changes related to ICH E6 (R2) is the introduction of guidance on a new risk-based approach to the management of quality in clinical research. The updated guideline calls for a formal approach to quality management that embraces technology and leverages access to real-time data to drive a more structured approach to risk management. Specifically, in ICH E6 (R2) Section 5.0, quality management now includes the efficient design of a trial to avoid unnecessary complexity, procedures, and data collection.

	In ICH E6 (R2), the addendum to Section 5.20.1 specifies that “a sponsor should perform root cause analysis and implement appropriate corrective and preventive actions,” i.e., CAPA. However, a limiting factor is that the preventive actions mentioned here are not explicitly linked to the control and reduction of risk in Section 5.0 Quality Management. 

	As an established pathway to identify and implement certain preventive actions, RBQM is the missing ingredient in effective CAPA plan execution. This risk-based approach to the management of quality in clinical research helps to ensure that corrective and preventive actions tackle the root cause of problems and minimize their reoccurrence. That’s because the basic premise of RBQM is the identification of risks on a continuous basis throughout the lifecycle of a clinical trial, from design and conduct to evaluation and reporting. Application of RBQM approaches can facilitate better, more informed decision-making and more efficient utilization of available resources.

	Imagine having access to all this information and knowing where things went wrong in the past and the conditions that led to the issues. The value of that knowledge cannot be taken for granted and RBQM makes it tangible. For example, when RBQM is part of the process from the initial stages, risk mitigation can inherently be built into the protocol. Furthermore, in this way, issues related to standard operating procedures (SOPs) and underlying conditions can be mitigated or proactively monitored, minimizing opportunities that lead to reoccurring problems. 

	Connecting the dots: Bringing CAPA and RBQM together

	Similar to breaking down data silos, we need to break down the silos that exist between CAPA and RBQM teams. Within R&D organizations, CAPA management often sits on the quality assurance (QA) side of the business, while RBQM has evolved into its own multi-disciplinary area. In practice, it’s as though these groups reside in different countries, where they don’t speak the same languages. Perhaps what’s needed is as simple as a translator to help them communicate and, ultimately, work together. 

	The FDA, the European Medicines Agency (EMA), and the ICH all advocate using a risk-based approach to monitoring data quality in clinical trials. While long touted as a more proactive way to reduce risk and improve data quality while streamlining costs, RBQM continues to be implemented in different ways, mainly because of the varying interpretations of the ICH E6 (R2) guidance. By using current and emerging issues to help us understand systemic root causes, we can implement preventive actions to reduce risk on current and future studies. After all, learning from current and emerging issues is a critical part of RBQM.

	While the changes in ICH E6 (R2) affect many clinical research stakeholders, the greatest impact continues to

 be on trial sponsors, who must ensure that the revisions are reflected in the processes and daily practice of study conduct. In bringing the RBQM language into ICH E6 (R2), there is an opportunity to improve CAPA processes through a clear link between applying learnings from current and emerging issues to risk control and reduction at the system level.
	 

	Connecting CAPA and RBQM in this way will enable real-time learning about existing risks, as they evolve, in addition to ongoing identification of emerging issues and potential threats. Armed with this continuous knowledge, sponsors and their CRO and vendor partners will be better equipped to reduce the reoccurrence of issues that repeatedly trigger CAPA processes, enabling more focused and and efficient trial operations.

Executive Director, WCG - Metrics Champion Consortium; email: lsullivan@metricschampion.org

Leveraging approaches in RBQM to enable effective corrective and preventive action processes.

Bridging the Gaps in CAPA Planning in Clinical Trials

Good clinical practice (GCP) focuses on data quality and integrity. Clinical trial sponsors must demonstrate strict oversight of studies to ensure proper conduct, safety of study subjects, and accuracy and completeness of clinical data.

	Centralized risk-based monitoring (RBM) of clinical trials greatly enhances this process. Traditionally, oversight of a clinical study includes on-site data monitoring, performed by or on behalf of the sponsors, with monitors visiting each study site at defined intervals to perform 100% source data verification (SDV) of information. This oversight is a labor-intensive, costly, and inefficient process.

	Electronic CRFs and technology enhance trial oversight

	Increasing use of electronic case report forms (eCRFs) opened the door to alternatives that provide more efficiencies and cost advantages than the SDV approach. The status of the eCRF whether it is entered, not entered, completed, open queries etc., provides valuable instant information to the statistician. A dashboard is a useful tool; it provides a visualization of these study progress metrics, along with offering insight into the study’s data at a glance. The centralized RBM alternative improves monitoring cost-effectiveness without compromising quality and integrity. It identifies trial areas at greatest risk and implements targeted measures and controls to manage trial quality. Additionally, RBM helps improve clinical trial design, conduct, oversight, recording, and reporting, while ensuring human subject protection and reliability of trial results.

	With ongoing RBM, cumulative data can be examined at subject and site levels, flagging potential errors that must be queried or systematic errors in process that may occur at a site (e.g., measurements that look too low or too high as compared to other sites). The data monitoring team can then take remedial action. For example, this could trigger an on-site monitoring visit, or further site training. Review of query rates by site, subject, or form can reveal possible data quality issues. Quality tolerance limits (QTL) can be set and monitored to focus resources on vulnerable areas to guide the level of action required. Centralized monitoring is discussed and encouraged in FDA guidiance

 and an European Medicines Agency (EMA) Reflection paper.

	Key risk indicators detect potential issues 

	Key risk indicators (KRIs) are critical data, as are other study variables or operational data that can be measured throughout a study to detect potential compliance issues before they become a problem. Operational data can highlight site-level concerns but have potentially limited direct impact on subject safety and data integrity. They can be visualized in a dashboard format for ease of monitoring. Dashboards are a way of automating the integration of the KRI datasets and help users to see outputs at a glance, spot trends, and compare metrics side by side. An example of a KRI is duration of open queries (see Figure 1).

	Quality tolerance limit can trigger investigations

	QTL is a level, point, or value associated with a trial variable that should trigger an investigation if a deviation is detected in order to determine if there is a possible systematic issue (i.e., a trend has occurred). QTLs are essential for the integrity of a trial, including key endpoints and patient safety. Whereas KRIs typically prompt risk mitigation actions at site level, QTLs are monitored at the trial level and are predefined before the trial commences from a review of historical data from similar trials and, where possible, using statistical methods and modeling.

	A QTL for protocol deviations can be created and tested using simulated data. Unusually high levels of deviations may indicate an issue at that site, but unusually low levels of deviations may indicate underreporting. QTLs should identify both. Investigations made in real time increase the chance of determining root causes. For example, deviations arising from missing protocol-required assessments may be due to inadequate resources, overlooked training needs, or equipment failure. Early identification of the root cause enables corrective actions or procedures to be put in place and subsequent impact on trial quality mitigated. 

	Easy data visualization against the QTL is key to successful RBM implementation. The example in Figure 2 below may be plotted against calculated limits to identify breaches and be seen easily in a dashboard format.

	Statistical methods can help identify patterns

	Centralized monitoring provides access to cumulative data across sites. The use of statistical methods helps find unusual or implausible patterns in the data to indicate, for example, potential manipulation or rates of adverse events (AEs); if one site has a low rate of AEs comparatively, this might indicate under-reporting, or difficulty in how to classify AEs based on symptoms, which should flag further investigation. Other patterns that can be used to identify potential data issues include:

 If a site or subject shows much less variability in a measurement than other subjects/sites, this may indicate the data is not real and trigger further investigation.

 Data that are invented by people tend to show preferences for certain numbers, such as rounding up. Data can be examined to see if the rates of any of the digits are higher than expected.

 Clusters of values very close to the mean may indicate fabricated data. 

	Centralized RBM: A successful clinical trial

	In summary, centralized RBM helps streamline trials while alleviating labor-intensive and costly SDVs. At the same time, it improves data quality by guiding and prioritizing site visits and setting and monitoring QTLs using statistical methods. All this adds up to the GCP mission of strict oversight and improved and more efficient approaches to clinical trial design, conduct, oversight, recording, and reporting while ensuring participant safety and clinical study data accuracy and completeness.

is Head, Clinical Data Operations, PHASTAR

	1. Guidance for Industry: Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring, Draft Guidance.  

http://www.fda.gov/downloads/Drugs/Guidances/UCM269919.pdf

	2. EMA Reflection Paper on Risk-based Quality Management in Clinical Trials. 

https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-risk-based-quality-management-clinical-trials_en.pdf

Implementing centralized risk-based monitoring can help meet strict GCP requirements for study conduct, oversight, and recording.

Central RBM Supports Reduced Cost, Higher Data Quality

	Medable, a decentralized clinical trial digital platform company, was co-founded in 2014 by Michelle Longmire, MD, who is also its CEO. The company has shown consistent business growth, but the ongoing COVID-19 pandemic has brought its services to the forefront. In the following Q&A, Longmire discusses her

 background, Medable, and how research will be transformed by the pandemic.

Can you give a brief history of how you became a physician-turned-CEO of a decentralized clinical trial digital platform company?

I was a physician scientist at Stanford and was researching the epigenetics in a very rare disease in identical twins, which is inherently a challenge for researchers.

	Through that research, I realized that the clinical setting had limitations. We were limited to patients in the immediate geographic area. Also, there’s a lot of health data that isn’t effectively captured. I realized there was an opportunity to build a new kind of suite of technologies that would open the door for a much broader set of research participants and participation. This technology would dramatically impact the quality of data, capturing real-time data that we would normally only capture on a very sporadic basis, and in a much more continuous or true-to-life way.

	Being at Stanford, where entrepreneurship is part of the lifeblood, and then the component parts of Silicon Valley from investors and other startups, the idea that you could start a company is much higher here, because you’re surrounded by the people who are actually doing it. I had to decide whether this work would be best done in the setting of academia or the setting of a private company. And I realized I was more aligned with the entrepreneurship mindset, which is one where you start from scratch and there are no structural limitations, outside of yourself. So, through that, I started the company, got funding, and built this awesome team that we get to work with on a daily basis.

There has been a lot of talk about remote, decentralized, and virtual trials for some time. What have been the true barriers to adoption that you have observed?

 We started out with this idea that through a mobile application, patients could participate in research anytime, anywhere. That was furthered by Apple with its work on Apple ResearchKit, which showed we don’t have to limit participation to a couple hundred people. When we go directly to participants through mobile devices, we can get hundreds of thousands to millions of participants. That reframed research, but was much more on an observation basis. However, taking a registration and conducting Phase II and Phase III studies in the context of direct-to-patient or patient in the home has evolved since then. 

	We’ve had to answer “how do we capture data that can be submitted to a health authority for a drug trial with high enough quality or even better quality, in this new setting?” That has changed over the last two years and specifically over the last couple months due to COVID-19. It went from a nice-to-have, to truly the only option for continuing to capture this type of data and to enable this type of healthcare delivery.

Now, with the COVID-19 pandemic, many are saying that the experience will finally push industry toward a new way of working. What are your thoughts?

Necessity is the mother of invention and even the mother of adoption, when it comes to this pandemic. Some of these new technologies and processes had already been invented, but not fully adopted because there was fear that the other way was safer. So, now when there’s no other way, you can prove it works; you have validation. I think everyone has been waiting for the proof. We needed to generate the proof that decentralized trials and remote or virtual research could be done well, safely, and conveniently.

	There’s no question that these new approaches work, and work better-in some cases-than traditional 

methods, so I don’t see the industry reverting at the scale we had traditionally done. 

	Medable and its partners are now working to ensure every site has telemedicine as a basic capability to provide remote care, because you never know when you’re going to need that. Moving forward, it will be a core clinical continuity/preparedness measure. We will also see very thoughtful and scaled planning around hybridization and decentralization for upcoming and in-flight clinical studies.

What specific examples during the early days of the pandemic from your company give you a signal that industry will be able to tip the scales toward decentralized trials?

 In the first couple of days, everyone from existing partners to totally new partners were calling us around the clock looking for help. We created a workstream for rapid validation and qualification of individual studies, as well as partners, so that we could deploy technology quickly. What was never thought possible before was suddenly turned around in a matter of days to a week. We literally turned months-long timelines into days-and this was key, as was the ability to deploy technologies with necessary processes around patient data privacy, security, and data quality. 

	One new partner wanted to go live in Italy during the peak of the COVID outbreak there. They had a critical visit the next Monday on a hard-to-recruit trial, and they really wanted to ensure they could capture the data from the patients who were set to come in that next week. We were contacted the Monday prior, and we worked day and night to launch that next week. We deployed pager monitoring and devices and trained the physicians and the site staff over the weekend, in their homes and us in our homes, but we got everything off the ground. We also learned where the roadblocks were and that helped us develop a much more efficient process.

There is a divide of resources going toward COVID research, and resources for the in-flight studies. What is Medable doing toward the COVID-19 research end? 

 We have seen a huge influx of biotechs and sponsors who want to do clinical trials on investigational treatments for COVID, either vaccines or interventions. Because we realize we needed to be able to move faster than we were before, we created ACCESS, as a research superhighway. 

	ACCESS is an existing or master protocol and technology system, where participants all over the United States who are engaged in active clinical trials collect key baseline data so ACCESS can match participants to 

. Next, the system dynamically enables participation through the same mobile application. We can onboard and launch new studies for biotechs through the system in a matter of a week or two, versus a standard startup timeline, where eight weeks would be considered short. 

	The ACCESS initiative, which is short for American COVID-19 Collaborative Enabling Seamless Science, also includes BioIntelliSense, Datavant, Parexel, PWNHealth, and the American Heart Association’s Center for Health Technology and Innovation. 

	The concept of a master protocol and an existing framework or research superhighway accelerates participation for those already moving down the research pathway and facilitates a much faster drug development process. 

Physician scientist-turned entrepreneur discusses the adoption of remote, decentralized, and virtual trials and the likely new research landscape post-pandemic. 

Quality Amid the Chaos: Breaking Down Barriers to New Data Capture Models

Since the outbreak of COVID-19, there has been disruption to clinical trial management including missed patient visits and assessments, laboratory tests not being performed and submitted, and lapses in study drug administration. Investigative site closures and patient non-attendance will raise the issue of best practice for restarting studies and sample sizes may need to be reviewed with statistical guidance as possible gaps occur.  

	All of these factors pose risks to the integrity of trial data and it is expected that these challenges will impact clinical trials for at least four to five months. Whenever feasible, sponsors are both ethically and economically obliged to continue ongoing trials, and ensuring that decisions around protocol revisions and other trial adjustments are properly captured and documented is essential to the scientific integrity of the trial.

	What are the regulatory authorities advising?

	Competent authorities are urging sponsors to utilize data monitoring committees (DMCs), if not already part of the study design, and regulatory agencies are issuing new and updated guidance as the situation evolves. The European Medicines Agency (EMA) has recommended that trials without a DMC should establish one now to help monitor subject safety and trial disruptions caused by the pandemic.

	EMA has also suggested that ongoing studies consider undertaking an interim analysis, examining the validity of the trial by the DMC. The DMC should examine trial outcomes that have the potential to be impacted by the presence of different trial populations: 

		Some patients completed the trial before the start of the pandemic. 

		Some will have been ongoing during the pandemic and have been impacted by the associated measures. 

		Some will be enrolled as normal activities resume after the end of the pandemic. 

	These timings and measures will be country- and region-specific. Measures taken in relation to the COVID-19 pandemic may interfere with study treatments. An interim analysis would enable the DMC to identify and address such concerns. For instance, assessing the impact of missing primary endpoint data, treatment gaps and discontinuity, and serious adverse events related to hospitalization due to infection and missed imaging assessments. EMA has also recommended that existing DMCs consider updating their charters in order to broaden their scope and enhance their ability to effectively guide and steer the trial during these unprecedented events. FDA guidance on the conduct of clinical trials of medical products during COVID-19

 also indicated that since a primary responsibility of the DMC in any trial is to assess patient safety, then they are well-placed to assess the impact of modifications required due COVID-19 conditions. 

	A properly constructed and well-organized DMC is one of the best ways to ensure patient safety, data integrity, and appropriate analysis and interpretation of a clinical trial’s data. In addition, DMCs can support objective decision-making regarding the study, while minimizing potential bias. The DMC charters are focused on preserving the integrity of the trial, and clearly record and outline necessary protocol revisions and changes to the planned analysis that may be necessary, steps which FDA recommended in its guidance outline.

 They will also be vital in continuing to monitor patient safety in ongoing studies. 

	While it is important to assess each trial for suitability, a DMC could also potentially make recommendations on patients’ eligibility to remain in the study after missed visits or dosages, interactions from other medicines taken during the course of the trial, sample size adjustments, and other induced potential biases. 

	Depending on the therapeutic area targeted in the study, there may be a need for new table listings that would support DMC decision-making. The DMC has the ability to look across various subsets of the trial and different countries and regions to evaluate any differences in the treatment groups based on exposure and infection rates. 

	Looking ahead, having a DMC could also be instrumental in helping study teams navigate considerations in restarting trials as the pandemic recedes, by reviewing possible adjustments in sample size and protocol revisions. Many of the changes would have to be approved by local ethics, institutional review boards (IRBs), and regulatory agencies, but the input and documentation from DMCs could help pave the way for pragmatic approaches and solutions adapted to the circumstances.

	Furthermore, DMCs and associated adjudication committees could be important in evaluating cardiac events, as many COVID-19 patients are presenting with elevated cardiac markers and ECG tracings consistent with myocardial infarction changes that could confound the data.

 DMCs could also help in making sure that trial endpoint definitions were updated to include clear and stringent language when determining whether hospitalizations or other endpoint events were COVID-related or not. 

	By creating or revising a charter with good, well thought out, and tight definitions and applying these to source data from the site, the event adjudication committee (EAC) can make decisions that the trial team can be confident in and would support the final findings. 

	If a sponsor is unfamiliar with DMCs, it is advisable to seek advice on setting up and managing this activity effectively. This support can extend to access to proven processes in charter development and connectivity into an already established global network of recognized thought leaders and physicians to serve on these committees. All of this can save valuable time during this evolving and dynamic period. Outsourcing the DMC can also avoid perceived bias in analysis and interpretation of the trial results, which is important in maintaining data integrity.  

	The use of technology is another consideration. There are electronic adjudication systems on the market that have real-time status reporting, and this enables the delivery of a streamlined, efficient, and transparent process for accurate event collection, with high quality results in a timely fashion.

is Senior Director, Adjudication & Data Monitoring Management, ICON Clinical Research Services

	1. EMA Guidance Document: “Points to consider on implications of Coronavirus disease (COVID-19) on methodological aspects of ongoing clinical trials” EMA/158330/2020, March 25, 2020, page 3, first paragraph

	2. FDA Guidance on Conduct of Clinical Trials of Medical Products during COVID-19 Pandemic Guidance for Industry, Investigators, and Institutional Review Boards, March 18, 2020, 

https://www.fda.gov/drugs/coronavirus-covid-19-drugs/clinical-trial-conduct-during-covid-19-pandemic

	3. Xiaowen Wang, MD, and Deepak L. Bhatt MD, MPH. COVID-19: An Unintended Force for Medical Revolution. 

Regulators are urging the use of data monitoring committees to support clinical trial management and decision-making during pandemic.

DMCs: Maintaining Data Integrity in COVID-19 Climate

	With hundreds of clinical trials for potential coronovirus therapies in the works-and even more in development-concerns have mounted about the emergence of conflicting data, useless results, and wasted efforts from multiple overlapping efforts. In response, the National Institutes of Health (NIH) announced a

 broad public-private partnership (PPP) to collaborate on prioritizing and coordinating research on COVID-19 drug and vaccine candidates.

 The Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) initiative has support from a number of federal agencies, and some 16 pharma and biotech companies are participating through the Foundation for the National Institutes of Health (FNIH).

	NIH Director Francis Collins emphasizes the importance of bringing “unassailable objectivity” to swiftly identify the most promising compounds from multiple stakeholders. Those priority therapies should gain fast and efficient regulatory oversight and vetting, moreover, through the involvement of FDA and the European Medicines Agency (EMA) in the partnership.

	This initiative builds on earlier efforts, such as the COVID-19 Therapeutics Accelerator  launched in March by the Bill & Melinda Gates Foundation to drive collaboration among drugmakers, regulators, and non-governmental organizations.

 In addition, more than a dozen biopharma companies formed the COVID R&D consortium to share public information on research efforts, while several international organizations are coordinating clinical trials and developing master protocols.

	The new NIH-based partnership appears poised to advance collaborative research more extensively, with industry and government partners providing infrastructure, subject matter expertise, and funding to identify top priority candidates for advanced testing. FNIH will manage an ACTIV steering committee to develop an inventory of potential candidates, launch master protocols with a single control arm, and set criteria for ranking potential candidates for first-wave and subsequent evaluation. Another subgroup will set standards for preclinical evaluation methods through a central repository to assess models, extend high-throughput screening facilities, and compare approaches for identifying informative assays. And a third group will tap NIH’s extensive clinical trial network infrastructure to build capacity for expediting trials and to study different populations and disease stages.

	To advance vaccine development, another ACTIV group will form a collaborative framework to map epitopes and develop assays, establish protocols for sampling and immunological analyses, collect clinical data on immunological responses and endpoints, and engage with regulators on surrogate endpoints for clinical evaluation.

	ACTIV’s initial goal is to select six to  eight compounds for clinical trials that assess different mechanisms of action, Collins explained in a recent webinar sponsored by the Duke Margolis Center for Health Policy (comments available at https://bit.ly/3fc45gi). He noted that the working groups met in early April to start the process of identifying the highest potential therapies, particularly those where manufacturing scale-up won’t be a major roadblock. The candidates will include anti-virals with well-established mechanisms of action, immune therapies, and immune suppressants for patients with advanced lung disease.

	Establishing “really organized clinical trial capacity” is critical to be able to slot promising compounds into networks utilizing master protocols, Collins emphasized. Even if current studies of existing therapies such as Gilead’s remdesivir show benefit, he predicts a strong need for combination therapies going forward.

	FDA is deeply engaged in all these working groups to ensure that the group’s clinical studies will meet regulatory standards. And it’s all moving “at rocket pace,” Collins observed, with the goal of identifying two to three promising therapies by July or August for further testing in the fall. A main benefit of this initiative, he noted, is that “all stakeholders are together around the table.” Biopharma companies are “not just pushing their own candidates,” he noted, and are offering clinical trial capacity “to get this done.”

https://www.nih.gov/news-events/news-releases/nih-launch-public-private-partnership-speed-covid-19-vaccine-treatment-options

https://www.biocentury.com/article/304947/nih-and-fnih-are-leading-a-public-private-parternship-to-defeat-covid-19

With hundreds of trials for potential coronavirus drugs in the works, concerns swirl around the emergence of conflicting data, useless results, and wasted efforts from multiple overlapping efforts. 

More Coordination Sought for Exploding COVID-19 R&D Efforts

	Europe may not be winning huge praise yet for its response to the COVID-19 pandemic-embroiled as it is by divisions over everything from sharing resources to exit strategies-but the European Medicines Agency (EMA) is coming out of it with flying colors, particularly for its timely responses to drug developers’ and clinicians’ needs.

	One was its judicious reminder last month of the risk of serious side effects with chloroquine and hydroxychloroquine, in the face of growing advocacy-including from US President Donald Trump-for its use in treating the novel coronavirus. EMA points out that these products are known to potentially cause heart rhythm problems that could be exacerbated at high dose or if combined with azithromycin or other medicines that have similar effects on the heart, as well as side effects on the liver and kidneys and nerve cell damage that can lead to seizures and hypoglycemia.

	“Clinical data are still very limited and inconclusive, and the beneficial effects of these medicines in COVID-19 have not been demonstrated,” EMA says. “A number of large, randomized clinical trials are looking at the benefits and risks of chloroquine and hydroxychloroquine in patients with COVID-19,” but “results from large, well-designed studies are needed to make any conclusions.”

	EMA’s engagement goes way beyond warnings, however. Already in March it created the COVID-19 pandemic task force, to manage and coordinate discussions on relevant product development, authorization and surveillance, and to conduct post-authorization follow-up of relevant authorized products. In particular, it is helping to meet the new challenges of the current health threat in the complexity of the disease, the variety of potential or repurposed medicinal products for prophylaxis or treatment, and the need for rapid clinical trials across Europe.

	In April, EMA produced updated guidance for medicine developers and pharmaceutical companies to help speed up medicine and vaccine development and approval for COVID-19, advising on how they should address the regulatory challenges arising from the pandemic. This encourages developers of potential vaccines or treatments to contact EMA as soon as possible to discuss their strategy for evidence-generation. Scientific advice is free of charge and can be fast-tracked for potential novel coronavirus treatments or vaccines, the agency reminds developers. It is urging the EU research community to prioritize large randomized controlled clinical studies as the most likely to generate the conclusive evidence needed to enable rapid development and approval of potential treatments, and to include all EU countries in these trials. And it is encouraging the pooling of EU research resources into large-scale, multi-center, multi-arm clinical trials.

	EMA has also produced advice for sponsors of other clinical trials affected by the pandemic, covering how they should adjust trial management during the pandemic-in situations where, for example, protocol deviations are needed for dealing with the need for isolating participants, or for limited access to public spaces or the reallocation of healthcare professionals. It outlines some regulatory flexibilities that can be applied to help pharmaceutical companies cope with the consequences of the pandemic, ranging from marketing authorizations and regulatory procedures to manufacturing and importation of active pharmaceutical ingredients and finished products, and from quality variations to labeling and packaging requirements. “EMA will be flexible and pragmatic during the assessment of affected clinical trial data submitted as part of marketing authorization applications,” it adds, encouragingly.

	The agency is taking a leading part in a new series of meetings of the International Coalition of Medicines Regulatory Authorities, being convened biweekly to allow medicine regulators worldwide to exchange information and work together in speeding COVID-19 medicine and vaccine development and approval. It has launched, also in April, a fast-track monitoring system to help prevent supply issues with crucial medicines used for treating COVID-19 patients.

	And EMA’s staff-already strained by the shift from London to a temporary home in Amsterdam, and then a further shift into its new custom-built premises, as well as by staff sickness related to COVID-19-not only gave up their Easter break to keep essential work going to combat the pandemic, but they will also worked through the Spring holiday that was between April 27 and May 1.

	As Shakespeare, whose birthday was celebrated last month, might have said of the agency:

 is a freelance journalist who specializes in European health affairs and is based in Brussels, Belgium

The agency has taken the lead in orchestrating a series of meetings of the International Coalition of Medicines Regulatory Authorities to help share information and collaborate in speeding COVID-19 drug and vaccine development and approval.

EMA Wins Points for Its COVID-19 Responsiveness

	Companies Launch New Framework to speed up COVID-19 Research

	Medable, Inc. has unveiled a multi-company research framework to accelerate development of diagnostics and treatments for COVID-19, providing individuals fast access to participate in leading-edge research for diagnostics, health monitoring, and interventional clinical trials. The ACCESS initiative-short for American COVID-19 Collaborative Enabling Seamless Science-provides a mobile consumer application and secure infrastructure to quickly connect health researchers and trial teams with up to millions of home-bound individuals in the U.S.

	ACCESS makes it easy for individuals to contribute specific information about their COVID-19 experience, combine it with health records and data from wearable devices, and opt in to participate in current and future studies for diagnostics, treatments, and vaccines.  

	Trial technology company Clinical Ink has announced a wide-ranging Study Assurance Program to minimize financial impact of the disruption from COVID-19 on new and existing clinical trials. Contracting adjustments include change order waivers of all platform licensing and help desk fees for a period of up to three months for all current and new studies.

	“This is a time to show that patients actually matter-it’s not just about money,” said Clinical Ink CEO Ed Seguine. “Clinical Ink is putting patients first by proactively waiving potentially millions of dollars in change orders so that new and current studies can continue without our customers having costs increase both now and in the long run. I invite other clinical technology and service providers to make a similar pledge so that our combined efforts can help ensure patients truly do come first.”

	AUM LifeTech, Inc., a preclinical stage biotech company developing RNA therapeutics, and AUM BioTech, LLC, a genetic research startup, have jointly launched a research program called Knockdown Coronavirus with a goal of developing a treatment for COVID-19. Under this initiative, the two Philadelphia-based companies are offering self-delivering RNA silencing research products, powered by FANA ASO technology, to the global coronavirus research community to facilitate research and fast-track therapeutic development. 

	AUM LifeTech is currently working with collaborators on developing a treatment for COVID-19 AUM BioTech’s gene silencing research tools have the capability to selectively knockdown RNA of the virus. Further, its RNA targeting technology can be used to perform high-throughput genetic screening to identify the function of viral and host genes and help identify new targets for COVID-19 therapy development. 

	Advarra, a provider of institutional review board (IRB), institutional biosafety committee (IBC), research technology solutions, and quality and compliance consulting services has acquired IRB Company, Inc. (IRBco), an AAHRPP-accredited central IRB that has conducted research reviews in the U.S. since 1981. Advarra continues to expand its presence as the largest provider of integrated IRB services in North America.

A compilation of recent notable news developments that pertain to the clinical trials industry.

News Notes

Even though the ICH GCP E6 R2 guideline had been released quite some time ago, companies still struggle with “taking a risk-based approach” to monitoring and quality management. The implementation of the commonly perceived “abstract” guideline seems challenged by the lack of practical experience and knowledge helping those supposed to convert the requirements into a day-to-day routine move forward more quickly.

	Originally, everybody was under the impression that this guideline is mainly looking into risk-based monitoring (RBM), i.e., a process to reduce source data verification and, as a result, save on monitoring resources while, in turn, not jeopardizing the quality of the data. Rapidly, it became clear that this understanding would mean to jump too short. For that reason, the term risk-based quality management (RBQM) had been introduced, moving away from the unilateral view of just monitoring. 

	RBQM starts already at the development level of the target project profile, and subsequently covers the development of the protocol, the electronic case report form (eCRF), the data structure, the decision on the inclusion and exclusion criteria to be applied, the processes around the data capture topic (EDC, ePRO, wearables, central lab, central ECG, biomarkers, PK data, etc.), the enrollment, the data entry, the response to queries, the quality of the data generated, and eventually also the processes around the data traveling through the many systems. For example, from source to EDC, to the backend data management system, partially into the CTMS, safety data into the pharmacovigilance or safety database and eventually into the CDISC SDTM and ADaM database structure.

	In real life, study teams, stakeholders, and leadership first have to understand the concept, the rationale, and the background of the updated ICH guideline before they feel confident enough to embrace the RBQM process integration and are able to contribute to its success. 

	As a consequence, Cyntegrity regularly gets approached by its clients with the request to provide education on RBQM process integration, addressing both the basic and more advanced aspects of it. About nine months ago, Cyntegrity was contacted by Merz Pharma with a similar request for professional education. Based on the current, revised regulatory view, Cyntegrity offered Merz a four-layer certification program, i.e., the MyRBQM Academy White, Green, Black, and Executive Belt curricula, and a real-life case study workshop that involved the retrospective analysis of an already completed trial.

	The White Belt course: The fundamental knowledge

	In order to make this fundamental RBQM knowledge level accessible to a broad clinical research audience, Cyntegrity decided to create a compact online course. The beauty of such a format is that everybody can run the training self-paced at any time when it is convenient. Importantly, such basic training on RBQM is relevant to almost everyone involved in the clinical trial arena and  beyond.

	The online White Belt certification covered the main aspects of the ICH E6 R2 guideline, as well as the latest ICH E8 draft, and provided real-life examples using engaging ask-the-expert style video clips. 

	This first belt level was received very positively by Merz’s e-course participants, i.e., all staff completed the course within the expected timeframe. At the end of each course chapter, participants had to complete a multiple-choice quiz. Fortunately, all participants managed to successfully complete the quiz at the end of their White Belt online course. 

	This was important since the successful completion of the White Belt course was the prerequisite to attending the subsequent belt levels. 

	The Green Belt course: Identifying risks, pinpointing patterns   

	This course was set up differently from the White Belt course. Since RBQM is a functionally overarching exercise, representatives from various functions were invited to this instructor-led training. 

	In preparation for the Green Belt course, Cyntegrity reviewed a sponsor protocol of a completed study, identified the potential risks, determined the key risk indicators (KRIs), and linked them to the risks. In addition, Cyntegrity had access to the data from a completed study and analyzed the data available with respect to their contribution of the study failure. The question to be answered was, if the implementation of RBQM prior to the start of the study would have facilitated an early detection of the factors driving the study failure.

	The sponsor company had been asked to also come prepared to the instructor-led workshop, so that the risks the sponsor identified could be compared with those risks determined by Cyntegrity. Comparing the two sets of risks and risk indicators revealed quickly a rather high congruence of the two sets. That means that even a team that had not been exposed to RBQM a lot faster can learn what is important and what is less important to look at.

	Based on this and many other similar exercises, Cyntegrity developed a list of “gold-standard” KRIs, i.e., a set of KRIs that are applicable to most of the clinical studies at Merz and even industry-wide. These KRIs included the enrollment over time and the adverse events/serious adverse events reporting.

	After identification of the risks and the analysis of the respective data, the workshop participants looked at the development of the risks over time. That means, for example, looking for a point in time when a particular site in the study, or several sites, started deviating from the expected pattern. Of course, this requires that the underlying system permits a rather tight screening of this data, meaning that the system must run quite frequently across the respective data. 

	It turned out that the system-had it been implemented-would have alerted the study project manager earlier on on the factors contributing to the study to fail. The sponsor company-like several others we provided training for-had also identified those risks in their studies, however, usually too late to act upon them in a timely fashion and with a lot of manual effort. 

	This hands-on workshop was perceived very beneficial to those closely involved in the rollout of RBQM, as well as those supposed to manage the system once the rollout had been completed. As many recognize, implementation of a new system in an organization requires a significant amount of preparatory work and is associated with change management, something Cyntegrity covered in the Black Belt course.

	Black Belt course: Continuous optimization

	Increasingly, pharma, biotech, and medical device companies, as well as clinical research organizations, are observing that their programs concerning the use of new technology are not achieving their intended outcomes. In particular, compliance programs are seen as too expensive, ineffective, and unable to keep up with the new and emerging ICH GCP standards and regulations. Compliance appears to be falling behind and exposing biopharma companies to unnecessary risk.

	So, when planning and implementing improvements to their research operations-whether that means creating new processes to ensure regulatory compliance or adopting a new piece of technology-it is essential that they’re not forgetting the basic principles of change management. The most difficult part of any initiative is the behavior change it calls for, and RBQM is no exception. The implementation of RBQM without an implementation strategy is just a wish.

	Cyntegrity developed the Black Belt course mainly to address the change management aspect when working on a RBQM rollout. The course had been created for a subgroup of those people that already passed the White and Green Belt courses successfully, and is intended for those in charge of a rollout of the processes and tools into the organization. It addresses the main aspects of change management, the various reactions usually surfacing in the staff’s mind when a change shows up on the horizon and how to address their concerns in a way that they support the change eventually.

	In addition, the Black Belt course offered the opportunity to address specific questions and situations related to the implementation of RBQM in the respective organization, which might be different from case to case.

	All of the completed educational journeys certainly lived up to Merz’s expectation with respect to a successful introduction to the RBQM topic, facilitating a sound understanding of the challenges and opportunities associated with RBQM, and even addressing the change management topic. 

	However, new processes and new technology linked with an initial investment also require the buy-in of leadership. To that end, Cyntegrity also developed a concise Executive Belt level.

	Executive ‘Gold’ Belt course: Calculate your ROI

	Other than the first three belt levels, the Executive Belt course especially focuses on the key performance indicators of the RBQM implementation, so that leadership fully understands the requirements by the health authorities and the benefits for their own organization. Using real-world evidence-as explained for the Green Belt course-helped a lot to highlight the importance and the benefits of RBQM.

	In addition to the Green Belt case study analysis, Cyntegrity developed an RBQM ROI calculator supporting the calculation of the “return on investment” performance measure and the benefits of RBQM permitting the user to enter their specific underlying data and calculating their resulting ROI. 

	Turning the abstract concept of RBQM into practical routines remains challenging for most biopharma companies. Overall, the implementation of RBQM requires a solid preparation. The provision of a set of training modules helped the organization-in this case study, Merz Pharmaceuticals-to initiate a smooth RBQM rollout. Based on the feedback of Merz and other clients, such an educational approach would also help others to jump onto this bandwagon and be successful in the RBQM arena.

PhD, is Chief Scientific Officer, Cyntegrity

A look at the effectiveness of one training and educational program in helping to initiate rollout of risk-based quality management practices.

The RBQM Implementation Journey: A Case Study

Click the title above to open the Applied Clinical Trials May 2020 issue in an interactive PDF format.

Applied Clinical Trials, May 2020 Issue (PDF)

	There’s been quite an overwhelming amount of industry-specific information regarding COVID-19 the last few weeks. It appears there are a number of patient registries from specific medical associations, as well as others such as the LabCorp/Ciox Health patient data registry. There are companies offering de-identified data to support clinical care or clinical research decisions, including Ancestry and Cerner. There are clinical trials

 service providers, including WCG and Bio-Optronics, offering internal benchmarking data on investigative sites, as well as investigative site survey data from Contiuum Clinical and Clinical SCORE. Consortiums have formed such as the COVID-19 Research Database, a secure repository of HIPAA-compliant, de-identified patient-level data sets, and EndPandemic National Data Consortium, with the goal of integrating data from all ongoing and future clinical studies to accelerate analysis on COVID-19 and SARS-CoV-2 research. 

, which as an aside lists even more websites that identify and track research on COVID therapies and vaccines-the many competing resources has led the NIH to form a broad public-private partnership to collaborate on prioritizing and coordinating research on COVID-19 drugs and vaccines. 

	And that’s where we are. When the dust settles we will have a bevy of COVID-19 clinical trials listing trackers, with patient recruitment matching capabilities, with a number of patient registries, and de-identified patient-level health data sets that don’t talk to each other? The industry has that now, it’s just not COVID-specific. On top of that, is the clear need for integrated real-world evidence to help with both clinical care response, as well as informing therapeutic intervention in real-time.

	Our Closing Thought op-ed contributors, Ülo Palm, MD, PhD, senior vice president, digital sciences for 

Allergan, and Iya Khalil, PhD, co-founder and CCO, GNS Healthcare, urge the use of AI to get to those more informed evidence and therapeutic decisions, not only in COVID, but moving forward as a whole in drug development. Co-founder and CEO Michelle Longmire, MD, of Medable, in our Q&A, envisions a way forward in drug development where decentralization of trials will lead to less expensive studies that can capture more long-term outcomes data, as well as increased participation, to boost innovation in therapies. 

	Clearly, we need change. Industry experts and observers have been saying that for years and many calling for disruption. Well, this is disruption. Is industry going to go back to its old ways, or will the pandemic result in lasting positive change?

With the bevy of coronavirus-specific information, databases and trackers, registries, and other tools out there at the moment, the challenge in store is integrating these resources to help in both clinical care response and informing drug intervention in real-time. 

Applied Clinical Trials-05-01-2020