Peter Schiemann, PhD

About a year ago, TransCelerate began working on a clinical trial Quality Management Systems (QMS) initiative, and 

. TransCelerate recently published a few articles in the Therapeutic Innovation & Regulatory Science journal (DIA), with 

one publication specifically focusing on issues management in quality management systems

. In this Applied Clinical Trials article, we will summarize our interpretation of the findings from the DIA publication, and analyze the publication, and evaluate potential issues during QMS framework implementation. 	 	

	 	According to the authors, good QMSs define Key Risk and Performance Indicators (KPIs/KRIs), established documentation systems based on risk levels that are based on data, proper notification systems and plans, corrective and preventative action (CAPA) initiation and resolution capabilities, and analytics to track improvements in the QMS

.	 	The authors introduced another concept that seems to expand on 

TransCelerate’s definition of critical data

: triaging ‘issues that matter’ (i.e., problems that impact data integrity, patient rights and safety, confidence in ethical clinical trial conduct, and regulatory requirements

). By filtering issues that matter from issues that don’t, sponsors can enhance and optimize resource utilization and allocation during clinical program administration. Moreover, this model allows managers to quantify studies that exhibit high levels of risk based on issue category frequencies.	 	

	 	Another directive introduced in the publication involves pinpointing the issue, and preventing reoccurrence. Study teams must conduct a thorough root cause analysis, which requires setting proper documentation and impact analysis procedures on patient safety and data integrity assessments, and exercising investigative analysis to uncover other areas of impact.	 	The authors argue that while the intention of CAPAs are to eliminate issues that matter, the activity is insufficient without a ‘CAPA effectiveness check

,’ which basically ensures CAPA resolution and measuring its impact on eliminating issues that matter.	 	

	 	Another concept introduced in the article involves the ability to incorporate clinical IT systems to enhance QMS oversight. For example, incorporating the planning, and execution of the QMS through technology-based workflows and forms allow management to collect data, generate visualizations to better pinpoint the types of issues that arise, and evaluate the effectiveness of the QMS over time.	 	Additionally, the incorporation of workflows through clinical IT systems empower management to assign stakeholders, which allows for accountability, bottleneck identification, and stakeholder performance assessments

	 	The authors write that TransCelerate is developing tools to support the above process. Further clarifications of the process are certainly needed. For instance, the publication does not give any definitions on how to perform the triage on the basis of objective criteria and not subjective assessments. Moreover, the term ‘materially impact’ needs a clear definition. The three examples provided on Page 2 of the publication are all severe, however, they are not comparable in their nature: to elaborate, in two cases, a grave mistake has already been made (informed consent issue, and the changes in the database) whereas in the other example, there have not been any negative consequences for patients and data, even though not sending the Serious Unexpected Adverse Reactions (SUSARs) to the sites is a grave mistake.	 	In addition, to standardize the Root Cause Analysis and the use of Clinical IT in issues management processes, more details about the processes need to be provided and guidance should go beyond the examples that have been offered.	 	Furthermore, the proposed QMS infrastructure does not consider incorporating self-improving QMSs through the use of clinical technology at study sites. One of the largest investment assets in clinical trials is operationalizing the study site. Incorporating quality feedback systems (fueled by clinical technology) that enhances communications between Sponsors and sites, facilitate site staff to automatically resolve quality issues as they arise, rather than depending on one-off engagements with CRAs. Moreover, a self-improving QMS not only saves a tremendous amount of resources, but it also enables the Sponsor to ameliorate quality oversight, and extends stakeholder accountability all the way down to site personnel.	 	Finally, the important area of vendor oversight is not addressed at all. Deficiencies in vendor oversight often lead to compliance issues and is now becoming both FDA and EMA inspector priorities. 	 	

	 	While the publication has proposed a solid and comprehensive issues management methodology via a QMS, there are general complications that sponsors must consider during execution. Specifically, defining and interpreting KPIs/KRIs can result in reporting inaccuracies, or misfires that would impact the quality of management analytics and visualizations of the QMS program. For example, Sharon Reinhard, President of Clinical Compliance indicated 

 that issues interpretation varies vastly when a CRA looks at an issue versus an auditor; 

a CRA can completely miss a protocol violation, whereas an auditor would capture it by questioning unusual data during an audit

. Accordingly, it could be a challenge to properly quantify clinical program risk due to the subjective nature of GCP, and ‘issues that matter’ interpretation.	 	

	 	The TransCelerate paper offers plenty of robust discussion on issues management, however, the process needs to include specific details and enhanced methodologies in order to comprehensively cover all aspects of a solid QMS. We are excited to see how the TransCelerate tools will develop.	 	Moe Alsumidaie, is a regular contributor to and member of 

 Editorial Advisory Board. Beat Widler, PhD, and Peter Schiemann, PhD are Managing Partners at Widler & Schiemann Ltd.	 	 	

		Susan Callery-D’Amico, BSN, RN, Leslie M. Sam, BA, Timothy H. Grey, BS, and Daniel J. Greenwood, MS 

TransCelerate’s Clinical Quality Management System: Issue Management 

Available at http://dij.sagepub.com/content/early/2016/07/21/2168479016657129.full.pdf?ijkey=zz1ebAKYemDCZXQ&keytype=infinite. Accessed September 6, 2016.

Articles

TransCelerates work on their clinical trial Quality Management Systems (QMS) initiative began about one year ago. This article summarizes findings from a recent DIA publication about the initiative and evaluates potential issues during QMS framework implementation.

Discussion on TransCelerate’s QMS Initiative

	The biopharmaceutical industry is starting to adopt TransCelerate’s Risk Assessment Categorization Tool (RACT) in order to identify risks and plan a comprehensive clinical trial risk mitigation strategy. We recently wrote about the 

, and the advantages of using such systems. Some of these advantages include the ability for study teams to evaluate R&D portfolio risks by collecting and analyzing RACT data in aggregate. In this article, we will leverage industry-wide RACT data (provided by Cyntegrity) to see what are the most common study-related risks that people choose. 

	In October 2015, Cyntegrity launched @RACT, a free cloud-based RACT tool that has been deployed across the industry. Over the span of seven months, the @RACT database collected 2,185 RACT data points from 28 RACTs from about 20 small and large pharmaceutical companies. It is important to emphasize that this data cannot be used as a reference for a particular trial, it demonstrates only the global trends in risk assessments, and may not truly represent actual study risks; it is highly advisable for study teams to evaluate their own study risks by completing their own RACTs.
	 

	If you are new to the RACT, the tool essentially analyzes risk from 13 categories (Figure 1), and helps study teams uncover critical study risks in a simple, structured, and standardized way.

	Study teams are required to assess Impact, Probability and Detectability measures from a variety of questions from each category in the RACT, in order to generate a risk score for that category. Figure 2 demonstrates that clinical trial Endpoints, Complexity, and Blinding posed the highest risk scores (in red) compared to other categories.

	Factors impacting Study Endpoints include endpoint data collection methods, and whether the study is event or outcome driven. Factors affecting Study Complexity include uncommon procedures beyond usual standard of care, subject burden, Adverse Events (AEs) requiring adjudication, and the number of sites and subjects. Factors impacting Blinding involve blinding setup, assignments, where blinded IP is created, and risk of unblinding.

Patterns with Probability, Detectability and Impact

	The rating system of the RACT requires study teams to categorize risk in each of the questions by Probability (how likely a risk is going to happen), Detectability (how easy it is to detect a risk), and Impact (extent of damage/consequences of a materialized risk).

	Figure 3 illustrates the effect of Probability, Detectability and Impact (risk measures) on average risk scores. Figure 3 shows that Probability has the highest impact on risk scores. However, it is important to delineate two observations; (a) each risk measure has a different effect on risk scores; for example, Low Probability is associated with the lowest average risk score, and vice versa for High Probability, and (b) the slopes differ for each risk measure.

	Analytical trends with this phenomenon introduces variability in risk score measurement, and this might be explained by the notion that the structure of RACT design suggests subjectivity in its assessments by allowing study teams to come to their own conclusions and interpretations of certain risks in specific risk categories.

	One weakness of the RACT is that many answers to the questions are left to the judgment of the individual and if you have two different teams complete the RACT on the same study, you would most likely get two different risk assessment outcomes.

	TransCelerate’s RACT was initially developed with the intention for study teams to identify risks at the beginning, or during study set up and during study conduct. RACT was designed with project management requirements in mind, as the categories are focused primarily on study operations. This approach is not wrong, however, it is easy to lose focus on the requirements of a risk assessment that regulators would want to see. To elaborate, when one looks at the safety section, it is clearly visible that the authors have given this a lot of thought, nevertheless, essential questions are missing, such as, "does the protocol contain clear definitions of adverse events and clear guidance on safety reporting?" or "how often does the study team plan to reconcile the clinical trial database against the serious adverse event database?" In addition, no quantitative reference is asked for but assessments are made on the basis of merely subjective – team driven – criteria. Further, questions about disclosure are missing.

	There are more examples, which show that project management requirements regarding timelines, meeting endpoints and budget were drivers of this RACT design rather than compliance. Nonetheless, the structured risk assessment does cover many aspects of compliance pertaining to data integrity and patient safety.

	An initiative that can standardize risk assessments (integrated into @RACT), was developed by PPH plus (questionnaire “RACT plus”), which includes tangible criteria on better defining Impact, Probability and Detectability for each of questions in RACT. This clarification tool is aimed to reduce subjectivity during the risk evaluation.

	Moreover, it is important to consider including fixed analytical weights to each question and category, since certain categories are always more important than others. However, leaving RACT scoring and weighting up to individual teams opens the door to inconsistency and hampers protocol / project to protocol / project reproducibility. Henceforth, the most consistent way for weighting may involve linking the weights to historical successes and failures for each trial, retrospectively. Cloud-based solutions enhance the ability to conduct aggregate assessments.

	In summary, RACT provides the first essential step towards the standardization of risk assessments in clinical trials. However, the analysis in this article demonstrates that the structure of RACT questions infers subjectivity. 

	Standardized criteria for defining “low”, “medium” and “high” for each risk measure, “Impact”, “Probability” and “Detectability” in the assessments as well as fixed analytical weighting can enhance the consistency of risk assessment outcomes.

	While the global risk assessment trend demonstrates that study personnel tend to give higher risk assessment scores to categories, such as “Endpoints” and “Complexity,” and “Blinding,” these risk categories may not properly reflect actual study risks due to subjectivity, and more data is needed in order to see consistency in categorical trends. 

	On the way to future perfection, it is important for the industry to evaluate risks that regulators pay particular attention to, such as disclosure, safety reporting, and clarifying AE detection. In addition, a section on outsourcing and vendor oversight would be a reasonable addition, as well as ethics and regulatory questions.

	While the RACT has created advancements in allowing study teams to interpret risk, it still has a long way to go and needs to undergo continuous transformation and real-world testing in order to generate consistency in identifying study risk outcomes. 

The biopharmaceutical industry is starting to adopt TransCelerate’s Risk Assessment Categorization Tool (RACT) in order to identify risks and plan a comprehensive clinical trial risk mitigation strategy. We recently wrote about the RACT moving to the cloud, and the advantages of using such systems. Some of these advantages include the ability for study teams to evaluate R&D portfolio risks by collecting and analyzing RACT data in aggregate.

Data from Global RACT Analysis Reveals Subjectivity

oncology trials are taking longer to complete

, and the article suggested that the duration of oncology clinical trials in certain phases increased by one year to one and a half years.

 Moreover, the article delineates that ameliorations in trial duration were most likely a result of increasing protocol complexity/trial design, numerous endpoints, increases in the number of subjects required, and longer treatment periods.

	However, industry experts have a different viewpoint as to why these trials are increasing in duration. This article will evaluate the duration of oncology clinical trials from an aggregated standpoint, using the 

, and will elaborate on different perspectives as to why oncology studies are increasing in duration.

Oncology Clinical Trials are Increasing in Duration

	In order to confirm findings from the aforementioned article, we evaluated median oncology industry sponsored clinical trial duration in all oncology disease modalities. We leveraged 

 aggregated data from 4,849 cancer clinical trials from 2002-2014, and findings are illustrated in 

Figure 1: Median Trial Duration: Oncology Trials (Source: 

	From an aggregated standpoint, median oncology clinical trial duration increased by 69% from 2002-2014, an average increase of 16 months. Phase I – IV cancer trial duration increased by 83% (Phase I), 19% (Phase II), 100% (Phase III), and 120% (Phase IV), respectively*.

 further dissects this data by categorizing oncology trial duration by a few disease modalities.

Figure 2: Trendline Analysis of Oncology Trial Duration by Disease (Source: 

illustrates that the duration of colon cancer trials increased the most, by 178%, whereas blood (60%), breast (50%), lung (42%), and solid tumors (22%) exhibited similar trends, accordingly*.

Research Leading Towards Why Cancer Trials are Increasing in Duration

	According to the literature, protocols are becoming more complex, more data is required, and subject enrollment timelines are, correspondingly, delayed as a result of increasing protocol complexity.

 For example, a study conducted on aggregated clinical trials in a variety of phases suggests that procedural frequency grew by 8.7% on an annual basis, study subject questionnaires increased significantly in Phase III and IV studies, and the average number of inclusion criteria skyrocketed by more than three times.

Additionally, study site work burden grew by 10% on an annual basis, and the median number of adverse events increased by 122%.

	Other studies evaluating Phase III protocols from 2002-2012 suggest that the total number of study endpoints has surged by 86%, procedures have risen by 58%, and eligibility criteria grew by 61%, number of investigative sites have increased by 58%, yet, the number of patients randomized declined by 18%.

However, We May Be Overlooking a Critical Factor: Antiquated Endpoint Design

 will elaborate on another perspective regarding why oncology trials are increasing in duration.  The reason why oncology trials in particular are increasing in duration is because standard of care (SOC) treatment options are so good at prolonging life, however, regulators and payers still require that an oncology study’s primary endpoint focus on patient death.

	To demonstrate, a multiple myeloma diagnosis used to be a death sentence before novel treatments underwent clinical trials; and patients that have undergone those trials in are still living today.  So, now that these drugs have become the SOC, newer oncology therapies are now experiencing patient populations that are living longer compared to before, or are not dying; and this factor is, subsequently, increasing oncology trial duration for studies that focus on death as the primary endpoint.

How Can Regulators, Payers and the Biopharmaceutical Industry Change?

	Regulators, payers and the biopharmaceutical industry need to rethink oncology trial endpoint design to focus on demonstrating improvements and value depending on the disease modality; especially in earlier cancer settings where there are now a significant number of patient alive 5 years after diagnosis such as breast cancer or colon cancer.

	Both regulators and payers need to exercise flexibility in endpoint requirements; maintaining the status quo would hinder novel oncology drug development, as these drugs will experience a prolongation to market and thus significantly slow down innovation in a therapeutic area with continuing high unmet need.  Moreover, the biopharmaceutical industry needs to think very carefully about their study design, in order to off set the longer duration by reducing complexity and focusing on data collection that really matters.

http://www.appliedclinicaltrialsonline.com/oncology-trials-are-taking-longer

http://csdd.tufts.edu/_documents/www/2816Getz.pdf

http://www.appliedclinicaltrialsonline.com/kadmon-ceo-discusses-biopharma-innovation

http://www.afmr.org/multimedia/2009/Clinical-Research-Conference/jim200301.pdf

	http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000601.jsp&mid=WC0b01ac05807d58ce

An article recently appeared in Applied Clinical Trials, which indicated that oncology trials are taking longer to complete, and the article suggested that the duration of oncology clinical trials in certain phases increased by one year to one and a half years.1 Moreover, the article delineates that ameliorations in trial duration were most likely a result of increasing protocol complexi

Why Are Cancer Clinical Trials Increasing in Duration?

	In a previous article, we discussed details on how to define clinical trial data quality.  The aforementioned article emphasized that a reduction in data quality ultimately leads to enrolling more clinical trial subjects in order to maintain equivalent statistical power.  In this article, we elaborate on three categories impacting clinical trial data quality.

	Environmental factors typically relate to the protocol’s design and the scope of study visit interventions, such as devices and human methods that measure and generate study data. These study measurement origins can create levels of data variances.

	For example, measuring the circumference of a lymphedema patient’s arm with measuring tape may vary from one coordinator to another; one coordinator may measure the arm with slack in the measuring tape, whereas another may compress the tape around the patient’s arm, both of which yield variances in data measurements; this can be risky if you have more than one coordinator taking the same measurements on one patient.  Another scenario can include differing brands of imaging devices which generate varying data formats, noise frequencies and imaging quality that can be interpreted differently by a medical reviewer. 

	How to mitigate study design risks:  Think about the protocol design and how you can standardize data measurement across the study. If you plan to use a medical device to conduct study measurements, investigate the device’s standardization specifications to uncover potential differences in data quality and frequency. If the measurement technique requires human intervention, consider implementing training programs that delineate appropriate measurement techniques, and limiting the amount of coordinators taking measurements per patient.  Minimizing data variances will likely reduce the required clinical trial sample size.  

 provides an example of standardizing and managing medical device use for measuring outcomes in clinical trials.

	Have you ever asked a clinical trial patient whether excessive sweating was an adverse drug reaction?  Most likely they’ll think excessive sweating is unrelated to medical product use. According to 

an analysis of FDA’s Adverse Event Reporting System

, adverse events are largely underreported.  Additionally, T.J. Sharpe, a cancer patient said 

, “before I started participating in clinical trials, I never paid attention to or worried about reporting adverse events of FDA approved medications.”   

	Oftentimes, inexperienced clinical trial participants will neither pay attention to, nor recognize the  importance of reporting adverse drug reactions in clinical trials.  Further, the doctor-patient barrier introduces risks associated with AE noncompliance; “Healthcare professionals aren’t out there to detect adverse events, and I had issues with the dialogue between healthcare professionals,” said Sharpe.  AE noncompliance can contribute towards increasing data variances (which reduces statistical power, requiring more patients), and lead towards generating non comprehensive AE profiles.

	How to Mitigate AE Noncompliance risks: Implement communications programs with patients and their families to instruct them on how to detect and report adverse events, and to remind them about the importance of timely adverse event reporting; verifying timeliness can reduce statistical variances associated with expected AEs).  You may also want to consider establishing AE reporting systems (i.e., call in centers, or web forms), and incorporate 

WHO’s guidelines for successful AE reporting system implementations

	As much as we would like to think that patients are honest and truthful, the aforementioned comments by Sharpe demonstrate that compliance requirements in medical settings pressure patients, and they subsequently become untruthful, or exercise misconduct. For example, a patient may indicate that they adhered to investigational medical product regiments, however, patient vitals and reported outcomes data would confirm nonadherence; 30% of patients either throw out or do not take investigational medical product in clinical trials.

forget to take investigational medical  product, and double dose thinking they’re catching up, however, such activities lead to increased toxicity levels, and can result in overdose-related adverse drug reactions; coupled with patient untruthfulness, medical adjudicators can, subsequently, draw irreconcilable conclusions.  Other forms of nonadherence include a lack of patient involvement in the trial, such as not adhering to patient reported outcomes requests and not appropriately following study visit procedures. 

	Nonadherence leads to inconsistency, inconsistency leads to larger statistical variances, and subsequently, enrolling more patients.  The causes of patient nonadherence can range from mental stability/health, race and language barriers, access to transportation, financial issues, to name a few,

 but, it is important to recognize that patients are people with busy lives, and researchers should not expect patients to be 100% adherent.

	The case study above demonstrates that while communications programs can assist with improving adherence, mobile technologies cross many barriers (including cultural and language) and are proven to work.  Multinational mHealth studies in Asia, for example, have shown that regular engagement through patient mobile devices have improved medication adherence, reduced nonadherence related symptoms, and even minimized the number of nonadherence related adverse events compared to control groups in healthcare settings.

Mitigating Nonadherence with Protocol Optimization, Patient/Investigator Centricity and mHealth:  Midata Case Study

 will elaborate on how protocol optimization techniques, patient involvement and mHealth can mitigate patient nonadherence while improving data quality:

	Involve patients in clinical trials! Not just as a “patient,” but as a stakeholder and “active participant.” Ask patients to help with designing protocols that meet their needs (especially in long-term studies), make patients understand that the data they are providing will lead to better 

medicines, offer them  smart apps and other tools to support study execution, and include them in clinical trial training programs.  Involving patients may take more time, but the outcome will outweigh the time invested.

	There is an ongoing pilot that is evaluating patient involvement in trials at The Department of Neurology of the Zurich (Switzerland) University Hospital with technical support from the Swiss Federal Institute of Technology (ETH-Z, Eidgenössische Technische Hochschule) Zurich.

		The concept of this innovative approach to clinical trials involves the following main elements:

		The study protocol is written with direct patient advocacy group input to ensure patients’ priorities are addressed.

		Through this approach, advocacy groups share protocol summaries (in lay language) with prospective patients.

		If interested, patients are asked to join the Midata Cooperative – a non-profit– and consent to offering the capture of all of their trial data.

		Through the Midata platform, subsequently, patient eligibility is evaluated, and if eligible, the patient is invited to ask his/her treating physician to become an investigator.

		Sophisticated baseline assessments, such as MRIs, will be conducted at the University Clinic while routine assessments are left to the treating physician to minimize patient burden.

 patients will receive push and pull information to drive compliance to the treatment regimen and to clarify gaps or “oddities” in the clinical data.

		Real time communication between sponsor and patients is enabled through the Midata platform.  The firewalled platform is fully compliant with data privacy and protection laws and regulations, and it allows the sponsor to submit queries to anonymized trial participants, and to resolve those data discrepancies and queries without bothering the investigator with trivia while the investigator is also kept in the loop as s/he has access to the section of the Midata platform that is related to the trial.

		Centralized monitoring tools enable the sponsor to conduct data integrity assessments and verify outliers.  If gaps or discrepancies are discovered the sponsor can follow up directly with the patient, reducing investigator administrative burden.

	The goal of this pilot trial is to demonstrate that sponsors can improve data quality and lower costs by a more systemic involvement of patients in routine clinical trial activities and by converting non-trial participating physicians into principal investigators.

	1. Patient Non-adherence in Clinical Trials: Could There Be a Link to Post Marketing Patient Safety? Dorothy L. Smith, PharmD, Drug Information Journal, 46 (I) 27-34, 2012

	2. Marian U.C. Worcester, Barbara M. Murphy, Virginia K. Mee, Susan B. Roberts and Alan J. Goble. Cardiac rehabilitation programmes: predictors of non-attendance and drop-out. European Journal of Cardiovascular Prevention & Rehabilitation 2004 11: 328

	3. Azfar-e-Alam Siddiqi, Alla Sikorskii, Charles W Given and Barbara Given. Early participant attrition from clinical trials: role of trial design and logistics. Clin Trials 2008 5: 328

	4. Sahar Khonsari, et al., Effect of a reminder system using an automated short messaging service on medication adherence following acute coronary syndrome.  Eur J Cardiovasc Nurs. published 02, February 2014

The aforementioned article emphasized that a reduction in data quality ultimately leads to enrolling more clinical trial subjects in order to maintain equivalent statistical power.

Three Categories Influencing Clinical Trial Data Quality

	The RbM Consortium is a transnational alliance of quality risk management industry experts, risk-based technology firms, data analysts, and biopharmaceutical business strategists. Members of this group include 

. The objective of this collaboration is to improve awareness of Risk-based Monitoring (RbM) in clinical trials, to advise biopharmaceutical enterprises, medical device companies, CROs, study sponsors, and study sites on best practices, and communicating the best implementation strategies on sound quality risk management systems, technologies, and business methodologies in RbM.  This guidance document will offer information about RbM strategies.

, data-driven monitoring or centralized monitoring have become interrelated terms of a hot topic, and there is hardly an organization that does not claim to already apply or considers to implement such an approach for clinical development and pharmacovigilance. This begs the question: is it just hype or a shift in paradigm? We are convinced that this is more than just hype. However, the shift in paradigm has not yet happened either. We-the 

RbM Consortium, which includes recognized experts from this area-

observe many approaches to a risk-based study management but came to the conclusion that it is too early to celebrate an advent of a new era in clinical trials and development. We notice that the majority of approaches proposed lack fundamental elements of Quality by Design and Quality Risk Management strategies.

Lack of a uniform and broad understanding of underlying principles, methodology and approach:


		A shared understanding of terminology, scope of tasks, deliverables, as well as roles and responsibilities across the entire stakeholder community involved in the planning, conduct, analysis, reporting, and assessment of clinical trials-e.g., sponsors, contract research organizations (CROs), investigators (and, as a result, patients)-is important to the realization of a risk-based study management approach.
		 

Decisions about risks are not based on soundly defined objective decision criteria:


		Risk assessments and their resulting decisions today are mostly based on the evaluation and opinion of teams or individuals. What happens if those teams or individuals are replaced by others? Will their decisions be the same? This is highly doubtful, since individuals base their decisions on their experience and the situation they are currently in. As important experience is for establishing a risk-based system, independent decision criteria or decision frameworks are needed to keep the human factor as small as possible once the system goes live.
		 

Emphasis is not put on the foundation of risk-based monitoring:


		The foundation of a sound risk-based approach starts much earlier than the trial itself with aspects such as protocol design, study start-up assessments, site qualification, patient enrollment optimization, to name a few.
		
		The common approach focuses only on a centralized review of incoming data or a strategy of reduced source document verification (SDV) and site monitoring but protocol design aspects, which are the blueprint for the trial, are neglected. As a result, protocols of inadequate design (structure and content) are implemented with a study management approach that is likely to be unfit for the complexity of the protocol.
		 

No integrated quality strategy is developed and implemented:


		Such strategy must include study design, site selection (investigators who have the required therapeutic and Good Clinical Practice (GCP) experience as well as access to the patients to be enrolled), study management (centralized/data driven monitoring) and general oversight aspects (quality management plan, third party oversight, audits, etc.).
		 

Risk assessments are conducted in silos and not shared within and between sponsors:


		In order to save time and money, sharing of assessments of “common” entities, such as CROs, central laboratories and others can contribute to a leaner and more effective approach to managing clinical trials. In addition, networks of service providers should be created to serve as one-stop shop for a sponsor of a clinical study covering all elements from protocol design to reporting of trial results including disclosure, submissions, training, qualifications, risk assessments, safety duties and many more.

	Based on the above-mentioned deficiencies we have put together a catalogue of ten key aspects on how to navigate QbD, QRM and RbM and to implement a sound 

1. Is it true that only larger pharma or device companies can apply a risk-based approach?

	Designing and implementing a risk-based approach does not depend on the size of a company. Smaller biotech, pharmaceutical and device companies as well as academia can do it, too. All will benefit as much as Big Pharma from a smart approach to monitoring. We all face the same challenges: lack of resources, time pressure and growing regulatory requirements.

	It is important that the designed solutions to RbM are meeting the criteria defined above in our statement. There are many solutions to RbM, whether they are built by using a spreadsheet or supported by a highly sophisticated IT system with automated data analysis. The more clinical studies and the more complex they are, i.e., global, multiple arms, comparators, etc., the better would be a solution that is technically sound and to a high degree automated. However, this is not needed in all scenarios. We will explain it further in the following.
	 

2. Do I need sophisticated IT tools to implement a risk-based approach?

	A risk-based approach starts with the identification of risks followed by the analysis as to their impact, likelihood and detectability, and the possible actions to mitigate them. For such an assessment, no sophisticated IT tools are required. The tracking of risk profiles – at the sponsor, CRO or clinical trial center level – and of actions triggered by the risk assessments can be performed with off-the-shelf office tools (e.g., spreadsheets). If larger volumes of data need to be assessed, more sophisticated web-based tools may be user-friendlier.

	However, when clinical trial data are to be regularly reviewed for outliers, errors or other non-compliances, and large volumes of data need to be processed; this would require the use of a validated and well-designed computerized system to minimize workload. In addition, data need to be presented in reports that are easy to understand, classified in risk categories that are based on objective criteria, and result in actionable mitigating actions.

3. Is it true that we need to change the way we write protocols and set up trials to implement successfully a risk-based approach?

	There is consensus that the current approach to clinical development is not sustainable due to the following reasons:

		About 50% of sites fail to reach the planned recruitment targets.

		The majority if not all milestones missed in the clinical trials-more than 95% of clinical trials do not end on time and on budget as planned in the first place.

		90% of the studies meet their recruitment goals, but at the expense of mostly twice as much time as originally planned.

 The costs attributable to these delays are gigantic.

		Too many avoidable protocol amendments, with a first amendment implemented even before the very first patient has been enrolled-most of them due to the rush towards “First-Patient-In”.

		Compliance flaws causing delays and additional costs.

		Bottom line: trials take much longer and cost much more than originally planned.

	A senior inspector summarized recently her reservations about the current approach as follows: “If sponsors and CROs do not learn to write better protocols, moving from the current trial monitoring approach to a risk-based or centralized monitoring approach, it is likely to end in a compliance disaster.”

	We are also absolutely convinced that the quality of protocols needs substantial improvement, for example, by implementing the following plan changes:

		Focusing on one study rationale and one or two primary objectives.

		Verifying with real-life data that inclusion criteria are fit for purpose and support the value proposition of the new product.

		Verifying that the study set-up is aligned with patients’ and investigators’ priorities and capabilities through patient-centric models and simulations.

		Verifying with real-life data that enrollment plans (recruitment goal and rate) are realistic and the right sites are being involved

	We should never forget that the rationale and objectives of a protocol should target a clear value proposition and that the key stakeholders of a protocol are not the editors of a scientific journal but the professionals performing reviews at health authorities and Ethics Committees/Institutional Review Boards (EC/IRB) prior to clinical trial approval or in the context of Health Technology Assessments (HTA) for the decision on pricing and reimbursement.
	 

4. What are KRIs and KPIs, and how can they support a risk-based approach?

	KRIs are used to measure in an objective manner the risk imposed on a process or system. We distinguish leading and lagging KRIs. A leading KRI measures parameters that indicate a problem building up before it materializes so there is still time to correct. Lagging KRIs are indicating deviations that already happened, but as a single event would not have a great impact, however, when piling up, impose a risk on the process/deliverables looked at. Generally, leading KRIs are more effective but are also more difficult to measure than lagging ones. A proper suite of KRIs combining leading and lagging KRIs allows a study management team to plan and execute a risk-based strategy, and KRIs can be benchmarked via empirical data analyses.

) is an example of a leading KRI. A delay in on-site monitoring does not per se threat patients’ safety, integrity and rights, and data integrity but increases the risk of tardy detection of a GCP issue, and as such it is sensible from a study oversight perspective to address such delay in a proactive manner. 

is an example of a lagging KRI: the GCP deviation has already occurred and although the deviation cannot be corrected its recurrence can be prevented.

	KRIs can be designed as “digital” triggers, i.e., the KRI is either 

: when the threshold is reached the KRI “fires” and generates a signal. This approach, rather than having “multi-level” KRI thresholds, avoids complexity in the calculation logic and also facilitates the “validation” of the threshold. Indeed, any threshold needs to be justified by means of objective criteria. Validation of a KRI threshold can in essence be achieved in three ways:

	a)   Thresholds taken from regulatory, SOP, industry, etc. standards, e.g., the KRI for processing a serious adverse event would be set at day 13 to avoid non-compliance with the 15 days reporting timelines.

	b)   Analyzing compliance data from past studies, i.e., analyzing the behavior of a KRI threshold in studies known for good compliance vs. studies with recognized GCP deficiencies.

	c)   Analyzing “real world” data, e.g., analyzing Electronic Health Records (EHR) to get clues about realistic expectations vs. protocol requirements. For instance, EHRs can be analyzed to determine how frequently and to what extent a patient’s visit deviates from the scheduled visit date.

	d)   If none of the above approaches can be applied, an arbitrary threshold can be set and the adequacy of the threshold can be verified or fine-tuned later through on-site monitoring and auditing approaches.

	When defining KRIs study teams must practice caution to measure what has an impact on compliance and process effectiveness and not what can easily be measured. For instance, measuring compliance against timelines may induce team members to take short cuts, which can be the root cause of new deficiencies.

	Reports aggregating KRI analyses data can be presented via tables, graphic mapping, odometers, and other analytical visualizations.

5. Will a risk-based approach make site selection/qualification and patient recruitment into my trials any easier, and will it help to get better data faster and cheaper?

	Risk-based approaches apply not only to monitoring, but also to many facets of a clinical trial, such as site selection/qualification, protocol design and subject enrollment/retention. Sponsors can mitigate clinical trial risks in the following areas:

Predict Study Site Enrollment and Quality Potential Prior to Initiation:

 Initiating poorly performing study sites is common and is costly. Up to one third of study sites never enroll a single patient 

, and it can cost up to 50,000 USD per site to seek, initiate, maintain and close poorly performing sites 

. With the availability of numerous data sources (i.e., patient populations, number of ongoing clinical trials at an investigative site, competitive trials, site resources, years of investigator experience, etc.), we advise sponsors/CROs to conduct empirical analyses and run predictive models against historical site performance to unveil factors that affect quality and enrollment outcomes. Quantifying site enrollment and quality potential prior to site initiation enables sponsors to minimize exposure to poorly performing sites, which saves costs and improves quality outcomes.

 According to FDA and EU inspectors, a well-designed protocol is the blueprint for quality clinical trials, and many studies experience timeline slippage and compliance issues because of poorly designed protocols, which result in numerous amendments, and costing more than 400,000 USD per amendment 

. In order to minimize timeline slippage, we advise sponsors/CROs to test protocols against real-time aggregated Electronic Medical Records data. This test enables sponsors/CROs to optimize protocol inclusion/exclusion criteria to better match real-world patient medical criteria, which can minimize enrollment slippage and the number of protocol amendments.

 Missing clinical trial data due to subject dropouts has a significant impact on clinical trial timelines, budgets and quality outcomes. Reasons for subject dropout can vary from subject geo-demographics, protocol complexity, subject commitment, education, distance from study site, employment status, physical activity and much more 

. Albeit there are technologies that focus on engaging patients, many are unproven in the realm of healthcare and clinical trials. We advise sponsors/CROs to not only test out feasible technologies, but also engage patients through validated technologies, such as SMS (Short Messaging Service, also known as Text Messages), which is a proven and cost-effective way to improve medication adherence 

, and motivate patients to stay in a clinical trial.

6. Will a risk-based approach allow me to stop SDV (source document verification) and reduce the burden of on-site monitoring?

	With RbM, source document verification (SDV) is not becoming obsolete but the way it is applied needs to change as outlined below. The FDA clearly states, “No single approach to monitoring is appropriate or necessary for every clinical trial. FDA recommends that each sponsor defines a monitoring plan that is tailored to the specific human subject protection and data integrity risks of the trial. Ordinarily, such a risk-based plan would include a mix of centralized and on-site monitoring practices.”

 Therefore, through the systematic use of a centralized review of the study data we can consider a reduction in the burden of on-site monitoring, improvements in clinical data quality and an increase in the clinical monitors’ effectiveness. All sources of data should be mined and analyzed such as the data from the CRF, its metadata (e.g., audit trail data), the CTMS, the TMF, safety database, etc. When such an approach to study management is applied, SDV will serve as a root cause analysis tool when needed rather than a data comparison tool. We should always keep in mind that the goal of any monitoring activity – regardless of the tools used – is to protect patients’ safety, integrity and rights as well as ensuring data integrity. A proactive approach to study oversight increases efficiency by reducing the need for corrective actions.

7.  How much money do I save by implementing a risk-based approach?

	
	Depending on the complexity of a clinical study, the costs of monitoring (personnel, travel, expenses, etc.) can reach magnitudes of 25-30 % of the clinical trial budget.

 Since quite some time this has been a target for cost-saving exercises at many pharmaceutical and biotech companies. However, none has actually achieved the goal of cost reduction in this budget line item. With risk-based monitoring, however, many believe that now has come the time, and finally the tools are available, to reduce the monitoring budget.

	Although certain tasks can now be done centrally, there is still a need for visiting the centers to ensure that patients’ safety and the integrity of the data are observed. Therefore, the current monitoring effort that is rather distributed equally will be shifted towards sites in greater need, such as centers with many patients, not experienced in conducting GCP trials, potential problems indicated through KRIs and central data analysis/risk assessment, etc. This means that we will rather see a shift in effort than a reduction of it. However, the positive result from this shift is that the sites in need now get the attention and support necessary to help them to deliver good results.

	Savings in the long run will be realized when through well planned and implemented clinical studies the timelines to data base lock and ultimately to the launch of the product will be shortened, which will ensure longer patent exclusivity. When we take a medicinal product with annual sales of 500 million USD and divide those annual sales by 365 days, we can see that the real savings of 1.65 million USD per day are waiting for those who set up their studies in the QbD spirit.

8.  How will Health Authorities react if they discover a major or critical finding that was not detected or not addressed through your risk management approach?

	We have discussed this aspect multiple times with Health Authorities’ inspectors and their feedback was clear and consistent with regulators’ messages about RbM: “errors in clinical trials are acceptable to regulators as long as had we perfect data we would still make the same decision and come to the same conclusion.” In other words, inspectors will continue to detect and report non-compliances, however if inspectors and reviewers come to the conclusion that through those non-compliances patients’ safety, integrity and rights as well as data integrity were not compromised, a trial/system/process will still be considered compliant in the sense that the goal of GCP (patients’ safety, rights, integrity, and data integrity) was reached. Obviously, even if such findings were not considered as critical, the sponsor must, however, implement adequate CAPAs.

	Additionally, the risk management approach has the advantage of knowledge formalization and incorporation of the critical findings into the future risk-monitoring process, which leads to continuous improvement.

9. How does a risk-based approach impact on my organization, i.e., study site, Quality Assurance, Data Management, Clinical Operations, Drug Safety, Biometrics, etc.?

	Our experience shows that change management activities around the development and rollout of an RbM/QRM strategy in study management and oversight are key to success. A “must do” is early communication, honestly and proactive alignment of the entire organization with the plans, goals and – most importantly – the organizational and people implications.

	Unfortunately, still today, a silo approach to study oversight activities is the norm. Within a sponsor company, Clinical Operations develops monitoring plans, QA audit plans and Clinical Science develops plans about the benefit–risk assessments. Resources are allocated by functions and not in an integrated manner. For instance, the data management budget may allow developing an EDC and data management solution but does not include resources for programming and maintaining KRIs. Similarly, the biostatisticians may be tasked to develop a Statistical Analysis Plan (SAP) but do not have resources to develop programs for the early detection of outliers or abnormalities in the data structure. When CROs are involved, the wealth of insight about process efficiencies and deficiencies are rarely shared across their clients to generate learnings and trigger process improvements, too.

	Therefore, when a company decides to adopt an RbM approach, a holistic, integrated risk-based study management plan needs to be developed. From this follows that all stakeholders need to work together to identify the required tools, activities and minimum tolerable error levels. The latter is nothing else than the definition of the “design space” for a given study and the justification with objective evidence of that “space.” RbM requires a change in mindset and, therefore, change management that is introduced at the very beginning of an RbM project is key to its success.

10. What is the best implementation strategy for a risk-based approach?

	Our experience has shown that a systematic introduction of an RbM approach rather than a big bang launch is the best implementation strategy for such a project. Indeed, a big bang strategy is likely to overwhelm an organization, underestimates the technical and organizational challenges and thus triggers resistance amongst concerned staff members. A four-step approach has proven multiple times to work best:

Step 1. Predictive Modeling and Empirical Analysis:


	In the predictive modeling and empirical analysis phase, study teams need to (a) determine clinical operations/business research objectives, and (b) conduct empirical research on data from numerous completed clinical trials to uncover factors that impact clinical trial risk and performance outcomes, for example, factors that predict clinical trial failure. Once teams identify risk factors, we advise teams to conduct statistical analyses to determine analytical parameters, which can be used as benchmarks, and to predict future outcomes. After developing a portfolio of statistically linked factors and analytical expected outcomes, study teams can use these parameters to proceed with the proof of concept phase (step 2), and examine the application of these predictive models in real time.


	In the PoC phase, the tools, methodologies and processes needed to implement a Quality by Design (QbD) and RbM approach are developed and applied in one to a maximum of three studies with teams that are eager to move to a modern RbM methodology. Involving people enthusiastic about RbM is essential as it allows overcoming hurdles and backslashes without being forced to fight skepticism. This PoC generates important learnings on how to implement RbM in the sponsor organization. Specifically, on how to address gaps, conditions for success and limitations to RbM imposed by external constraints. For instance, it can give an overview of the capabilities of current IT systems that cannot be easily fixed. The PoC also delivers “success stories” that can be shared within the sponsor organization as a preparation for step 2. Typically, in the PoC there is little IT involvement and the tools developed and employed use simple technology such as structured questionnaires built in spreadsheets.