OR WAIT 15 SECS
The eClinical Forum Risk Based Monitoring Taskforce offers some best practices for ensuring clinical data quality.
Companies engaged in commercial clinical research have, since the introduction of formalized good clinical practices (GCP), adopted highly defensive practices around the monitoring of clinical trials. The reasons, while diverse, include fear based on perception and interpretation of regulatory requirements. These processes have played a very large role in the high costs associated with clinical trials, without delivering commensurate value. Reasons are many, but surely include propagation of practices established during the years that trials relied on paper-based case report forms (CRFs) without considering optimization of the processes to take advantage of the technology provided by electronic data capture (EDC). All of this has resulted in the institutionalization of high cost, unproven value, manpower-intensive practices that do little to serve the interests of subject safety or quality data. Pharmaceutical, biotech, and medical device industries experiencing unprecedented economic upheaval can no longer justify practices that deliver little value at exorbitant cost. Rather than continuing to spend time, energy, and money focusing on minutiae (e.g., the ability of site personnel to properly transcribe observations from one medium to another), those engaged in clinical trial conduct must re-focus their energies on the aspects of the clinical trial that matter most, such as protocol compliance, subject safety, data timeliness, and data integrity. Through the use of more up-to-date methodologies and technologies, companies can effectively address these concerns while saving both time and money. This article addresses the regulatory and business rationales for adopting monitoring-related tools and processes that, if implemented thoughtfully, should deliver higher-quality trial data, faster, and at significantly lower cost.
Most areas of business and government-including finance, insurance, public health, agrochemicals, manufacturing, and pharmacovigilance-apply risk-management principles, leveraging mathematical principals (namely statistical inference) in circumstances where it is simply impossible to ensure the quality of a large volume of services or products by exhaustingly checking the accuracy of every item. Clinical research-related industries, operating under GCP, appear as a notable exception for the following reasons:
Quality risk management has been rather limited in the area of GCP. Until recently, research and development personnel (as well as their representatives/CROs) have believed that they were required to engage in 100% verification of all records and reports obtained in the context of Phase I-IV clinical research.1
These deeply ingrained beliefs have led to practices that are not only prohibitively expensive, but also fail to deliver value that can justify these costs. As a result, the pharmaceutical and device industries have institutionalized practices and behaviors that deliver low value at exorbitant cost. These costs include both the direct and indirect expenses associated with monitoring investigative sites, as well as the opportunity costs associated with these antiquated practices. Exhaustive manual verifications have shown their limits in particular when it comes to identifying (in a timely manner):
The almost concurrent release in 2011 of draft guidance documents from two of the three major GCP regulatory agencies related to "risk-based monitoring" has triggered numerous discussions among clinical research operatives and peers, including sponsors of drug and device clinical research, contract research organizations (CROs), and academic clinical research organizations.
A survey conducted by the Clinical Trials Transformation Initiative7 indicated that there was a large and somewhat inconsistent range of monitoring practices during the conduct of clinical trials. The survey found that monitoring practices vary in intensity, focus, and methodology and include:
For major efficacy and safety trials, companies typically conduct on-site monitoring visits at approximately four to eight week intervals, at least partly because of the perception that frequent on-site monitoring with 100% source document verification (SDV) is the regulator's preferred way for sponsors to meet their monitoring obligations. In addition, overall source document review (SDR) remains as an on-site monitoring obligation. However, FDA also recognizes that data from critical outcome studies (e.g., National Institutes of Health-sponsored trials, Medical Research Council-sponsored trials in the United Kingdom, and International Study of Infarct Survival), which had no regular on-site monitoring and relied largely on centralized and other alternative monitoring methods, have been relied on by regulators and practitioners.
The general nature of the guidance documents is a potential gift to sponsor organizations. It encourages the industry to define strategies for risk-based monitoring, and perhaps create standards, that can be customized for individual protocols. Sponsors can now take full advantage of EDC as centralized monitoring tools are developed to capture intra- and inter-site outcomes, as a quality window on GCP.
The unrealistic fear of regulatory agency findings at the time of pre-approval inspections leads to a more conservative approach that is both necessary and required. A paradigm shift must take place whereby sponsors realize that inspection findings are not to be feared, but are ways to improve processes, and collaborate with the regulatory agencies to better define clinical trial operations and patient safety.
The eClinical Forum in the context of its chartered mission to serve the pharmaceutical and device industries focuses on those systems, processes, and roles relevant to electronic data capture, data handling, and regulatory submission of clinical trial data. The eClinical Forum has already submitted comments directly to both the FDA and EMA, and released a white paper to promote awareness of the concepts shared by both regulatory documents. By doing so, the eClinical Forum can assuage any apprehension expressed by the more traditionalist organizations which continue to interpret the regulations and predicate rules as calling for full verification and monitoring. We propose that by discussing the key elements of the guidance documents and by proposing best practices on actual tasks (whats and how-tos), we can greatly contribute to demystifying and adopting risk-based quality management of clinical trials. This approach should also allow the regulatory authorities to get access to honest, unbiased, substantiated previews of the organizational and procedural response by industry stakeholders as well as provide previews that are the product of reflective listening to regulatory expectations and that underline the challenges and work effort required by organizations as they adopt, adapt, and measure up to the guidance documents. As a result, the eClinical Forum should contribute actively to the further alignment between the regulatory agencies and the pharmaceutical and device industries.
Although the guidance document was assessed versus previous and current corporate procedures and best practices, it was necessary or timely to advocate specifically how individual companies might choose to implement changes or adaptations to their quality/risk management. We expect that the roles within each organization that will be asked to perform those tasks will differ, with some companies deciding to assign responsibilities to monitoring, programming or data management job profiles.
From its inception, EDC provided an opportunity for clinical research operations to re-evaluate the tasks traditionally performed by the data management and site monitoring roles. Moving towards adoption of integrated quality risk management systems calls for further, bolder, and continuous adaptation of systems, workflows, and people. Moving from the ingrained notions of traditional source document review and point-to-point SDV to one of quality processes (focus on optimizing site processes, QC of site compliance, detection of fraud, IMP distribution chains, etc.) will encourage us to:
Risk is someone or something that creates or suggests a hazard. Sponsors of clinical investigations are required to provide oversight to ensure adequate protection of the rights, welfare, and safety of human subjects and the quality and integrity of the resulting data submitted to regulatory agencies; in other words, avoidance of a hazardous situation.2 As a result, risk must be defined and maintained across all stakeholders.
Quality is degree of excellence. According to the FDA guidance, quality is a systems property that must be built into an enterprise and cannot be achieved by oversight or monitoring alone.1 Quality is something that needs to be maintained throughout the lifecycle of the product. The evaluation of the risk to quality should be based on scientific knowledge and connected to the protection of the patient.
There is a need to pause to consider the difference between quality control and quality management. Quality control allows the ability to outline a process to spot check data and operation by reviewing a slice in time. Quality management on the other hand, is used to direct, control, and monitor quality as a more fluid and dynamic activity.
The ICH Q9 guideline merges the two control processes as it speaks to quality risk management as a standard practice.3 Q9 provides guidance with examples that attempt to outline risk management across the pharmaceutical industry. Figure 1, taken from the EMA Reflection Paper,4 is an excellent starting place for organizations to turn as they design quality risk management; a systematic process for the assessment, control, communication, and review of risks to the quality of the drug (medicinal) product across the product lifecycle.
The ICH Q9 and EMA reflection paper guidance documents provide good direction on what to address when crafting a quality risk-management plan. However, neither truly addresses regulatory expectations for minimal requirements to accomplish this in a holistic manner. While it is one thing to talk about managing risk through traditional data management and site monitoring processes, it is quite another to propose a concrete way to manage risk through trend setting (electronic transparency on entry time, account use, etc.) and standard metrics (time between tasks, accuracy of input, etc.). Variability in the extent to which organizations approach quality risk management may confuse the situation even more. Fear of having decisions not meet regulatory approval also may keep some companies from updating their approach to quality management. In addition, the payoff may not be as large as indicated.
While there is no clear indication that quality is inadequate today, there is evidence that the industry overspends to perform tedious reconciliation between source and sponsor data that do not significantly affect the outcome of clinical trials.5
Traditional monitoring plans outline what the sponsor considers to be their responsibility and action plan for ensuring site compliance with the protocol including GCP and other applicable regulations. We should consider the value of combining elements of monitoring, data management, and statistical plans into a comprehensive integrated quality plan. It is important not to jeopardize the relationship between sponsor and the clinical sites and care should be taken to ensure that this remains in the forefront of the plan. For example, a communication plan is useful to document standard communication when there is no need for a site visit. This could be used to praise the site for good work and maintain monthly communication. Minimally, a consistent point of contact should be assigned to the site to represent the face of the sponsor.
It is important to emphasize that monitoring plans are not unique to the CRA role but also can provide an overall cross functional plan that emphasizes what must be done in order to assure that the data received from clinical research sites are of high quality and integrity. The monitoring plans include not only what is expected to occur at a site level, but also within the data system and program level.
A quality monitoring plan, which can be part of the clinical monitoring plan, may include information from traditional plans that had previously been separated into role or task based activity, for example, each therapeutic area may be different so several quality plans may be needed on that basis. In considering the monitoring plan, adequate risk assessment must be performed prior to the start of the study, in order to ensure patient protection as well as the quality and integrity of data. Our recommendation is that monitoring plans should exist at the program, protocol, and site levels, and outline what will be included in on-site monitoring as well as central monitoring.
Program level plans. This plan addresses overall quality aspects of the program and should be developed at the start of the clinical program. Risks that are known or anticipated for the disease population and investigational product under study should be documented and clear instruction given as to what, why, and how those risks should be monitored. As a general outline, a program level plan could minimally include the following:
Companies may wish to consider creating standard quality checks pertinent to all programs, and which may also include therapeutic area specific sections to be selected as relevant. Ensuring that the entire study team understands the principles and purpose of the program level monitoring plan will help avoid duplication and encourage study teams focus effort where it is most effective.
This is an ideal opportunity to take a step back from familiar routines and look hard at current processes; assess what is valuable and what is not; invest in processes that bring most value from a quality perspective. The goal is to develop an integrated clinical data monitoring plan, based on compound and historical knowledge, which can respond proactively to data signals during trial execution.
Study level plans. This plan should be specific to a particular protocol and draw upon the information from the program level plan as to what is needed to ensure quality. We suggest that the study level plans are categorized as those activities that occur during study startup, study execution, and study closure. This plan allows flexibility for site level variations, (e.g., highly compliant and performing sites, new sites with little experience, high volume sites, and non-compliant sites). The following specific minimal list of topics should be addressed in such a plan:
By focusing on study level tasks, rather than roles, study managers will be able to assign responsibilities to the most appropriate team member(s). This adds flexibility to the process, while ensuring that there are no omissions. It also will allow for role changes/combinations in the future. Plans should be easily readable and referable-limit bulk text; use graphics and bullet points. If a suitable electronic tool can be used, it should be feasible to pull out a sub-set of tasks for each function on the study from the overall plan.
Site level plans. This plan should be specific to each investigative site, combining information from Program/TA level plans and the protocol(s) that the specific site is involved with. There is a large degree of variety in the way regional, country, or hub monitoring units are being organized, therefore a critical aspect of the risk assessment will be the concurrent (and possibly competitive) workload and commitments that shape the possibilities and constraints of both the sponsor/CRO and site staff when collecting and reviewing data. Another key consideration in defining a site level risk management plan is to never project any task or activity over time as being flat or a product of the volume of data flow only. What's more, it is an oversimplification (with potentially disastrous results) to even consider data quality as being inversely proportionate to its volume. Some of the minimally required aspects to cover in the site level plan are:
Of paramount importance is a clear documentation and escalation strategy to the study and program teams. Escalation should not be considered to be a "for information" activity only as it risks accentuating instead of overcoming the compartmentalization between central and distributed monitoring management. The documentation of all issues and corrective actions as well as the elaboration of preventive planning at the site level is therefore not hierarchically dependent or "trailing" to the program and study level plans.
Risk identification and mitigation works in both directions; it can well be that a monitor works on more studies within the compound than a study manager and their perspective is of vital importance in the success of a clinical development strategy.
In addition to including the information described above, the clinical data monitoring plan should contain details to categorize risk, thereby allowing the sponsor to have clear criteria by which to trigger site visitation and remediation.
Triggered monitoring adopts three key approaches that differ from traditional monitoring. Rather than:
Utilizing categories and ranking of clinical data, based on experience and empirical findings, sponsors can compute a composite risk score that will quickly and easily point them to potential issues. A high risk score does not automatically mean that a particular site is problematic, but it serves as an indicator to look further into the particular items of note.
These categories should be defined by the sponsor and include but are not limited to:
It is recommended that study teams include a triggers and metrics section in their protocol level plan to provide detail that are used to evaluate when an increase in on-site monitoring visits should occur. While certain triggers may result in an immediate on-site visit, such as a major safety or GCP concern; other factors may require review and interpretation by the monitoring team and could result in increased training or site contact/coaching to supplement on-site visits. The plan should address the weight of the trigger and resulting outcome and actions to be taken by the monitor. The list below highlights areas related to site quality, data integrity and subject safety for consideration by the study teams. It is not expected that these metrics would be included in the CSR.
Site activity and personnel:
Important measures of safety:
Important measures of efficacy:
Suspected misconduct or fraud:
As well as triggers that indicate when a site monitoring visit should occur, there also are triggers to indicate when the percentage of SDV should increase or decrease for a specific site. Although some items on this list overlap with the list above, these triggers are more specific to the data authenticity and integrity and can identify potential training issues, compliance, and fraud.
Triggers to increase the percentage of SDV currently being performed at a site:
Clinical data and processes that should always be subject to more monitoring:
Scoring can serve as a way to find a potentially problem site and at minimum take a look at the core issues being identified. It may be possible then to apply common formulas such as failure mode and effects analysis or sponsor specific risk assessments as well as normalization factors. Scoring is not the only way to define when and why on-site monitoring should be performed. It is up to the sponsor to determine how to use the triggers and metrics discussed in this section. Risk indicators can be applied to the site performance, applied by the monitor, and/or an overall composite site assessment.
Examples of risk indicators are:
Although the protocol level plan will include details to help assess when on-site monitoring visits should increase or decrease, it is important to establish a mechanism to ensure monitors are following the plan and are compliant with the process. Since this will be such a paradigm shift for monitors it may be a difficult adjustment for them and they will need training, support, and continuous oversight to reinforce these changes. Creating SOPs and training materials will be essential to the success of risk-based monitoring and steps should be taken to verify consistency among teams. An example of how this could be done is by including time during regular team meetings for monitors to review and provide status updates on their sites as well as any actions they are taking as a result of their in-house review. This would provide an opportunity for the monitors to learn from each other and at the same time allow their manager to validate that the team is following the plan that has been established and that the proper actions are being performed as a result of their findings. It is imperative that the monitors review the data continuously so they can detect trends and create remediation plans when necessary and a team review such as this will ensure accountability and compliance.
Risk-based monitoring includes centralized and on-site monitoring of source records, which are two complementary activities that ensure data quality and integrity. Overall centralized and risk-based data monitoring activities need to:
However, from a process stand point, centralized monitoring is intended to be performed remotely from the investigational site while SDR and SDV are to be completed on-site. Centralized monitoring may include comparing data across multiple sites and countries while SDV is focused on one site and one variable at a time. As a result, systems have to adjust to both process needs.
High-level system requirements for centralized monitoring should include:
High level system requirements for on-site monitoring should include:
What approaches support centralized review? There are a number of central data review roles that may vary by sponsor. These include pharmacovigilance, clinical research, medical monitoring, data management, data science (DS), and remote CRA. Additionally, review of metrics may fall upon the biostatistics/data management group in cooperation with clinical operations in order to assure implementation of the on-site and centralized monitoring strategies per the clinical data monitoring plan.
Requirements of this multi-leveled array of data should include roles-based controls on data across multiple data platforms including CTMS; customized vendor reports (IXRS and ePRO, for example); EDC database and reports; safety database; raw data outputs; metadata summaries; data listings; dashboard displays; and validated SAS tables, listings, and figures. These roles also need to be configured by study as well. Since there are many sources of data, thought needs to be given to making the access simple and secure-and ideally, accessible remotely.
Systems supporting centralized review:
What systems support do you need to have a reasonable level of comfort with centralized review? In order to implement centralized review, the sponsor should have a good understanding of what attributes are required to evaluate current systems and capabilities and determine the need for any further enhancements. Systems/process attributes that may be required for centralized review:
Are our recommendations consistent with the guidance papers? In the "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring" FDA draft guidance,1 there is clear support for innovating the monitoring practices. In fact, in this guidance it is stated that by moving away from 100% SDV, we can "improve a sponsor's ability to ensure the quality and integrity of clinical trial data." The section on centralized monitoring (IV.A.1) suggests specific types of data review that are most suitable for central review (these include ranges, missing data, unusual statistical distributions, and site quality metrics). Along with the support for innovation, there also comes the burden of documenting how we will achieve these improvements. In section IV.B, the guidance emphasizes the importance of starting with the protocol to define the critical efficacy and safety endpoints and to then follow through with a clear data monitoring plan, quality plan, and risk mitigation plan in order to document with what frequency and intensity these endpoints and other critical data points will be followed throughout the trial. Additionally, the monitoring plan should cover both on-site and centralized monitoring activities. Ideally, the monitoring plan will describe what method of monitoring will be employed for the pre-identified critical data and also how any findings will be documented and communicated.
Some considerations for planning and documenting monitoring tasks:
In this article, we have laid out the case for re-examining the standard monitoring processes that have been in place for many years, and which have not been able to evolve past paper-based clinical trials. Since the inception of the use of EDC, our industry has had the opportunity to fundamentally shift our overall thinking regarding what we mean by data quality, and the processes and tools that we employ for ensuring clinical trial data fit for purpose. For a variety of reasons, monitoring processes have not been able to fully benefit from technology's ability to enhance efficiency, effectiveness and data quality.
This article provides very useful detail regarding the elements to consider in developing newer, risk-based (alternatively known as data-driven) approaches to clinical trial monitoring. At a high level, though, the drivers are quite simple: we can do better; and we must do better, for the costs associated with maintaining status quo processes are not sustainable. With the support from regulatory bodies, we have a wonderful confluence of opportunity and incentive-recent regulatory guidances have encouraged our industry to adopt more rational processes, and that by doing so we better position ourselves to shorten timelines, enhance data quality, and dramatically reduce costs.
While contemplating the contents of this article, it is important to keep in mind that to obtain the most benefit, we must be diligent in following through on our plans. If the results of your organization's risk assessment raise a red flag about the competency of (and associated data quality arising from) one of your sites, this awareness produces no value in and of itself. Its value derives from having plans for dealing with the results, and the intent to implement those plans.
One key element to adopting new monitoring practices is transparency. We must ensure that our study protocols include sufficient detail about how we will ensure the quality of our data and the integrity of our trials.
To make the kinds of quantum leaps required of us, we will need to overcome native fears and move from our comfort zones. We should take some comfort, though, that the recent guidance documents reflect serious regulatory intent to help move our industry forward. For example:
The "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring" FDA draft guidance,1 states that by moving away from 100% SDV, we can "improve a sponsor's ability to ensure the quality and integrity of clinical trial data."
The inference is clear-100% SDV represents a hindrance in the eyes of FDA to achieving data quality and trial integrity. We have opportunity, incentive, and a reasonably clear path for making significant enhancements in overall performance by adopting the kinds of plans and tools described in this article. And, we have every reason to believe that by doing so we can speed up clinical trials, improve data quality and dramatically reduce costs.
Editor's Note: A white paper on this topic is available for downloap at http://www.eclinicalforum.org/Default.aspx?id=76&tabid=59.
Julie M. Brothers is Senior Clinical Data Project Manager at Array BioPharma. Dean A. Gittleman is Senior Director of Operations at Target Health. Thomas Haag is Data Integrity Process Expert at Novartis. Darlene Kalinowski* is Associate Director eClinical Operations at Bristol Myers-Squibb, e-mail: Darlene.Kalinowski@bms.com . Ioannis Karageorgos is Senior Protocol Data Manager at Bristol Myers-Squibb. Robert King is Senior Director, Global Functional Service Provision at PPD. Lisa Lucero is Consultant RBM at Eli Lilly. Debbie McCann is Sr. Director, Client Accounts and Quality Practices at Pharmapros. Paula McHale is Senior Director of Product Management at Perceptive Informatics. Jules Mitchel is President at Target Health. Patrick Nadolny is Vice President Data Management and Programing at Allergan. Selina Sibbald is Strategic Alliance Director at Quintiles. Brett Wilson is Associate Director, Business Operations at Bristol Myers-Squibb.
*To whom all correspondence should be addressed.
1. Department of Health and Human Services, "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring," (2011), http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf.
2. Code of Federal Regulations Title 21, "21 CFR part 312, subpart D generally (Responsibilities of Sponsors and Investigators)" and "21 CFR part 812, subpart C generally (Responsibilities of Sponsors)," http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=312&showFR=1&subpartNode=21:220.127.116.11.3.4.
3. EMA, "Quality Risk Management (ICH Q9)," (2011).
4. EMA, "Reflection Paper on Risk Based Quality Management in Clinical Trials," draft (2011), http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2011/08/WC500110059.pdf.
5. K. A. Getz, "Flying Blind on CRA Workload, Time Demands," Applied Clinical Trials, 21 (7) 22-24 (2012).
6. "Risk-Based Implementation of Trial Regulation: An Overview of Key Activities," Clinical Research Advisor, (2012), http://index.canarybooks.com/LinkPages/PDF/Advisor%20Issues/Advisor%20302.pdf|>http://index.canarybooks.com/LinkPages/PDF/Advisor%20Issues/Advisor%20302.pdf.
7. B. Morrison, C. Cochran, J. Giangrande, et al., "Monitoring the Quality of Conduct of Clinical Trials: A Survey of Current Practices," Clinical Trials, 8, 342–349 (2011).