Risk-Based Approaches

Companies engaged in commercial clinical research have, since the introduction of formalized good clinical practices (GCP), adopted highly defensive practices around the monitoring of clinical trials. The reasons, while diverse, include fear based on perception and interpretation of regulatory requirements. These processes have played a very large role in the high costs associated with clinical trials, without delivering commensurate value. Reasons are many, but surely include propagation of practices established during the years that trials relied on paper-based case report forms (CRFs) without considering optimization of the processes to take advantage of the technology provided by electronic data capture (EDC). All of this has resulted in the institutionalization of high cost, unproven value, manpower-intensive practices that do little to serve the interests of subject safety or quality data. Pharmaceutical, biotech, and medical device industries experiencing unprecedented economic upheaval can no longer justify practices that deliver little value at exorbitant cost. Rather than continuing to spend time, energy, and money focusing on minutiae (e.g., the ability of site personnel to properly transcribe observations from one medium to another), those engaged in clinical trial conduct must re-focus their energies on the aspects of the clinical trial that matter most, such as protocol compliance, subject safety, data timeliness, and data integrity. Through the use of more up-to-date methodologies and technologies, companies can effectively address these concerns while saving both time and money. This article addresses the regulatory and business rationales for adopting monitoring-related tools and processes that, if implemented thoughtfully, should deliver higher-quality trial data, faster, and at significantly lower cost.

Most areas of business and government-including finance, insurance, public health, agrochemicals, manufacturing, and pharmacovigilance-apply risk-management principles, leveraging mathematical principals (namely statistical inference) in circumstances where it is simply impossible to ensure the quality of a large volume of services or products by exhaustingly checking the accuracy of every item. Clinical research-related industries, operating under GCP, appear as a notable exception for the following reasons:

• The honest and honorable intent to ensure the highest possible quality and safety for products that have a direct impact on the life and well-being of us all

• The pre-GxP beginnings of pharmaceutical research based on the use of paper CRFs

• Misinterpretations of regulations

• Fear of failure based on results of regulatory inspections

Quality risk management has been rather limited in the area of GCP. Until recently, research and development personnel (as well as their representatives/CROs) have believed that they were required to engage in 100% verification of all records and reports obtained in the context of Phase I-IV clinical research.¹

These deeply ingrained beliefs have led to practices that are not only prohibitively expensive, but also fail to deliver value that can justify these costs. As a result, the pharmaceutical and device industries have institutionalized practices and behaviors that deliver low value at exorbitant cost. These costs include both the direct and indirect expenses associated with monitoring investigative sites, as well as the opportunity costs associated with these antiquated practices. Exhaustive manual verifications have shown their limits in particular when it comes to identifying (in a timely manner):

• Issues related to protocol compliance

• Safety related signals and trends across large datasets

• Intra- and inter-rater variability

• Potential fraud

The almost concurrent release in 2011 of draft guidance documents from two of the three major GCP regulatory agencies related to "risk-based monitoring" has triggered numerous discussions among clinical research operatives and peers, including sponsors of drug and device clinical research, contract research organizations (CROs), and academic clinical research organizations.

A survey conducted by the Clinical Trials Transformation Initiative⁷ indicated that there was a large and somewhat inconsistent range of monitoring practices during the conduct of clinical trials. The survey found that monitoring practices vary in intensity, focus, and methodology and include:

• Frequent, comprehensive on-site visits to all clinical investigator sites by company personnel or representatives (e.g., clinical monitors or clinical research associates)

• Targeted on-site visits to higher-risk clinical investigators (e.g., where centralized monitoring indicates problems at a site)

• Centralized monitoring of clinical data by clinical, data management, and statistical personnel at some location other than the study site

For major efficacy and safety trials, companies typically conduct on-site monitoring visits at approximately four to eight week intervals, at least partly because of the perception that frequent on-site monitoring with 100% source document verification (SDV) is the regulator's preferred way for sponsors to meet their monitoring obligations. In addition, overall source document review (SDR) remains as an on-site monitoring obligation. However, FDA also recognizes that data from critical outcome studies (e.g., National Institutes of Health-sponsored trials, Medical Research Council-sponsored trials in the United Kingdom, and International Study of Infarct Survival), which had no regular on-site monitoring and relied largely on centralized and other alternative monitoring methods, have been relied on by regulators and practitioners.

The general nature of the guidance documents is a potential gift to sponsor organizations. It encourages the industry to define strategies for risk-based monitoring, and perhaps create standards, that can be customized for individual protocols. Sponsors can now take full advantage of EDC as centralized monitoring tools are developed to capture intra- and inter-site outcomes, as a quality window on GCP.

The unrealistic fear of regulatory agency findings at the time of pre-approval inspections leads to a more conservative approach that is both necessary and required. A paradigm shift must take place whereby sponsors realize that inspection findings are not to be feared, but are ways to improve processes, and collaborate with the regulatory agencies to better define clinical trial operations and patient safety.

The eClinical Forum in the context of its chartered mission to serve the pharmaceutical and device industries focuses on those systems, processes, and roles relevant to electronic data capture, data handling, and regulatory submission of clinical trial data. The eClinical Forum has already submitted comments directly to both the FDA and EMA, and released a white paper to promote awareness of the concepts shared by both regulatory documents. By doing so, the eClinical Forum can assuage any apprehension expressed by the more traditionalist organizations which continue to interpret the regulations and predicate rules as calling for full verification and monitoring. We propose that by discussing the key elements of the guidance documents and by proposing best practices on actual tasks (whats and how-tos), we can greatly contribute to demystifying and adopting risk-based quality management of clinical trials. This approach should also allow the regulatory authorities to get access to honest, unbiased, substantiated previews of the organizational and procedural response by industry stakeholders as well as provide previews that are the product of reflective listening to regulatory expectations and that underline the challenges and work effort required by organizations as they adopt, adapt, and measure up to the guidance documents. As a result, the eClinical Forum should contribute actively to the further alignment between the regulatory agencies and the pharmaceutical and device industries.

Although the guidance document was assessed versus previous and current corporate procedures and best practices, it was necessary or timely to advocate specifically how individual companies might choose to implement changes or adaptations to their quality/risk management. We expect that the roles within each organization that will be asked to perform those tasks will differ, with some companies deciding to assign responsibilities to monitoring, programming or data management job profiles.

From its inception, EDC provided an opportunity for clinical research operations to re-evaluate the tasks traditionally performed by the data management and site monitoring roles. Moving towards adoption of integrated quality risk management systems calls for further, bolder, and continuous adaptation of systems, workflows, and people. Moving from the ingrained notions of traditional source document review and point-to-point SDV to one of quality processes (focus on optimizing site processes, QC of site compliance, detection of fraud, IMP distribution chains, etc.) will encourage us to:

• Delineate QA from QC in a risk-based quality system.

• Identify standard approaches to centralized and on-site monitoring activities.

• Assess how information technology tools and associated processes can best support centralized and risk-based data review.

• Determine which metrics (thresholds/error rates) and alerts will drive the documentation of quality as well as trigger corrective actions when the quality thresholds are not met.

• Evaluate the impact on the preparation and planning for protocol-driven research.

What is risk and what is quality?

Risk is someone or something that creates or suggests a hazard. Sponsors of clinical investigations are required to provide oversight to ensure adequate protection of the rights, welfare, and safety of human subjects and the quality and integrity of the resulting data submitted to regulatory agencies; in other words, avoidance of a hazardous situation.² As a result, risk must be defined and maintained across all stakeholders.

Quality is degree of excellence. According to the FDA guidance, quality is a systems property that must be built into an enterprise and cannot be achieved by oversight or monitoring alone.¹ Quality is something that needs to be maintained throughout the lifecycle of the product. The evaluation of the risk to quality should be based on scientific knowledge and connected to the protection of the patient.

There is a need to pause to consider the difference between quality control and quality management. Quality control allows the ability to outline a process to spot check data and operation by reviewing a slice in time. Quality management on the other hand, is used to direct, control, and monitor quality as a more fluid and dynamic activity.

The ICH Q9 guideline merges the two control processes as it speaks to quality risk management as a standard practice.³ Q9 provides guidance with examples that attempt to outline risk management across the pharmaceutical industry. Figure 1, taken from the EMA Reflection Paper,⁴ is an excellent starting place for organizations to turn as they design quality risk management; a systematic process for the assessment, control, communication, and review of risks to the quality of the drug (medicinal) product across the product lifecycle.

Figure 1. Quality risk management is a systematic process for the assessment, control, communication, and review of risks to the quality of the product

The ICH Q9 and EMA reflection paper guidance documents provide good direction on what to address when crafting a quality risk-management plan. However, neither truly addresses regulatory expectations for minimal requirements to accomplish this in a holistic manner. While it is one thing to talk about managing risk through traditional data management and site monitoring processes, it is quite another to propose a concrete way to manage risk through trend setting (electronic transparency on entry time, account use, etc.) and standard metrics (time between tasks, accuracy of input, etc.). Variability in the extent to which organizations approach quality risk management may confuse the situation even more. Fear of having decisions not meet regulatory approval also may keep some companies from updating their approach to quality management. In addition, the payoff may not be as large as indicated.

While there is no clear indication that quality is inadequate today, there is evidence that the industry overspends to perform tedious reconciliation between source and sponsor data that do not significantly affect the outcome of clinical trials.⁵

Elements of the integrated quality plan

Traditional monitoring plans outline what the sponsor considers to be their responsibility and action plan for ensuring site compliance with the protocol including GCP and other applicable regulations. We should consider the value of combining elements of monitoring, data management, and statistical plans into a comprehensive integrated quality plan. It is important not to jeopardize the relationship between sponsor and the clinical sites and care should be taken to ensure that this remains in the forefront of the plan. For example, a communication plan is useful to document standard communication when there is no need for a site visit. This could be used to praise the site for good work and maintain monthly communication. Minimally, a consistent point of contact should be assigned to the site to represent the face of the sponsor.

Sample Table of Contents for an Integrated Risk Based Data Quality Plan

It is important to emphasize that monitoring plans are not unique to the CRA role but also can provide an overall cross functional plan that emphasizes what must be done in order to assure that the data received from clinical research sites are of high quality and integrity. The monitoring plans include not only what is expected to occur at a site level, but also within the data system and program level.

A quality monitoring plan, which can be part of the clinical monitoring plan, may include information from traditional plans that had previously been separated into role or task based activity, for example, each therapeutic area may be different so several quality plans may be needed on that basis. In considering the monitoring plan, adequate risk assessment must be performed prior to the start of the study, in order to ensure patient protection as well as the quality and integrity of data. Our recommendation is that monitoring plans should exist at the program, protocol, and site levels, and outline what will be included in on-site monitoring as well as central monitoring.

Program level plans. This plan addresses overall quality aspects of the program and should be developed at the start of the clinical program. Risks that are known or anticipated for the disease population and investigational product under study should be documented and clear instruction given as to what, why, and how those risks should be monitored. As a general outline, a program level plan could minimally include the following:

• Planning-a priori identification of risks of the investigational product.

• Design-define protocol eligibility criteria, efficacy, and safety risk factors during design; for inclusion in the eCRF and edit build, as well as define key protocol deviations anticipated for study.

• Execution-clear direction in the oversight and monitoring of parameters leading up to study endpoints, integrated study level monitoring plan, clear direction on evaluating fraud.

• Analysis-clear definition of data measurement and reporting, defined elements of statistical analysis of protocol endpoints, safety, etc.

• Disclosure-publish outcome data, bringing all the above together in the clinical study report (CSR).

Companies may wish to consider creating standard quality checks pertinent to all programs, and which may also include therapeutic area specific sections to be selected as relevant. Ensuring that the entire study team understands the principles and purpose of the program level monitoring plan will help avoid duplication and encourage study teams focus effort where it is most effective.

This is an ideal opportunity to take a step back from familiar routines and look hard at current processes; assess what is valuable and what is not; invest in processes that bring most value from a quality perspective. The goal is to develop an integrated clinical data monitoring plan, based on compound and historical knowledge, which can respond proactively to data signals during trial execution.

Study level plans. This plan should be specific to a particular protocol and draw upon the information from the program level plan as to what is needed to ensure quality. We suggest that the study level plans are categorized as those activities that occur during study startup, study execution, and study closure. This plan allows flexibility for site level variations, (e.g., highly compliant and performing sites, new sites with little experience, high volume sites, and non-compliant sites). The following specific minimal list of topics should be addressed in such a plan:

• Site selection factors-include key protocol requirements that affect decision making on site selection.

• Site training considerations-include information on how the site is prepared to conduct trial.

• Information on all forms of data review.

• Identification of critical data-safety, endpoint related.

• Delineation of what is reviewed on-site versus central.

• Site closure activities.

• Assessment of site understanding of protocol.

• Assessment of fraud.

By focusing on study level tasks, rather than roles, study managers will be able to assign responsibilities to the most appropriate team member(s). This adds flexibility to the process, while ensuring that there are no omissions. It also will allow for role changes/combinations in the future. Plans should be easily readable and referable-limit bulk text; use graphics and bullet points. If a suitable electronic tool can be used, it should be feasible to pull out a sub-set of tasks for each function on the study from the overall plan.

Site level plans. This plan should be specific to each investigative site, combining information from Program/TA level plans and the protocol(s) that the specific site is involved with. There is a large degree of variety in the way regional, country, or hub monitoring units are being organized, therefore a critical aspect of the risk assessment will be the concurrent (and possibly competitive) workload and commitments that shape the possibilities and constraints of both the sponsor/CRO and site staff when collecting and reviewing data. Another key consideration in defining a site level risk management plan is to never project any task or activity over time as being flat or a product of the volume of data flow only. What's more, it is an oversimplification (with potentially disastrous results) to even consider data quality as being inversely proportionate to its volume. Some of the minimally required aspects to cover in the site level plan are:

• Site experience in therapeutic area and/or compound.

• Studies by the same sponsor conducted concurrently (internal competition) and/or publicly available information on the site being involved in similar protocols/programs for other sponsors or being a sponsor of a research program themselves (external competition). Add risks about misconceptions on the objectives and endpoints of the different protocols and study populations. Avoid the trap of considering a group of protocols as a convenient early access program for the compound under investigation.

• Own monitoring resource workload and financial, schedule, and data access constraints.

• Site staff attrition.

• Site staff "fatigue" over time.

• External factors that can generate bias or change a site's standpoint for a compound (patient advocacy groups and/or social networks, regional regulatory and/or governmental changes in their stance towards therapeutic options and research, brand or generic competition pressure).

Of paramount importance is a clear documentation and escalation strategy to the study and program teams. Escalation should not be considered to be a "for information" activity only as it risks accentuating instead of overcoming the compartmentalization between central and distributed monitoring management. The documentation of all issues and corrective actions as well as the elaboration of preventive planning at the site level is therefore not hierarchically dependent or "trailing" to the program and study level plans.

Risk identification and mitigation works in both directions; it can well be that a monitor works on more studies within the compound than a study manager and their perspective is of vital importance in the success of a clinical development strategy.

Assessment of risk tolerance: triggers and metrics

In addition to including the information described above, the clinical data monitoring plan should contain details to categorize risk, thereby allowing the sponsor to have clear criteria by which to trigger site visitation and remediation.

Triggered monitoring adopts three key approaches that differ from traditional monitoring. Rather than:

• Having monitors conduct periodic monitoring visits based on a schedule rather than a need, site monitoring is undertaken in response to key metrics (e.g. data quantity, subject enrollment, and safety signals), which can be used to predict risk.

• Treating all sites the same, the triggered model recognizes that some sites pose more of a risk than others (e.g., by virtue of metrics such as workload, quality and safety issues, and patient enrollment and retention) and focuses attention on them accordingly.

• Treating all safety events equally, the triggered model catches safety signals as they happen and triggers a site visit.⁶

Utilizing categories and ranking of clinical data, based on experience and empirical findings, sponsors can compute a composite risk score that will quickly and easily point them to potential issues. A high risk score does not automatically mean that a particular site is problematic, but it serves as an indicator to look further into the particular items of note.

These categories should be defined by the sponsor and include but are not limited to:

• Trigger points for on-site monitoring and what requires SDV and SDR

• Information on monitoring priorities

• Definitions of monitoring assessment of site quality

• Definition of deviations occurring within the tolerance range as acceptable

It is recommended that study teams include a triggers and metrics section in their protocol level plan to provide detail that are used to evaluate when an increase in on-site monitoring visits should occur. While certain triggers may result in an immediate on-site visit, such as a major safety or GCP concern; other factors may require review and interpretation by the monitoring team and could result in increased training or site contact/coaching to supplement on-site visits. The plan should address the weight of the trigger and resulting outcome and actions to be taken by the monitor. The list below highlights areas related to site quality, data integrity and subject safety for consideration by the study teams. It is not expected that these metrics would be included in the CSR.

Site activity and personnel:

• Staff turn-over

Subject rates:

• Higher/lower enrollment than expected (ensure proper screening and validity of subjects)

• Higher/lower percentage of screen failures across sites

• High discontinuation rate across sites (ensure proper screening, adherence to the study procedures, subject retention)

Edit-checks/Queries:

• Site-to-site rate disparity

• Average time from query generation to response (could indicate training issues)

• Query rates (critical metric for defining thresholds which will flag sites for additional monitoring/auditing): high number of manually generated queries as a result of SDV and SDR (could indicate that additional transcription errors may exist); automatically generated queries via edit checks (could indicate poor data quality or error in edit check specification); re-query rates (could indicate training issues)

Important measures of safety:

• High occurrence of adverse events (AE) (potential safety concerns, proper study procedures, and reporting)

• Lower-than-average number of adverse events (review for unreported AEs)

• High number of SAEs (potential immediate trigger)

• Reconciliation issues from SAEs to CRFs

Important measures of efficacy:

• Unexpected data variability

• Unexpected data uniformity

• Outliers for specific fields and trending to see if the same data is entered for multiple patients (to identify fraud)

Data-entry timeliness:

• CRF entry compliance (centralized monitoring cannot be performed unless all data are entered in a timely manner)

• Pattern of delays

• Data entered in boluses rather than at a rate that reflects actual subject visits (see also "fraud")

Protocol compliance:

• Site audit findings

• High number of protocol deviations: as identified by the monitors and manually entered into CTMS; data driven from edit checks within EDC; study drug preparation, dosage, administration related deviations (potential immediate trigger)

• Review for patterns of deviations or violations demonstrating lack of understanding of protocol, or compliance failures

• High number of GCP issues or other site issues

Suspected misconduct or fraud:

• Repeated measures display unexpected uniformity, lack of variability, especially when coupled with higher-than-expected enrollment

• Data entered in boluses rather than at a rate that reflects actual subject visits (see also "data-entry timeliness")

As well as triggers that indicate when a site monitoring visit should occur, there also are triggers to indicate when the percentage of SDV should increase or decrease for a specific site. Although some items on this list overlap with the list above, these triggers are more specific to the data authenticity and integrity and can identify potential training issues, compliance, and fraud.

Triggers to increase the percentage of SDV currently being performed at a site:

• High rate of SDV errors where the CRF value does not match source

• High rate of data revisions for a field after the first submission (could indicate poor quality of data entry or possibly fraud)

• Data anomalies for primary endpoint values, study drug dosage, and calculations

• The number of data revisions due to manual queries (from SDV)

• Lower-than-average number of adverse events

• Audit findings for site audits (some audit findings might question the quality of the site which should then in turn question the quality of the data)

• Third party data reconciliation issues

Clinical data and processes that should always be subject to more monitoring:

• Critical study endpoints

• Protocol-required safety assessments

• Adherence to protocol eligibility criteria

• Maintenance of study blind

• Reporting of SAEs/UADEs, deaths and withdrawals

Assessment of risk tolerance: examples

Scoring can serve as a way to find a potentially problem site and at minimum take a look at the core issues being identified. It may be possible then to apply common formulas such as failure mode and effects analysis or sponsor specific risk assessments as well as normalization factors. Scoring is not the only way to define when and why on-site monitoring should be performed. It is up to the sponsor to determine how to use the triggers and metrics discussed in this section. Risk indicators can be applied to the site performance, applied by the monitor, and/or an overall composite site assessment.

Examples of risk indicators are:

• Critical-data related to any critical indicator weighs high in computing risk at a site.

• High-data related to a high-risk indicator has the potential to point to a site with risk potential.

• Medium-data related to a medium risk indicator has the potential to point to a site with possible risk potential.

• Low-data related to a low risk indicator has the potential to point to a site with possible risk potential or at minimum possible re-training needs.

Although the protocol level plan will include details to help assess when on-site monitoring visits should increase or decrease, it is important to establish a mechanism to ensure monitors are following the plan and are compliant with the process. Since this will be such a paradigm shift for monitors it may be a difficult adjustment for them and they will need training, support, and continuous oversight to reinforce these changes. Creating SOPs and training materials will be essential to the success of risk-based monitoring and steps should be taken to verify consistency among teams. An example of how this could be done is by including time during regular team meetings for monitors to review and provide status updates on their sites as well as any actions they are taking as a result of their in-house review. This would provide an opportunity for the monitors to learn from each other and at the same time allow their manager to validate that the team is following the plan that has been established and that the proper actions are being performed as a result of their findings. It is imperative that the monitors review the data continuously so they can detect trends and create remediation plans when necessary and a team review such as this will ensure accountability and compliance.

Systems and processes in support of risk-based and centralized data monitoring

Risk-based monitoring includes centralized and on-site monitoring of source records, which are two complementary activities that ensure data quality and integrity. Overall centralized and risk-based data monitoring activities need to:

• Increase quality (clearly identify and address risks)

• Ensure data origin, validity, quality, and accuracy

• Be verifiable, reproducible, and documented

• Be risk-based and reactive

• Be as easy as possible to implement

• Allow/support fraud detection

• Ensure ease of internal/external inspection of the source (sponsor audit/regulatory inspection)

• Be more efficient (better use of resources)

However, from a process stand point, centralized monitoring is intended to be performed remotely from the investigational site while SDR and SDV are to be completed on-site. Centralized monitoring may include comparing data across multiple sites and countries while SDV is focused on one site and one variable at a time. As a result, systems have to adjust to both process needs.

High-level system requirements for centralized monitoring should include:

• Integrating with existing systems (EDC, CTMS, etc.)

• Reviewing study data across sites (EDC, IVRS, ePRO, etc), coded or not

• Aggregating operational data (e.g., query metrics, data entry cycle time, screen failure rate, etc.) and study data (e.g., adverse events, protocol deviations, reasons for early study termination, etc.)

• Aggregating operational data across studies (e.g., look at site data across multiple studies)

• Generating basic statistics across data elements and data types

• Establishing role-based data reviews (e.g., able to limit access to aggregate data to those performing centralized monitoring)

• Setting thresholds for alerts (e.g., x times outside the standard deviation, Y% above or below study average)

High level system requirements for on-site monitoring should include:

• EDC systems supporting custom SDV strategies (declining SDV, complexity based SDV, form based SDV, etc.)

• On-site monitoring and SDV strategies for each study (SDV baseline-e.g. 100% for first three patients)

• Quantification of the allowable source error (i.e., SDV error rate: Percentage of transcription error made by the site)

• Adjustment of on-site monitoring strategy at the patient, site or country level based on quantified error rates (i.e., the actual "risk")

• Change and reason for changing monitoring strategies during the study

PAGE BREAK

Reflective review of systems versus the FDA guidance and EMA reflection paper

What approaches support centralized review? There are a number of central data review roles that may vary by sponsor. These include pharmacovigilance, clinical research, medical monitoring, data management, data science (DS), and remote CRA. Additionally, review of metrics may fall upon the biostatistics/data management group in cooperation with clinical operations in order to assure implementation of the on-site and centralized monitoring strategies per the clinical data monitoring plan.

Requirements of this multi-leveled array of data should include roles-based controls on data across multiple data platforms including CTMS; customized vendor reports (IXRS and ePRO, for example); EDC database and reports; safety database; raw data outputs; metadata summaries; data listings; dashboard displays; and validated SAS tables, listings, and figures. These roles also need to be configured by study as well. Since there are many sources of data, thought needs to be given to making the access simple and secure-and ideally, accessible remotely.

Systems supporting centralized review:

• Centralized electronic data capture systems (ideally)

• Standardized clinical database for all sites

• Standardized safety database for all sites

• CTMS

• Development and implementation of quality performance measurement system

• Standardized data (classes and methods) across sites and CROs

• Standardized training programs (for EDC and clinical conduct)

• Standardized data management (e.g., standardized archiving processes, data processing deadlines, etc.)

What systems support do you need to have a reasonable level of comfort with centralized review? In order to implement centralized review, the sponsor should have a good understanding of what attributes are required to evaluate current systems and capabilities and determine the need for any further enhancements. Systems/process attributes that may be required for centralized review:

• EDC: training and/or certification of reviewers; audit trail with ability to track multiple types of reviewers (roles differentiation); site/sponsor interoperability; system configurable to change monitoring processes during the study

• EDC/clinical and safety databases: safety reporting data listings; efficacy reporting data listings; other data listings as determined by data monitoring plan; query trend reporting; missing data reporting

• Risk management plan (i.e., assessment, identification, mitigation) and metrics

Are our recommendations consistent with the guidance papers? In the "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring" FDA draft guidance,¹ there is clear support for innovating the monitoring practices. In fact, in this guidance it is stated that by moving away from 100% SDV, we can "improve a sponsor's ability to ensure the quality and integrity of clinical trial data." The section on centralized monitoring (IV.A.¹) suggests specific types of data review that are most suitable for central review (these include ranges, missing data, unusual statistical distributions, and site quality metrics). Along with the support for innovation, there also comes the burden of documenting how we will achieve these improvements. In section IV.B, the guidance emphasizes the importance of starting with the protocol to define the critical efficacy and safety endpoints and to then follow through with a clear data monitoring plan, quality plan, and risk mitigation plan in order to document with what frequency and intensity these endpoints and other critical data points will be followed throughout the trial. Additionally, the monitoring plan should cover both on-site and centralized monitoring activities. Ideally, the monitoring plan will describe what method of monitoring will be employed for the pre-identified critical data and also how any findings will be documented and communicated.

Some considerations for planning and documenting monitoring tasks:

• Tasks/systems must support the criteria set out in the data monitoring plan, quality plan and risk mitigation plan as outlined in this article

• Specified frequency of review

• Specified review roles

• Documentation of drivers/triggers for reduced on-site monitoring visits and those requiring increased on-site monitoring visits

• Documentation of drivers/triggers for adjusting centralized monitoring activities

• Documentation for traceability of decisions made and actions taken. Greater reliance on CTMS and DMP/QP/RMP documentation demonstrating that we are following the set criteria

Conclusion

In this article, we have laid out the case for re-examining the standard monitoring processes that have been in place for many years, and which have not been able to evolve past paper-based clinical trials. Since the inception of the use of EDC, our industry has had the opportunity to fundamentally shift our overall thinking regarding what we mean by data quality, and the processes and tools that we employ for ensuring clinical trial data fit for purpose. For a variety of reasons, monitoring processes have not been able to fully benefit from technology's ability to enhance efficiency, effectiveness and data quality.

This article provides very useful detail regarding the elements to consider in developing newer, risk-based (alternatively known as data-driven) approaches to clinical trial monitoring. At a high level, though, the drivers are quite simple: we can do better; and we must do better, for the costs associated with maintaining status quo processes are not sustainable. With the support from regulatory bodies, we have a wonderful confluence of opportunity and incentive-recent regulatory guidances have encouraged our industry to adopt more rational processes, and that by doing so we better position ourselves to shorten timelines, enhance data quality, and dramatically reduce costs.

While contemplating the contents of this article, it is important to keep in mind that to obtain the most benefit, we must be diligent in following through on our plans. If the results of your organization's risk assessment raise a red flag about the competency of (and associated data quality arising from) one of your sites, this awareness produces no value in and of itself. Its value derives from having plans for dealing with the results, and the intent to implement those plans.

One key element to adopting new monitoring practices is transparency. We must ensure that our study protocols include sufficient detail about how we will ensure the quality of our data and the integrity of our trials.

To make the kinds of quantum leaps required of us, we will need to overcome native fears and move from our comfort zones. We should take some comfort, though, that the recent guidance documents reflect serious regulatory intent to help move our industry forward. For example:

The "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring" FDA draft guidance,1 states that by moving away from 100% SDV, we can "improve a sponsor's ability to ensure the quality and integrity of clinical trial data."

The inference is clear-100% SDV represents a hindrance in the eyes of FDA to achieving data quality and trial integrity. We have opportunity, incentive, and a reasonably clear path for making significant enhancements in overall performance by adopting the kinds of plans and tools described in this article. And, we have every reason to believe that by doing so we can speed up clinical trials, improve data quality and dramatically reduce costs.

Editor's Note: A white paper on this topic is available for downloap at http://www.eclinicalforum.org/Default.aspx?id=76&tabid=59.

Julie M. Brothers is Senior Clinical Data Project Manager at Array BioPharma. Dean A. Gittleman is Senior Director of Operations at Target Health. Thomas Haag is Data Integrity Process Expert at Novartis. Darlene Kalinowski* is Associate Director eClinical Operations at Bristol Myers-Squibb, e-mail: Darlene.Kalinowski@bms.com . Ioannis Karageorgos is Senior Protocol Data Manager at Bristol Myers-Squibb. Robert King is Senior Director, Global Functional Service Provision at PPD. Lisa Lucero is Consultant RBM at Eli Lilly. Debbie McCann is Sr. Director, Client Accounts and Quality Practices at Pharmapros. Paula McHale is Senior Director of Product Management at Perceptive Informatics. Jules Mitchel is President at Target Health. Patrick Nadolny is Vice President Data Management and Programing at Allergan. Selina Sibbald is Strategic Alliance Director at Quintiles. Brett Wilson is Associate Director, Business Operations at Bristol Myers-Squibb.

*To whom all correspondence should be addressed.

References

1. Department of Health and Human Services, "Guidance for Industry Oversight of Clinical Investigations-A Risk-Based Approach to Monitoring," (2011), http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf.

2. Code of Federal Regulations Title 21, "21 CFR part 312, subpart D generally (Responsibilities of Sponsors and Investigators)" and "21 CFR part 812, subpart C generally (Responsibilities of Sponsors)," http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=312&showFR=1&subpartNode=21:5.0.1.1.3.4.

3. EMA, "Quality Risk Management (ICH Q9)," (2011).

4. EMA, "Reflection Paper on Risk Based Quality Management in Clinical Trials," draft (2011), http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2011/08/WC500110059.pdf.

5. K. A. Getz, "Flying Blind on CRA Workload, Time Demands," Applied Clinical Trials, 21 (7) 22-24 (2012).

6. "Risk-Based Implementation of Trial Regulation: An Overview of Key Activities," Clinical Research Advisor, (2012), http://index.canarybooks.com/LinkPages/PDF/Advisor%20Issues/Advisor%20302.pdf|>http://index.canarybooks.com/LinkPages/PDF/Advisor%20Issues/Advisor%20302.pdf.

7. B. Morrison, C. Cochran, J. Giangrande, et al., "Monitoring the Quality of Conduct of Clinical Trials: A Survey of Current Practices," Clinical Trials, 8, 342–349 (2011).