Ioannis Karageorgos

	With expansion of clinical research across the globe, organizations need to consider a wide range of regulations, and guidance documents from multiple regulatory authorities, in addition to the country specific laws. Currently, there is an extraordinary number of regulations and guidance documents pertaining to electronic clinical research data that have been issued from ICH, FDA and EMA, PMDA, and MHRA, with full or partial adoption into similar documents by other regulatory bodies.

	Sponsors and Contract Research Organizations (CROs) must interpret and attempt to follow them all if they are doing submissions in > 1 region. It is a daunting task. Each company must decide which are most constrictive/comprehensive and which are encompassed by others. In addition, these documents are of different “ages” and do not always agree with each other. They must also be interpreted in a way that is consistent with industry thinking, with backing to ensure a defendable position. It is imperative to garner this information in a practical way.

	There are no “universal” guidance documents or tools … but by combining like items, the eClinical Forum has offered a method to assist in this review.

	The eClinical Forum is a non-commercial, non-profit global organization with over 45 member companies active in regulated clinical research processes and technologies and has been involved since 2006 in the establishment of conformance criteria for electronic systems used in clinical research. The “eClinical Forum Requirements for Electronic Data for Regulated Clinical Trials” (or simply, “eCF Requirements”) is a list of succinct statements, all based on regulatory documents, that guide a user through this complex task. This is a significant accomplishment by the eClinical Forum’s “Regulatory Expert Group” (REG) as they have put in countless hours over several years to be able to produce such a comprehensive checklist for evaluating electronic systems, that will manage data used in regulated clinical research, against regulations and guidance from ICH, FDA, EMA, PMDA, and MHRA. This work is a result of the vast experience of the REG members who come from a variety of different eCF Member companies. They have spent hours debating each regulation or guidance and how to word the eCF Requirements to correctly meet the needs in the regulatory document.

	The eCF Requirements are available publicly and for free via the eClinical Forum website. They are based upon statements in documents prepared and issued by regulatory authorities that pertain to the design, development, implementation, and management of electronic systems that support clinical research data, as well as those statements that pertain to the handling of data that will be used in a regulated clinical trial.

	Each eCF Requirement has, as its basis, one or more statements from one or more of the regulatory authority documents. The eCF Requirements are in a linked .pdf file such that one can review the requirement and then the statements from regulatory authority documents used as a basis for the eCF Requirement.

	The eCF Requirements can be used to determine if systems, which originate, process, manage and/or retain data that will become part of a regulated clinical trial are consistent with regulatory requirements and recommendations. In particular, the eCF Requirements can be used to assist with self-assessment of systems, planning for system upgrades, writing proposals, Requests for Information (RFIs) and contracts, writing system requirements, writing system test scripts, etc. Depending on the type of system being evaluated, not all eCF Requirements may apply. It is up to the user of these eCF Requirements to review the underlying regulatory basis statements of any eCF Requirement to determine if it should be included in what the user is working on. When it is used to develop test cases, the user should review all statements from the predicate regulatory documents in the mappings to be sure test suites confirm all required items.

	There is also a separate tool called eSRA (eSource-Readiness Assessment), based on the eCF Requirements, which can be used for the evaluation of Electronic Health/Medical Record systems (EHR/EMR systems). It can be found on the eClinical Forum website in the eSRA tab. Sponsors are encouraged to use this assessment rather than their own custom assessment template such that sites can use the same completed eSRA for all of their sponsors.

	The eCF Requirement work is an active process. eCF continues to proactively evaluate and map new regulatory authority documents as they come into effect. It is our intent to release updated mappings to the eCF Requirements yearly. The newest version is released for eCF members only with the previous release provided publicly for free.

 'Determining if Data from Electronic Health Record Systems Can Be Trusted in a Clinical Trial Setting.'

 is the North America Facilitator for eClinical Forum, USA; 

 is the Quality Manager for PCG, Sweden; 

 is the Senior Director, Regulatory Risk and Compliance, Oracle, UK; 

 is the Manager External Data Integration for Bristol Myers Squibb, Belgium; 

 of the Department of Health Informatics, Rutgers the State University of NJ, USA; 

 is the Senior Compliance Manager for CSV, Boehringer Ingelheim, Germany; 

 is the Regulatory Policy and Intelligence Director for Novartis, Switzerland; 

 is the President of Target Health, USA; 

 is Scientific Advisor and General Manager for ERT Geneva, Switzerland; 

 is the Director of Quality Assurance for eClinical Solutions, USA; 

 is the Director of Electronic Systems Quality Assurance for Astellas Pharma Global Development, Inc., USA; 

 is a Quality Consultant for Eli Lilly and Co, Italy.

Articles

Keeping Up with the Regulatory Expectations

	Companies engaged in commercial clinical research have, since the introduction of formalized good clinical practices (GCP), adopted highly defensive practices around the monitoring of clinical trials. The reasons, while diverse, include fear based on perception and interpretation of regulatory requirements. These processes have played a very large role in the high costs associated with clinical trials, without delivering commensurate value. Reasons are many, but surely include propagation of practices established during the years that trials relied on paper-based case report forms (CRFs) without considering optimization of the processes to take advantage of the technology provided by electronic data capture (EDC). All of this has resulted in the institutionalization of high cost, unproven value, manpower-intensive practices that do little to serve the interests of subject safety or quality data. Pharmaceutical, biotech, and medical device industries experiencing unprecedented economic upheaval can no longer justify practices that deliver little value at exorbitant cost. Rather than continuing to spend time, energy, and money focusing on minutiae (e.g., the ability of site personnel to properly transcribe observations from one medium to another), those engaged in clinical trial conduct must re-focus their energies on the aspects of the clinical trial that matter most, such as protocol compliance, subject safety, data timeliness, and data integrity. Through the use of more up-to-date methodologies and technologies, companies can effectively address these concerns while saving both time and money. This article addresses the regulatory and business rationales for adopting monitoring-related tools and processes that, if implemented thoughtfully, should deliver higher-quality trial data, faster, and at significantly lower cost.

	Most areas of business and government-including finance, insurance, public health, agrochemicals, manufacturing, and pharmacovigilance-apply risk-management principles, leveraging mathematical principals (namely statistical inference) in circumstances where it is simply impossible to ensure the quality of a large volume of services or products by exhaustingly checking the accuracy of every item. Clinical research-related industries, operating under GCP, appear as a notable exception for the following reasons:

		The honest and honorable intent to ensure the highest possible quality and safety for products that have a direct impact on the life and well-being of us all

		The pre-GxP beginnings of pharmaceutical research based on the use of paper CRFs

		Fear of failure based on results of regulatory inspections

	Quality risk management has been rather limited in the area of GCP. Until recently, research and development personnel (as well as their representatives/CROs) have believed that they were required to engage in 100% verification of all records and reports obtained in the context of Phase I-IV clinical research.

	These deeply ingrained beliefs have led to practices that are not only prohibitively expensive, but also fail to deliver value that can justify these costs. As a result, the pharmaceutical and device industries have institutionalized practices and behaviors that deliver low value at exorbitant cost. These costs include both the direct and indirect expenses associated with monitoring investigative sites, as well as the opportunity costs associated with these antiquated practices. Exhaustive manual verifications have shown their limits in particular when it comes to identifying (in a timely manner):

		Safety related signals and trends across large datasets

	The almost concurrent release in 2011 of draft guidance documents from two of the three major GCP regulatory agencies related to "risk-based monitoring" has triggered numerous discussions among clinical research operatives and peers, including sponsors of drug and device clinical research, contract research organizations (CROs), and academic clinical research organizations.

	A survey conducted by the Clinical Trials Transformation Initiative

 indicated that there was a large and somewhat inconsistent range of monitoring practices during the conduct of clinical trials. The survey found that monitoring practices vary in intensity, focus, and methodology and include:

		Frequent, comprehensive on-site visits to all clinical investigator sites by company personnel or representatives (e.g., clinical monitors or clinical research associates)

		Targeted on-site visits to higher-risk clinical investigators (e.g., where centralized monitoring indicates problems at a site)

		Centralized monitoring of clinical data by clinical, data management, and statistical personnel at some location other than the study site

	For major efficacy and safety trials, companies typically conduct on-site monitoring visits at approximately four to eight week intervals, at least partly because of the perception that frequent on-site monitoring with 100% source document verification (SDV) is the regulator's preferred way for sponsors to meet their monitoring obligations. In addition, overall source document review (SDR) remains as an on-site monitoring obligation. However, FDA also recognizes that data from critical outcome studies (e.g., National Institutes of Health-sponsored trials, Medical Research Council-sponsored trials in the United Kingdom, and International Study of Infarct Survival), which had no regular on-site monitoring and relied largely on centralized and other alternative monitoring methods, have been relied on by regulators and practitioners.

	The general nature of the guidance documents is a potential gift to sponsor organizations. It encourages the industry to define strategies for risk-based monitoring, and perhaps create standards, that can be customized for individual protocols. Sponsors can now take full advantage of EDC as centralized monitoring tools are developed to capture intra- and inter-site outcomes, as a quality window on GCP.

	The unrealistic fear of regulatory agency findings at the time of pre-approval inspections leads to a more conservative approach that is both necessary and required. A paradigm shift must take place whereby sponsors realize that inspection findings are not to be feared, but are ways to improve processes, and collaborate with the regulatory agencies to better define clinical trial operations and patient safety.

	The eClinical Forum in the context of its chartered mission to serve the pharmaceutical and device industries focuses on those systems, processes, and roles relevant to electronic data capture, data handling, and regulatory submission of clinical trial data. The eClinical Forum has already submitted comments directly to both the FDA and EMA, and released a white paper to promote awareness of the concepts shared by both regulatory documents. By doing so, the eClinical Forum can assuage any apprehension expressed by the more traditionalist organizations which continue to interpret the regulations and predicate rules as calling for full verification and monitoring. We propose that by discussing the key elements of the guidance documents and by proposing best practices on actual tasks (whats and how-tos), we can greatly contribute to demystifying and adopting risk-based quality management of clinical trials. This approach should also allow the regulatory authorities to get access to honest, unbiased, substantiated previews of the organizational and procedural response by industry stakeholders as well as provide previews that are the product of reflective listening to regulatory expectations and that underline the challenges and work effort required by organizations as they adopt, adapt, and measure up to the guidance documents. As a result, the eClinical Forum should contribute actively to the further alignment between the regulatory agencies and the pharmaceutical and device industries.

	Although the guidance document was assessed versus previous and current corporate procedures and best practices, it was necessary or timely to advocate specifically how individual companies might choose to implement changes or adaptations to their quality/risk management. We expect that the roles within each organization that will be asked to perform those tasks will differ, with some companies deciding to assign responsibilities to monitoring, programming or data management job profiles.

	From its inception, EDC provided an opportunity for clinical research operations to re-evaluate the tasks traditionally performed by the data management and site monitoring roles. Moving towards adoption of integrated quality risk management systems calls for further, bolder, and continuous adaptation of systems, workflows, and people. Moving from the ingrained notions of traditional source document review and point-to-point SDV to one of quality processes (focus on optimizing site processes, QC of site compliance, detection of fraud, IMP distribution chains, etc.) will encourage us to:

		Delineate QA from QC in a risk-based quality system.

		Identify standard approaches to centralized and on-site monitoring activities.

		Assess how information technology tools and associated processes can best support centralized and risk-based data review.

		Determine which metrics (thresholds/error rates) and alerts will drive the documentation of quality as well as trigger corrective actions when the quality thresholds are not met.

		Evaluate the impact on the preparation and planning for protocol-driven research.

	Risk is someone or something that creates or suggests a hazard. Sponsors of clinical investigations are required to provide oversight to ensure adequate protection of the rights, welfare, and safety of human subjects and the quality and integrity of the resulting data submitted to regulatory agencies; in other words, avoidance of a hazardous situation.

 As a result, risk must be defined and maintained across all stakeholders.

	Quality is degree of excellence. According to the FDA guidance, quality is a systems property that must be built into an enterprise and cannot be achieved by oversight or monitoring alone.

 Quality is something that needs to be maintained throughout the lifecycle of the product. The evaluation of the risk to quality should be based on scientific knowledge and connected to the protection of the patient.

	There is a need to pause to consider the difference between quality control and quality management. Quality control allows the ability to outline a process to spot check data and operation by reviewing a slice in time. Quality management on the other hand, is used to direct, control, and monitor quality as a more fluid and dynamic activity.

	The ICH Q9 guideline merges the two control processes as it speaks to quality risk management as a standard practice.

 Q9 provides guidance with examples that attempt to outline risk management across the pharmaceutical industry. Figure 1, taken from the EMA Reflection Paper,

 is an excellent starting place for organizations to turn as they design quality risk management; a systematic process for the assessment, control, communication, and review of risks to the quality of the drug (medicinal) product across the product lifecycle.

	The ICH Q9 and EMA reflection paper guidance documents provide good direction on what to address when crafting a quality risk-management plan. However, neither truly addresses regulatory expectations for minimal requirements to accomplish this in a holistic manner. While it is one thing to talk about managing risk through traditional data management and site monitoring processes, it is quite another to propose a concrete way to manage risk through trend setting (electronic transparency on entry time, account use, etc.) and standard metrics (time between tasks, accuracy of input, etc.). Variability in the extent to which organizations approach quality risk management may confuse the situation even more. Fear of having decisions not meet regulatory approval also may keep some companies from updating their approach to quality management. In addition, the payoff may not be as large as indicated.

	While there is no clear indication that quality is inadequate today, there is evidence that the industry overspends to perform tedious reconciliation between source and sponsor data that do not significantly affect the outcome of clinical trials.

	Traditional monitoring plans outline what the sponsor considers to be their responsibility and action plan for ensuring site compliance with the protocol including GCP and other applicable regulations. We should consider the value of combining elements of monitoring, data management, and statistical plans into a comprehensive integrated quality plan. It is important not to jeopardize the relationship between sponsor and the clinical sites and care should be taken to ensure that this remains in the forefront of the plan. For example, a communication plan is useful to document standard communication when there is no need for a site visit. This could be used to praise the site for good work and maintain monthly communication. Minimally, a consistent point of contact should be assigned to the site to represent the face of the sponsor.

	It is important to emphasize that monitoring plans are not unique to the CRA role but also can provide an overall cross functional plan that emphasizes what must be done in order to assure that the data received from clinical research sites are of high quality and integrity. The monitoring plans include not only what is expected to occur at a site level, but also within the data system and program level.

	A quality monitoring plan, which can be part of the clinical monitoring plan, may include information from traditional plans that had previously been separated into role or task based activity, for example, each therapeutic area may be different so several quality plans may be needed on that basis. In considering the monitoring plan, adequate risk assessment must be performed prior to the start of the study, in order to ensure patient protection as well as the quality and integrity of data. Our recommendation is that monitoring plans should exist at the program, protocol, and site levels, and outline what will be included in on-site monitoring as well as central monitoring.

This plan addresses overall quality aspects of the program and should be developed at the start of the clinical program. Risks that are known or anticipated for the disease population and investigational product under study should be documented and clear instruction given as to what, why, and how those risks should be monitored. As a general outline, a program level plan could minimally include the following:

		Planning-a priori identification of risks of the investigational product.

		Design-define protocol eligibility criteria, efficacy, and safety risk factors during design; for inclusion in the eCRF and edit build, as well as define key protocol deviations anticipated for study.

		Execution-clear direction in the oversight and monitoring of parameters leading up to study endpoints, integrated study level monitoring plan, clear direction on evaluating fraud.

		Analysis-clear definition of data measurement and reporting, defined elements of statistical analysis of protocol endpoints, safety, etc.

		Disclosure-publish outcome data, bringing all the above together in the clinical study report (CSR).

	Companies may wish to consider creating standard quality checks pertinent to all programs, and which may also include therapeutic area specific sections to be selected as relevant. Ensuring that the entire study team understands the principles and purpose of the program level monitoring plan will help avoid duplication and encourage study teams focus effort where it is most effective.

	This is an ideal opportunity to take a step back from familiar routines and look hard at current processes; assess what is valuable and what is not; invest in processes that bring most value from a quality perspective. The goal is to develop an integrated clinical data monitoring plan, based on compound and historical knowledge, which can respond proactively to data signals during trial execution.

 This plan should be specific to a particular protocol and draw upon the information from the program level plan as to what is needed to ensure quality. We suggest that the study level plans are categorized as those activities that occur during study startup, study execution, and study closure. This plan allows flexibility for site level variations, (e.g., highly compliant and performing sites, new sites with little experience, high volume sites, and non-compliant sites). The following specific minimal list of topics should be addressed in such a plan:

		Site selection factors-include key protocol requirements that affect decision making on site selection.

		Site training considerations-include information on how the site is prepared to conduct trial.

		Information on all forms of data review.

		Identification of critical data-safety, endpoint related.

		Delineation of what is reviewed on-site versus central.

		Assessment of site understanding of protocol.

	By focusing on study level tasks, rather than roles, study managers will be able to assign responsibilities to the most appropriate team member(s). This adds flexibility to the process, while ensuring that there are no omissions. It also will allow for role changes/combinations in the future. Plans should be easily readable and referable-limit bulk text; use graphics and bullet points. If a suitable electronic tool can be used, it should be feasible to pull out a sub-set of tasks for each function on the study from the overall plan.

 This plan should be specific to each investigative site, combining information from Program/TA level plans and the protocol(s) that the specific site is involved with. There is a large degree of variety in the way regional, country, or hub monitoring units are being organized, therefore a critical aspect of the risk assessment will be the concurrent (and possibly competitive) workload and commitments that shape the possibilities and constraints of both the sponsor/CRO and site staff when collecting and reviewing data. Another key consideration in defining a site level risk management plan is to never project any task or activity over time as being flat or a product of the volume of data flow only. What's more, it is an oversimplification (with potentially disastrous results) to even consider data quality as being inversely proportionate to its volume. Some of the minimally required aspects to cover in the site level plan are:

The eClinical Forum Risk Based Monitoring Taskforce offers some best practices for ensuring clinical data quality.

Ioannis Karageorgos

Articles

Keeping Up with the Regulatory Expectations

Risk-Based Approaches