Dean A. Gittleman

There has been a recent thrust within the pharmaceutical industry to promote the transformation of the clinical trial process. In 2008, the Clinical Trial Transformational Initiative (CTTI), a public-private partnership with the FDA, and Duke University, was established to identify practices that, through broad adoption, will increase the quality and efficiency of clinical trials. A major publication by CTTI of monitoring practices has helped to transform the monitoring of clinical trials.

	According to the Food and Drug Administration (FDA) eSource Guidance of 2013: “Electronic source data are data initially recorded in electronic format. They can include information in original records and certified copies of original records of clinical findings, observations, or other activities captured prior to or during a clinical investigation used for reconstructing and evaluating the investigation.”

 Saying it differently, eSource data are original subject data that are collected digitally without having to record the data on a piece of paper first, and then transcribe it to an electronic data capture (EDC) system.

	It has been shown by many that changes made to the study database as a result of source document verification (SDV) affect neither study results nor the interpretation of those results.

 Therefore, despite the support of regulators, including the FDA

, it is difficult to explain 1) why many of those managing clinical research operations they still respond to a perceived risk, instead of adopting systematic and scientifically substantiated approaches for data management and cleaning, and 2) why there has been tepid adoption of eSource solutions coupled with “intelligent” monitoring of clinical trials. Perhaps failure to appreciate the wisdom contained in the quotes below explains why those doing clinical research spend a lot of their time on non-productive activities.

“Life is really simple, but we insist on making it complicated”

“Simplicity is the ultimate sophistication”

“The art of being wise is the art of knowing what to overlook”

	There is broad agreement across the pharmaceutical industry that it no longer makes sense to collect clinical trial data by writing results and observations first on a piece of paper, then to transcribe those data into an EDC system, and then to save that paper so that a clinical research associate (CRA) can check how precisely a person transcribed information from a piece of paper into the EDC system. Sadly, we continue to see residual, frequent emphasis on checking these transcription activities when electronic systems can easily eliminate this highly inefficient process. Doing the math, moving to electronic systems can provide a dramatic return on investment by 1) reducing labor costs at the clinical research sites and sponsoring companies, 2) reducing travel costs, and 3) providing improvements in data quality.

	In 2006, the Electronic Source Data Interchange (eSDI) Group within CDISC issued a paper entitled “Leveraging the CDISC Standards to Facilitate the use of Electronic Source Data within Clinical Trials.”

 The eSDI document was the product of the CDISC eSDI Initiative, the purpose 

of which was “to investigate the use of electronic technology in the context of existing regulations for the collection of eSource data (including that from eDiaries, EHR, EDC) in clinical trials for regulatory submission by leveraging the power of the CDISC standards, in particular the Operational Data Model (ODM)." The overall goal of the initiative was to make it easier to collect data once in an industry standard format.

	Eight years later, on Jan. 29, 2014, FDA addressed these issues in a webinar reviewing the final guidance for industry titled “Electronic Source Data in Clinical Investigations."

Leonard V. Sacks, associate director, Office of Medical Policy (CDER), Ron Fitzmartin, Office of Strategic Programs (CBER), and Jonathan S. Helfgott, associate director for risk science (acting), Office of Scientific Investigations (CDER).  Clearly, FDA is supporting efforts by industry to adopt eSource solutions.

	Even earlier, however, in 2010, the EMA GCP Inspectors Working Group eSource subteam released the “EMA Reflection Paper On Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials.”

 The reflection paper-which is very similar in nature to an FDA guidance-was followed up by direct exchanges with industry representatives (the Society for Clinical Data Management and the eClinical Forum, among others). Those discussions are captured in a series of questions and answers that are frequently updated on the EMA website.

 of an independent contemporaneous investigator copy (ICIC) of the case report form (CRF) has led to numerous and passionate debates among the professional and industry associations. The basic concept of the ICIC is still far from being consistently understood and implemented in the different geographies; regardless, ICIC has had a considerable impact and has brought about an extension of the concept of ALCOA, that source data should not only be

attributable; legible; contemporaneous; original; and accurate, but also complete; consistent; enduring and available, when needed

As a result of this lack of consensus of the regulatory understanding of how to optimally generate and review electronic source records, apprehension and uncertainty among solutions vendors, contract research operatives, and pharmaceutical company sponsor remains.

 Good Clinical practice states that “The sponsor should ensure that it is specified in the protocol or other written agreement that the investigator(s)/institution(s) will permit trial-related monitoring, audits, IRB/IEC review, and regulatory inspection(s) by providing direct access to source data/documents.” There are no statements that paper records must be kept, just that there needs to be direct access to source data/documents.

	The FDA, in September 2013, issued a final “Guidance for Industry: Electronic Source Data in Clinical Investigations.”

 This guidance is consistent with the EMA “Reflection Paper on Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials.”

	Regulators seem ahead of the game in supporting the transformation of how the pharmaceutical industry manages the performance of clinical trials. The following summarizes aspects of the perspectives of FDA and EMA.

	According to the FDA Guidance, source data include all findings, observations, or other activities in original and certified copies of original records, which are used by regulators to reconstruct and evaluate a clinical trial. All interested parties need to review source data and ensure adequate protection of human clinical trial subjects and the quality and integrity of the clinical data. Source data should be ALCOA and must meet the regulatory requirements for recordkeeping. FDA has proposed that electronically capturing and transmitting source data to the eCRF should:

		Encourage entering source data during a subject’s visit, where appropriate

		Eliminate unnecessary duplication of data entry and transcription errors

		Facilitate remote monitoring of data and promote real-time access for data review

		Facilitate the collection of accurate and complete data

	According to the FDA, data can be entered into the eCRF either manually or electronically using “direct entry of data into the eCRF.”  For example, there are many data elements in a clinical investigation, such as blood pressure, that can be entered directly into the eCRF at the time of the office visit. FDA also indicated that “data elements originating in an electronic medical record (EMR) can be automatically transmitted directly into the eCRF.  Unlike a direct transmission to an eCRF from instruments or medical devices, EMRs can use intervening processes (e.g., algorithms for the selection of the appropriate data elements).” In cases where the EMR serves as the source, the pertinent data for the subjects in the clinical study should be made available for review during regulatory inspections and monitoring visits.

	And although many may know this, the FDA added that the eCRF should be capable of recording metadata (e.g., data about the data identifying who entered or generated the data and when the data were entered or generated).  Also, changes to the data must not obscure the original entry, and must record who made the change, when, and why. Data element identifiers should be attached to each data element as it is entered or transmitted by the originator into the eCRF. Data element identifiers for each subject should identify the:

		Originators of the data element (including those entered manually (e.g., by the clinical investigator(s)) or automatically by devices

		Date and time that the data element was entered into the eCRF

	These data element identifiers allow sponsors, FDA, and other authorized parties to examine the audit trail of the eCRF data and to provide information that will allow FDA to reconstruct and evaluate the clinical investigation.

	The following are from the EMA Reflection Paper

In order to meet the requirements of maintenance of an independent contemporaneous copy of source records, a certified copy of the data should be created before the transfer to the sponsor and retained at the investigator site. The method of transfer should be validated. 

The recording of a clinical observation is made at the same time as when the observation occurred. If this is not possible the chronology of events should be recorded. An acceptable amount of delay should be defined and justified prior to trial recruitment.

The source data and their respective capture methods should be clearly defined prior to trial recruitment (i.e., in the protocol or study-specific source data agreement). The sponsor should describe which data will be transferred, the origin and destination of the data, the parties with access to the transferred data, the timing of the transfer, and any actions that may be triggered by real-time review of those data. There should only be one source defined at any time for any data element.

	According to the EMA, the source data and their respective capture methods should be clearly defined prior to trial recruitment (i.e. in the protocol or study specific source data agreement). The sponsor should describe which data will be transferred, the origin and destination of the data, the parties with access to the transferred data, the timing of the transfer, and any actions that may be triggered by real-time review of those data. There should only be one source defined at any time for any data element. The EMA has focused on the issue of source data control and specifically stated that “the investigator should maintain the original source document or a certified copy (Requirement 5, ICH GCP 2.11, 5.15.1); that source data should only be modified with the knowledge or approval of the investigator (Requirement 6, ICH GCP 4.9.3, 4.9.4 and chapter 8); and the sponsor should not have exclusive control of a source document. (Requirement 10, ICH GCP 8.3.13).”

	The EMA also requires that “all data generated in a clinical trial relevant to patient care must be made available to the investigator at all times during and after the trial, and all data held by the sponsor that has been generated in a clinical trial should be verifiable to a copy not held by the sponsor.”  The EMA also stated that “the requirements above are not met if data are captured in an electronic system and the data are stored on a central server under the sole control of the sponsor.” The EMA requires that a “contemporaneous certified copy of the data should be retained at the investigator site in addition to the record maintained on a central server.”

	In addition, the business requirement of any electronic system can be summarized to include:

		To be a separate legal entity from the sponsor and from the investigator, with a detailed contract defining duties and responsibilities of each party

		An assured mechanism for investigators to be informed about and/or approves of modifications to the data should be clearly established, and this control by the investigator should be demonstrable 

		Any changes to the data should be captured by the audit trail

	The variations in regulatory understanding have confounded many stakeholders. In order to find a practical way forward, industry and professional associations should concentrate efforts on identifying and addressing concerns around clinical data quality. Specific provisions associated with quality assurance and the transparency of data flow should be present in the protocol, protocol-associated plans, and any contractual agreements with the vendors and study sites. Failure to create, implement, and document such risk mitigation associated with source data would be a major shortcoming, as it could:

		Hinder the scientific interpretability of the data-and renders any clinical decision support impossible

	Not surprising, the EMA has directly warned sponsors and contract research organizations (CROs) that documentation, procedural, and systemic shortcomings will compromise (or even block) application package submission to the Committee for Medicinal Products for Human Use (CHMP). Similar reactions in presence of blatant lack of compliance are to be expected in the case of NDAs (CDER), BLAs (CBER) and PMAs (CDRH).

Source records during development process

	Despite the philosophical and historical differences explained earlier, it is possible to implement a clinical data acquisition environment that meets all recognized regulatory expectations. Source records, for example, can be maintained in many formats, including but not limited to paper records (e.g., data files, XML files, and PDF files). Regardless of the medium or the format, compliant maintenance of source records:

		Assures the ability of independent auditors to confirm the accuracy of data presented to regulatory agencies by pharmaceutical and device companies

		Allows for the reconstruction of study data in the event of catastrophic loss of data by the sponsor

	Source data should not be the same “version” of the data that the sponsor (or CRO) “controls.” Confirmation of the accuracy of the data submitted to regulatory agencies can be accomplished by review of the validation of electronic systems used as original data sources, manually reviewing paper records and comparing results submitted by sponsoring companies and/or performing “data compare” of electronic source records to data stored in locked databases. Ideally, evaluating extracts of, or entire datasets against CRF data elements could provide corroborating evidence, and ultimately a measure of confidence, as to the accuracy of reported assessments and endpoints. This, however, would require access to the electronic source data from EMRs, machines used in the clinical trial process (e.g., raw MRI records) and EDC databases prior to transfer to the sponsor of the clinical trial. While all of these tasks can be used to assure precise data (i.e., data elements agree between databases), these processes do not holistically address data accuracy. Sensitivity analyses done on real clinical trial datasets have repeatedly shown that database changes, as a result of SDV and SDR, have little or no impact on final means and standard deviations (SD).

	Therefore, key areas of focus during monitoring, audits or inspections are the following:

		Were the clinical site personnel qualified, as documented by education and experience, to perform the activities for which they were responsible?

		Were the clinical site personnel properly trained on the protocol and the electronic systems used for the trial (e.g., EDC, eSource storage, etc.)?

		Did the clinical site understand and follow the protocol?

		Were there any differences in outcomes between sites, suggesting lack of understanding of the protocol or possible fraud?

		Were basic regulations followed that assure patient safety (e.g., proper informed consent and safety monitoring)?

	The above is certainly not exhaustive-but it is an example of the strict minimums that assure quality.

	Sites may maintain source records in many formats, including electronic data stored by an independent third party prior to the source data arriving at the EDC database. 

	Source records could come from direct data entry at the time of the study visit, from the EMR, from patient reported outcomes (PROs), and from output from machines. The sponsor should have no more access to electronic source records than if given access to a site’s paper source records. One advantage of the source record existing as a PDF file in the cloud, with read-only system privileges, is that it is impossible for registered users to manipulate the file. It is the same as viewing a document on the FDA or EMA website, where one can view but cannot modify the file. PDF files can also be uploaded into most EMR systems and searched if saved as “readable” files. Generating electronic source as .XML can further enhance the ability to search.

	One way to create an ICIC of the site’s submitted study data is to adopt a data flow such as that depicted in Figure 1. In this model, at the time of the

 subject visit, rather than create a paper record of a study-related observation-if they have the means-the site staff can enter the information directly into a properly configured EDC system. As depicted in the figure, the system then takes a snapshot of the entered information (e.g., pdf, xml), transmits the snapshot to a repository controlled by the site (step 3), receives acknowledgment that the data were received without error (step 4), and then transmits the data to the EDC database (step 5). This approach is consistent with regulatory concerns in that sites at all times maintain control over original data. This process also reduces the site’s work burden by eliminating maintenance of paper copies and subsequent transcription to EDC systems, and provides sponsors with real-time data with which to make timely decisions. In today’s world of cloud-based computing, there is a growing recognition of the benefits of capturing source records using databases and web applications, vs. local data storage devices.

	There is no legislation or regulatory guidance that prohibits the investigator from delegating the hosting and maintenance of the site’s ICIC of original date via contractual agreement to an outside vendor or CRO. This applies even if the system is set up and financed by the

 sponsor. At the same time, it is the responsibility of the sponsor to have control of the study database and the data management process. This task is often delegated to a CRO, via a written agreement. These two tasks can very well be delegated to two independent CROs. However, when the same CRO takes on both tasks in the same study, independent systems must be in place to control access by employees of the CRO to the data management and eSource software products. If the appropriate controls of systems are not in place, this could indicate a conflict of interest where the independence of the CRO may be questioned.

	A key question is whether it is reasonable for the EMA, FDA, and other regulatory bodies to view independent CROs performing EDC and concurrent data management services as independent of sponsors. Clearly, no one other than the clinical site should enter or modify data and all transactions must be subject to an audit trail. So the fact that a CRO is performing data management should not disqualify it from hosting eSource data as long as:

Articles

How to satisfy regulatory concerns about EDC data integrity and site controls over its source records.

eSource Records in Clinical Research: Keeping it Simple

	There has been a recent thrust within the pharmaceutical industry to promote the transformation of the clinical trial process. In 2008, the Clinical Trial Transformational Initiative (CTTI), a public-private partnership with the FDA, and Duke University, was established to identify practices that, through broad adoption, will increase the quality and efficiency of clinical trials. A major publication by CTTI of monitoring practices has helped to transform the monitoring of clinical trials.

 The eSDI document was the product of the CDISC eSDI Initiative, the purpose of which was “to investigate the use of electronic technology in the context of existing regulations for the collection of eSource data (including that from eDiaries, EHR, EDC) in clinical trials for regulatory submission by leveraging the power of the CDISC standards, in particular the Operational Data Model (ODM)." The overall goal of the initiative was to make it easier to collect data once in an industry standard format.

 The reflection paper-which is very similar in nature to an FDA guidance-was followed up by direct exchanges with industry representatives (the Society for Clinical Data Management and the eClinical Forum, among others). Those discussions are captured in a series of questions & answers that are frequently updated on the EMA website.  

The following are from the EMA Reflection Paper

		Violate fundamental patient rights
		 

	One way to create an ICIC of the site’s submitted study data is to adopt a data flow such as that depicted in Figure 2. In this model, at the time of the subject visit, rather than create a paper record of a study-related observation-if they have the means-the site staff can enter the information directly into a properly configured EDC system. As depicted in the figure, the system then takes a snapshot of the entered information (e.g., pdf, xml), transmits the snapshot to a repository controlled by the site (step 3), receives acknowledgment that the data were received without error (step 4), and then transmits the data to the EDC database (step 5). This approach is consistent with regulatory concerns in that sites at all times maintain control over original data. This process also reduces the site’s work burden by eliminating maintenance of paper copies and subsequent transcription to EDC systems, and provides sponsors with real-time data with which to make timely decisions. In today’s world of cloud-based computing, there is a growing recognition of the benefits of capturing source records using databases and web applications, vs. local data storage devices.

	There is no legislation or regulatory guidance that prohibits the investigator from delegating the hosting and maintenance of the site’s ICIC of original date via contractual agreement to an outside vendor or CRO. This applies even if the system is set up and financed by the sponsor. At the same time, it is the responsibility of the sponsor to have control of the study database and the data management process. This task is often delegated to a CRO, via a written agreement. These two tasks can very well be delegated to two independent CROs. However, when the same CRO takes on both tasks in the same study, independent systems must be in place to control access by employees of the CRO to the data management and eSource software products. If the appropriate controls of systems are not in place, this could indicate a conflict of interest where the independence of the CRO may be questioned.

eSource Records in Clinical Research

In order to support the transformation of how the pharmaceutical industry manages the performance of clinical trials, in 2013, the Food and Drug Administration (FDA) issued its final "Guidance for Industry: Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring,"

  and a "Guidance for Industry: Electronic Source Data in Clinical Investigations."

  These guidances are consistent with similar recommendations on those topics issued by the European Medicines Agency (EMA): Reflection papers on risk-based quality management in clinical trials

  and expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials.

In 2011, a critical publication on the varied practices of monitoring clinical trials

  was published by the Clinical Trials Transformation Initiative (CTTI), a public-private partnership formally established in 2008 by the FDA and Duke University, to identify practices that, through broad adoption, will increase the quality and efficiency of clinical trials.  This publication has served as an impetus for the pharmaceutical and device industries, together with regulators, to address the monitoring of clinical trials, which as currently practiced, is inefficient, costly, and, therefore, unsustainable.

  In 2010, a comprehensive paper on the pros and cons of risk-based monitoring (RBM) was published,

 with a follow-up paper imploring the industry that it was time to change to RBM.

Quality by design (QbD) is a concept first outlined by Joseph M. Juran and is based on the premise that quality should be part of the project-planning process.

  According to Juran, most quality crises and problems are due to the way quality is originally planned. While QbD methodologies have been used to advance product and process quality in every industry, they have most recently been adopted by the FDA for drug manufacturing.

  In clinical research, the protocol identifies the quality requirements, and detailed plans and activities complement what is in the protocol. Key factors to QbD methodologies include a well-designed protocol, proper execution of the protocol, steps to assure protocol compliance, corrective and preventative action methodologies, and clear and concise communication strategies.  

FDA recently published a paper on QbD methodologies, describing how clinical research is changing, and how FDA and other regulatory authorities are fostering these changes.

  have recently published thoughtful papers on how the pharmaceutical and device industries could address RBM. Meanwhile, an approach to quality assurance in the 21st century was published in 

which described a QbD methodology for clinical research.

Results from a Phase II study using RBM and direct data entry (DDE), where the clinical site entered each subject's data into an electronic datacapture (EDC) system at the time of the office visit, demonstrated a major reduction in on-site monitoring compared to comparable studies that use paper source records; EDC edit checks were able to be modified early in the course of the clinical trial, and protocol compliance issues could be identified in real time and rapidly corrected.

  The use of DDE and near real-time monitoring also led to rapid detection of safety issues. The site reported major cost savings, and estimated that just in terms of data entry, it was able to save 70 hours of labor by not having to transcribe data from paper source records into the EDC system.

This article reports the results of the clinical trial initiated in the U.S. and Canada, which included 18 sites and 180 treated subjects; all the sites performed DDE and all the clinical research associates (CRAs) performed RBM, with the bulk of monitoring activities occurring centrally from the home office. 

In addition to a well-designed protocol, the following QbD elements were operationally incorporated into the clinical trial: 

Risk-based clinical data monitoring plan (CDMoP):

 A written strategy was developed to address the review of site-specific source data/documents, the schedule for on-site monitoring, the frequency of central monitoring, and the issuance of central monitoring reports. The CDMoP specified roles and responsibilities as well as the specific monitoring requirements to ensure that the clinical sites complied with the study protocol and regulatory requirements.  

The CDMoP also indicated that monitors were to record all monitoring reports in the EDC system, and that all sponsor and study documents were to be maintained in the electronic trial master file (eTMF). 

Within the CDMoP, a risk mitigation strategy identified a total of 23 risks to subject safety and/or trial outcome. Each risk was assigned a low to high probability score (1-3) and severity score (1-3). Each risk was then assigned a score, which was a multiple of the two scores, as well as a risk mitigation strategy. For example, "subject dropouts" was one risk to the trial outcome, since any dropout was potentially to be considered a treatment failure. Therefore, subject dropout was assigned a severity score of 3 and a probability score of 2, for a total score of 6. The risk mitigation strategy incorporated was "training and evaluating and resolving reasons for dropouts, phone alerts prompted by the eCRF, and review of online management reports." 

 The CDMoP documented that the study would use DDE at the time of the clinic visit. The electronic clinical trial record (eCTR) allowed the study sites to have a contemporaneous electronic copy of the subject's record. To comply with regulations, access to the eCTR was controlled by the clinical investigator or designee and not the pharmaceutical company sponsoring the trial; these original data were stored in "a trusted, third-party repository" prior to the data being transmitted to the EDC database.  

 Initially, weekly meetings were held with key team members to review all monitoring activities. Integrated online data management reports addressed safety, quality, compliance, and study-specific issues. As the study progressed, the frequency of these meetings was changed to every two weeks. 

On-site and central monitoring activities:

 As part of the approach to RBM, the CDMoP identified the need to perform both on-site and central monitoring. To avoid the need to duplicate data already within the EDC system, key metrics from the EDC system were displayed within the monitoring reports.  The monitoring reports were generated online within the EDC portal and signed electronically by the CRA and the CRA's supervisor. Lists of observations requiring follow-up were also maintained within the EDC portal. 

 A detailed safety monitoring plan was developed. There was nothing unique in this approach to safety monitoring except that an Adobe Acrobat version of an FDA-approved online MedWatch Form 3500A and CIOMS Form 1 could be generated directly from the EDC system for  original and follow-up reports. The investigator and the medical monitor could enter online narratives and the medical monitor could control the finalization of the original and follow-up reports needed for regulatory submissions. These reports became an integral part of the EDC system and could be retrieved on demand, based on permissions, anywhere in the world. In addition to the agreed-upon procedures involving serious adverse event (SAE) reporting to the sponsor and regulatory authorities, email alerts occurred at the time of data entry for any SAE and if any SAE data were modified. In addition, the EDC system summarized all adverse events, and it was possible to assess adverse events across sites. 

 As part of the QbD methodology, initially, weekly meetings of approximately one hour occurred with the clinical team (n=3), the sponsor (n=2), and an outside expert who performed quality oversight (n=1). Over the course of eight months, this represented a total of 20 meetings and 80 hours (two weeks) of human resources. This effort was roughly equivalent to three on-site monitoring visits. 

 One of the key advantages never consistently accomplished with EDC was the ability to have rapid access to the clinical trial data from the time of the office visit. With DDE, the site was "forced" to enter the data at the time of the office visit. However, as this involved a change in behavior at the clinical site, there was no guarantee that the sites would comply. Therefore, the time to data entry from the day of the clinic visit was assessed. However, not all data could be entered directly at the time of the office visit since sites maintained certain source records outside the EDC system. As a result, some of the data associated with these source records were entered after the clinic visit. For example, unreported medical histories and medications were identified during "chart review" at the time of the monitoring visit. 

In spite of DDE being a "disruptive innovation," 92% of data were entered on the day of the office visit and 95% within five days and 98% within eight days (see Figure 1). Some of the outliers were due to findings during the monitoring visits and delays in data entry when the sites waited for additional information to complete a form.  

 The time to data review by the monitors is a key factor in optimizing RBM, since without having access to real-time data, the same errors are repeated and any corrective actions are delayed. Key forms included in this analysis were: 					

A total of 13,124 forms were analyzed from 180 subjects (see Figure 2). Results showed that 50% of the forms were reviewed within 13 hours (0.54 days) of data entry, 75% within 27 hours (1.1 days), 95% within 124 hours (5.2 days), and 100% within 335 hours (14 days). It should be noted, however, that occasionally a form was "missed" by the CRA and a small number of forms were "saved" awaiting additional information or conclusions based on consultations with the principal investigator (PI).  

 Between August 1, 2012 and May 31, 2013, 31 on-site monitoring visits were performed at the 18 sites. No other on-site monitoring was deemed necessary based on the observations at the initial on-site visit, daily review of online eCRFs, in-house review of the eTMF, and site audits by quality assurance. The bulk of the second monitoring visit was combined with the closeout visit since most of the subjects had completed treatment at the time of the visit.  The final closeout activities were performed over the phone.  

Since measurements on the last day of the three-month treatment phase of the study included evaluation of the primary endpoint, prior to the first subject arriving for that final visit, each site was retrained over the phone as to the required activities taking place on final day of the study. In addition, each site was instructed to inform the CRA when the first subject was to arrive for the Day 90 visit, so that the CRA could immediately review all of the data entered on that day. An email alert was also sent to the project team at the time the Day 90 visit date was entered within the EDC system. 

A total of 211 central monitoring reports were issued, and once it was clear that the sites and monitors were adequately trained, the frequency of issuing these reports was changed from every two weeks to every four weeks. 

 For this study, there were 27,957 EDC "pages" entered for 29 unique CRFs. As part of the approach to RBM, the CDMoP identified specific data elements collected at the sites either within the electronic medical record (EMR) or on paper charts for source data verification (SDV).  

A total of 5,581 of these paper/electronic source records were reviewed at the site and compared with the trial database. These records represented about 20% of all entered pages. Results showed that only 13 of the 29 forms had any changes, with a total of 48 changes made to the database as a result of SDV (see Table 1). These changes represented a 0.86% "error rate."  The vast majority of the changes (66.6%) occurred in just three forms: medications (13; 27%), medical history (10; 21%), and clinical laboratory result (9; 19%). 

In order to evaluate this "0.86% error rate," Table 2 at right identifies examples of types of changes made to the database as a result of SDV. As illustrated, only one modification, titration result (278.3 changed to 123.2), could have had any impact on the study. However, as this parameter was defined as critical to quality (CTQ)—a specific risk to protocol compliance and subject safety—a copy of the record was available to the CRA at the same time the site received it. In addition, all of the changes identified via the SDV process would have had no impact on subject safety, data integrity, or protocol compliance.  

 There were 1,099 queries generated from 27,966 CRFs entered by the sites. This represents an overall form query rate of 3.9%. However, only 403 (37.6%) of the queries resulted in changes to the database. Thus, only 1.4% (403/27,966) of forms had database changes as a result of CRA-generated queries.  

In order to measure efficiencies of CRA review activities, the time from data entry to query generation was assessed.  Strikingly, 39% of queries were generated on the same day as the office visit—58% within one day and 70.6% within five calendar days. What this really means is that corrective actions were able to occur early and rapidly during the clinical trial. 

Time from query generation to resolution:

 Queries were generated in response to an edit check being fired at the time of data entry (auto queries) for which the reason provided by the site required additional information, or as a result of a de novo request for additional information based on clinical review of the CRFs. For example, an ongoing diagnosis of type 2 diabetes was reported but no treatment was documented. While online monitoring was done in real time with current snapshots of data, queries were also generated from online batch edits, which were generated within the EDC system, based on cumulative comparisons of information entered across forms, and over time, that suggested data inconsistencies. 

In the previous tables, it was demonstrated that with central monitoring and DDE, it is possible to rapidly enter data and generate queries from the time of data entry. The next challenge was to assure that queries are resolved in a rapid manner. As illustrated in Figure 4, the time to query resolution was assessed for all generated queries, including those done manually, those done based on edit checks being fired at the time of data entry, and those resulting from batch queries run at night within the EDC system.  

As shown, with central monitoring, 22% of queries were resolved on the same day they were generated—78% within five calendar days, 91% within 10 days, and 99% within 30 days. 

One of the keys to a successful outcome of a clinical trial includes timely data entry and data review, and ideally for data entry and data review to occur at the time of the office visit. 

Risk has to do with the probability and impact of an event to the outcome of a trial, and risk mitigation strategies are put in place to manage that risk. Clearly, researchers should not put the same effort into monitoring variables that "do not matter" as they do into the ones that "do matter." RBM is not about more or less monitoring visits, or SDV, but rather targeted, efficient, and intelligent monitoring. CRAs need to be retrained in their way of monitoring by focusing on the elimination of errors that matter. 

DDE can dramatically reduce or even eliminate paper records, and as a result, SDV should also be dramatically reduced. Since SDV typically assesses how well people transcribe from one medium to another, and since such transcription "error rates" are typically below 1%, SDV as currently performed, should have no impact on the study results. However, as part of the risk assessments performed at the beginning and during the study, the rationale and scope of SDV should be defined. SDV requirements will most likely be replaced, in part, with source data review (SDR) or what would be better described as chart review. Chart review truly allows for a snapshot of the study subject, where critical information "buried" in the chart can be discovered. 

After all, monitoring is all about training and oversight.  Think about a typical Phase I pharmacokinetic (PK) study. Should the same effort be invested to verify the date of an appendectomy 10 years in the past as would to verify the time of critical PK draws, storage conditions of the samples, and shipping procedures for analysis and methods validation? 

The main lessons learned from the study were: 					

Study results using a quality-by-design method, risk-based monitoring, and real-time direct data entry.

Three-Pronged Approach to Optimizing Trial Monitoring

	Companies engaged in commercial clinical research have, since the introduction of formalized good clinical practices (GCP), adopted highly defensive practices around the monitoring of clinical trials. The reasons, while diverse, include fear based on perception and interpretation of regulatory requirements. These processes have played a very large role in the high costs associated with clinical trials, without delivering commensurate value. Reasons are many, but surely include propagation of practices established during the years that trials relied on paper-based case report forms (CRFs) without considering optimization of the processes to take advantage of the technology provided by electronic data capture (EDC). All of this has resulted in the institutionalization of high cost, unproven value, manpower-intensive practices that do little to serve the interests of subject safety or quality data. Pharmaceutical, biotech, and medical device industries experiencing unprecedented economic upheaval can no longer justify practices that deliver little value at exorbitant cost. Rather than continuing to spend time, energy, and money focusing on minutiae (e.g., the ability of site personnel to properly transcribe observations from one medium to another), those engaged in clinical trial conduct must re-focus their energies on the aspects of the clinical trial that matter most, such as protocol compliance, subject safety, data timeliness, and data integrity. Through the use of more up-to-date methodologies and technologies, companies can effectively address these concerns while saving both time and money. This article addresses the regulatory and business rationales for adopting monitoring-related tools and processes that, if implemented thoughtfully, should deliver higher-quality trial data, faster, and at significantly lower cost.

	Most areas of business and government-including finance, insurance, public health, agrochemicals, manufacturing, and pharmacovigilance-apply risk-management principles, leveraging mathematical principals (namely statistical inference) in circumstances where it is simply impossible to ensure the quality of a large volume of services or products by exhaustingly checking the accuracy of every item. Clinical research-related industries, operating under GCP, appear as a notable exception for the following reasons:

		The honest and honorable intent to ensure the highest possible quality and safety for products that have a direct impact on the life and well-being of us all

		The pre-GxP beginnings of pharmaceutical research based on the use of paper CRFs

		Fear of failure based on results of regulatory inspections

	Quality risk management has been rather limited in the area of GCP. Until recently, research and development personnel (as well as their representatives/CROs) have believed that they were required to engage in 100% verification of all records and reports obtained in the context of Phase I-IV clinical research.

	These deeply ingrained beliefs have led to practices that are not only prohibitively expensive, but also fail to deliver value that can justify these costs. As a result, the pharmaceutical and device industries have institutionalized practices and behaviors that deliver low value at exorbitant cost. These costs include both the direct and indirect expenses associated with monitoring investigative sites, as well as the opportunity costs associated with these antiquated practices. Exhaustive manual verifications have shown their limits in particular when it comes to identifying (in a timely manner):

		Safety related signals and trends across large datasets

	The almost concurrent release in 2011 of draft guidance documents from two of the three major GCP regulatory agencies related to "risk-based monitoring" has triggered numerous discussions among clinical research operatives and peers, including sponsors of drug and device clinical research, contract research organizations (CROs), and academic clinical research organizations.

	A survey conducted by the Clinical Trials Transformation Initiative

 indicated that there was a large and somewhat inconsistent range of monitoring practices during the conduct of clinical trials. The survey found that monitoring practices vary in intensity, focus, and methodology and include:

		Frequent, comprehensive on-site visits to all clinical investigator sites by company personnel or representatives (e.g., clinical monitors or clinical research associates)

		Targeted on-site visits to higher-risk clinical investigators (e.g., where centralized monitoring indicates problems at a site)

		Centralized monitoring of clinical data by clinical, data management, and statistical personnel at some location other than the study site

	For major efficacy and safety trials, companies typically conduct on-site monitoring visits at approximately four to eight week intervals, at least partly because of the perception that frequent on-site monitoring with 100% source document verification (SDV) is the regulator's preferred way for sponsors to meet their monitoring obligations. In addition, overall source document review (SDR) remains as an on-site monitoring obligation. However, FDA also recognizes that data from critical outcome studies (e.g., National Institutes of Health-sponsored trials, Medical Research Council-sponsored trials in the United Kingdom, and International Study of Infarct Survival), which had no regular on-site monitoring and relied largely on centralized and other alternative monitoring methods, have been relied on by regulators and practitioners.

	The general nature of the guidance documents is a potential gift to sponsor organizations. It encourages the industry to define strategies for risk-based monitoring, and perhaps create standards, that can be customized for individual protocols. Sponsors can now take full advantage of EDC as centralized monitoring tools are developed to capture intra- and inter-site outcomes, as a quality window on GCP.

	The unrealistic fear of regulatory agency findings at the time of pre-approval inspections leads to a more conservative approach that is both necessary and required. A paradigm shift must take place whereby sponsors realize that inspection findings are not to be feared, but are ways to improve processes, and collaborate with the regulatory agencies to better define clinical trial operations and patient safety.

	The eClinical Forum in the context of its chartered mission to serve the pharmaceutical and device industries focuses on those systems, processes, and roles relevant to electronic data capture, data handling, and regulatory submission of clinical trial data. The eClinical Forum has already submitted comments directly to both the FDA and EMA, and released a white paper to promote awareness of the concepts shared by both regulatory documents. By doing so, the eClinical Forum can assuage any apprehension expressed by the more traditionalist organizations which continue to interpret the regulations and predicate rules as calling for full verification and monitoring. We propose that by discussing the key elements of the guidance documents and by proposing best practices on actual tasks (whats and how-tos), we can greatly contribute to demystifying and adopting risk-based quality management of clinical trials. This approach should also allow the regulatory authorities to get access to honest, unbiased, substantiated previews of the organizational and procedural response by industry stakeholders as well as provide previews that are the product of reflective listening to regulatory expectations and that underline the challenges and work effort required by organizations as they adopt, adapt, and measure up to the guidance documents. As a result, the eClinical Forum should contribute actively to the further alignment between the regulatory agencies and the pharmaceutical and device industries.

	Although the guidance document was assessed versus previous and current corporate procedures and best practices, it was necessary or timely to advocate specifically how individual companies might choose to implement changes or adaptations to their quality/risk management. We expect that the roles within each organization that will be asked to perform those tasks will differ, with some companies deciding to assign responsibilities to monitoring, programming or data management job profiles.

	From its inception, EDC provided an opportunity for clinical research operations to re-evaluate the tasks traditionally performed by the data management and site monitoring roles. Moving towards adoption of integrated quality risk management systems calls for further, bolder, and continuous adaptation of systems, workflows, and people. Moving from the ingrained notions of traditional source document review and point-to-point SDV to one of quality processes (focus on optimizing site processes, QC of site compliance, detection of fraud, IMP distribution chains, etc.) will encourage us to:

		Delineate QA from QC in a risk-based quality system.

		Identify standard approaches to centralized and on-site monitoring activities.

		Assess how information technology tools and associated processes can best support centralized and risk-based data review.

		Determine which metrics (thresholds/error rates) and alerts will drive the documentation of quality as well as trigger corrective actions when the quality thresholds are not met.

		Evaluate the impact on the preparation and planning for protocol-driven research.

	Risk is someone or something that creates or suggests a hazard. Sponsors of clinical investigations are required to provide oversight to ensure adequate protection of the rights, welfare, and safety of human subjects and the quality and integrity of the resulting data submitted to regulatory agencies; in other words, avoidance of a hazardous situation.

 As a result, risk must be defined and maintained across all stakeholders.

	Quality is degree of excellence. According to the FDA guidance, quality is a systems property that must be built into an enterprise and cannot be achieved by oversight or monitoring alone.

 Quality is something that needs to be maintained throughout the lifecycle of the product. The evaluation of the risk to quality should be based on scientific knowledge and connected to the protection of the patient.

	There is a need to pause to consider the difference between quality control and quality management. Quality control allows the ability to outline a process to spot check data and operation by reviewing a slice in time. Quality management on the other hand, is used to direct, control, and monitor quality as a more fluid and dynamic activity.

	The ICH Q9 guideline merges the two control processes as it speaks to quality risk management as a standard practice.

 Q9 provides guidance with examples that attempt to outline risk management across the pharmaceutical industry. Figure 1, taken from the EMA Reflection Paper,

 is an excellent starting place for organizations to turn as they design quality risk management; a systematic process for the assessment, control, communication, and review of risks to the quality of the drug (medicinal) product across the product lifecycle.

	The ICH Q9 and EMA reflection paper guidance documents provide good direction on what to address when crafting a quality risk-management plan. However, neither truly addresses regulatory expectations for minimal requirements to accomplish this in a holistic manner. While it is one thing to talk about managing risk through traditional data management and site monitoring processes, it is quite another to propose a concrete way to manage risk through trend setting (electronic transparency on entry time, account use, etc.) and standard metrics (time between tasks, accuracy of input, etc.). Variability in the extent to which organizations approach quality risk management may confuse the situation even more. Fear of having decisions not meet regulatory approval also may keep some companies from updating their approach to quality management. In addition, the payoff may not be as large as indicated.

	While there is no clear indication that quality is inadequate today, there is evidence that the industry overspends to perform tedious reconciliation between source and sponsor data that do not significantly affect the outcome of clinical trials.

	Traditional monitoring plans outline what the sponsor considers to be their responsibility and action plan for ensuring site compliance with the protocol including GCP and other applicable regulations. We should consider the value of combining elements of monitoring, data management, and statistical plans into a comprehensive integrated quality plan. It is important not to jeopardize the relationship between sponsor and the clinical sites and care should be taken to ensure that this remains in the forefront of the plan. For example, a communication plan is useful to document standard communication when there is no need for a site visit. This could be used to praise the site for good work and maintain monthly communication. Minimally, a consistent point of contact should be assigned to the site to represent the face of the sponsor.

	It is important to emphasize that monitoring plans are not unique to the CRA role but also can provide an overall cross functional plan that emphasizes what must be done in order to assure that the data received from clinical research sites are of high quality and integrity. The monitoring plans include not only what is expected to occur at a site level, but also within the data system and program level.

	A quality monitoring plan, which can be part of the clinical monitoring plan, may include information from traditional plans that had previously been separated into role or task based activity, for example, each therapeutic area may be different so several quality plans may be needed on that basis. In considering the monitoring plan, adequate risk assessment must be performed prior to the start of the study, in order to ensure patient protection as well as the quality and integrity of data. Our recommendation is that monitoring plans should exist at the program, protocol, and site levels, and outline what will be included in on-site monitoring as well as central monitoring.

This plan addresses overall quality aspects of the program and should be developed at the start of the clinical program. Risks that are known or anticipated for the disease population and investigational product under study should be documented and clear instruction given as to what, why, and how those risks should be monitored. As a general outline, a program level plan could minimally include the following:

		Planning-a priori identification of risks of the investigational product.

		Design-define protocol eligibility criteria, efficacy, and safety risk factors during design; for inclusion in the eCRF and edit build, as well as define key protocol deviations anticipated for study.

		Execution-clear direction in the oversight and monitoring of parameters leading up to study endpoints, integrated study level monitoring plan, clear direction on evaluating fraud.

		Analysis-clear definition of data measurement and reporting, defined elements of statistical analysis of protocol endpoints, safety, etc.

		Disclosure-publish outcome data, bringing all the above together in the clinical study report (CSR).

	Companies may wish to consider creating standard quality checks pertinent to all programs, and which may also include therapeutic area specific sections to be selected as relevant. Ensuring that the entire study team understands the principles and purpose of the program level monitoring plan will help avoid duplication and encourage study teams focus effort where it is most effective.

	This is an ideal opportunity to take a step back from familiar routines and look hard at current processes; assess what is valuable and what is not; invest in processes that bring most value from a quality perspective. The goal is to develop an integrated clinical data monitoring plan, based on compound and historical knowledge, which can respond proactively to data signals during trial execution.

 This plan should be specific to a particular protocol and draw upon the information from the program level plan as to what is needed to ensure quality. We suggest that the study level plans are categorized as those activities that occur during study startup, study execution, and study closure. This plan allows flexibility for site level variations, (e.g., highly compliant and performing sites, new sites with little experience, high volume sites, and non-compliant sites). The following specific minimal list of topics should be addressed in such a plan:

		Site selection factors-include key protocol requirements that affect decision making on site selection.

		Site training considerations-include information on how the site is prepared to conduct trial.

		Information on all forms of data review.

		Identification of critical data-safety, endpoint related.

		Delineation of what is reviewed on-site versus central.

		Assessment of site understanding of protocol.

	By focusing on study level tasks, rather than roles, study managers will be able to assign responsibilities to the most appropriate team member(s). This adds flexibility to the process, while ensuring that there are no omissions. It also will allow for role changes/combinations in the future. Plans should be easily readable and referable-limit bulk text; use graphics and bullet points. If a suitable electronic tool can be used, it should be feasible to pull out a sub-set of tasks for each function on the study from the overall plan.

 This plan should be specific to each investigative site, combining information from Program/TA level plans and the protocol(s) that the specific site is involved with. There is a large degree of variety in the way regional, country, or hub monitoring units are being organized, therefore a critical aspect of the risk assessment will be the concurrent (and possibly competitive) workload and commitments that shape the possibilities and constraints of both the sponsor/CRO and site staff when collecting and reviewing data. Another key consideration in defining a site level risk management plan is to never project any task or activity over time as being flat or a product of the volume of data flow only. What's more, it is an oversimplification (with potentially disastrous results) to even consider data quality as being inversely proportionate to its volume. Some of the minimally required aspects to cover in the site level plan are:

The eClinical Forum Risk Based Monitoring Taskforce offers some best practices for ensuring clinical data quality.

Dean A. Gittleman

Articles

eSource Records in Clinical Research: Keeping it Simple

eSource Records in Clinical Research

Three-Pronged Approach to Optimizing Trial Monitoring

Risk-Based Approaches