Jonathan Helfgott

The recent evolution within the pharmaceutical industry to the use of technologies associated with clinical trial data collection and storage is quite impressive. Even the risk-averse clinical trial sites are now comfortable departing from recording clinical trial data on pieces of paper to the use of technology-driven data collection solutions. As a result, new requirements must be put in place to ensure data quality and integrity (DQI). At the same time we must avoid adding new complexities to the clinical trial paradigm that attempt to solve problems that do not exist. We suggest referencing an article entitled "When Should the Audit Trail Begin? co-authored by the eClinical Forum, published in 2021, as a particularly good start.

 The article addresses issues such as evaluating software that has the capability to track keystrokes prior to hitting “submit” in an electronic data capture (EDC) system. The authors concluded that if companies or regulators would recommend this as a requirement, it would add additional workflow data for analysis without any proof of value to DQI, or the ability to address any perceived malfeasance. What all this means is that we have be cautious when instituting any new quality associated technology so as not to add requirements/features just because we can. The pharmaceutical industry and regulators must also not use paper-based processes as gold standards as they critically review the root reasons behind any new rules. Any new rules must be clearly documented why they are indeed necessary and how they add to DQI.

The historical record of writing things down started around 3100 BCE when people started to write on clay tablets.

 Around 3000 BCE, the Egyptians begin writing and painting on papyrus, a material prepared from the stem of the water plant Cyperus papyrus. Ordinary leather in the form of parchment began to be used for writing around 2500 BCE. During the second century BCE, a better form of parchment was introduced such that both sides could be written on, but it was not until 400 years later that parchment rivaled papyrus. According to Chinese tradition, paper as we know it today was invented in the year 105 CE by Cai Lun, a eunuch who resided at the imperial court. Fragments of this paper product, made from rags and the fibers of mulberry, laurel and Chinese grass, are currently available in museums. Nevertheless, parchment was the standard writing surface for European scribes up and until the 15th century. Historically, for the Jewish bible, a Torah or biblical scroll could only be written on parchment from the skin of a kosher animal, and the five pages of the U.S. Constitution as well as the Declaration of Independence, the Bill of Rights, and the Articles of Confederation were all written on parchment.

A detailed summary of clinical trials that were conducted between 562 BCE and 1537 was published in 2010.

 It should be noted that during these early times there were no regulators other than kings and curious researchers, and no papyrus or parchment records were being checked by clinical research associates (CRAs). According to the article, the world's first clinical trial is recorded in the Bible’s “Book of Daniel.” This experiment was conducted by King Nebuchadnezzar of Babylon who conquered Jerusalem in 586 BCE and destroyed the Jewish Temple. During his rule, there was a time when all subjects were ordered to eat only meat and drink only wine to keep them in sound physical condition. However, when several of the king’s relatives who preferred to eat vegetables objected, the king allowed them just to eat legumes and drink water for 10 days. When the experiment ended and those eating vegetables appeared better nourished than the “meat-eating controls,” the king graciously permitted the vegetarians to continue on their diet. According to the article, this may have been the first time in the evolution of humans that an open-label clinical trial guided a public health decision.

More recently, the first “controlled” clinical trial was performed by Dr. James Lind (1716-1794), who while working as a ship’s surgeon, was appalled by the high incidence and mortality of scurvy amongst sailors. As a result, he planned and executed a comparative trial of the most promising cures for scurvy. His insightful and vivid description of the trial covers the essential elements of a controlled clinical trial. Dr James Lind's “Treatise on Scurvy” was published in Edinburgh, Scotland in 1753.

Early computers had a type of disk storage composed of a thin and flexible disk using magnetic technology. The original “floppies” stored 384K of data. Later, 720K (double-density) and 1.44MB (high-density) disks were developed. In the mid-80’s, Dr. Fred Wolfe’s rheumatology clinical research unit in Wichita, Kansas had electronic records stored on a computer with backups on floppy disks. Dr. Wolfe had a plan to ship the floppy disks directly to the data management department for processing at the pharmaceutical company where the senior author of this article was running the clinical trial. This could have been the beginning of direct data capture and the elimination of source data verification (SDV). Unfortunately, the idea did not gain acceptance and was abandoned.

Evolution of paper-based data collection in clinical trials

Until the onset of the use of EDC systems in the 1990’s, clinical trial case report forms (CRFs) were composed of three-part NCR (no carbon required) paper with the original copy remaining at the site, and the other copies submitted to the sponsoring pharmaceutical company. The capture of clinical trial data directly onto the paper CRFs was not permitted. Instead, an independent original source record was to be maintained at the clinical research site and the basic function of the CRAs at the “site visit” was to compare the precision and accuracy of the CRF data transcribed from a source record onto a paper CRF, a process defined as source data verification (SDV). In order to complete the data validation process, the paper CRFs were then sent to the pharmaceutical company for data entry into the study database by two separate individuals, a process known as double-key data entry. These data were then compared and any inconsistencies resolved.

Once EDC systems were introduced, paper CRFs were eliminated, and data entry into the study database was shifted to the clinical research sites. This process did not change the SDV workflow as it was still no longer certain that site staff always entered the data accurately into the EDC system. While the data within the databases of the EDC systems could have been used as source data, it was not initially accepted that data entered directly into EDC systems could replace the “source” and “transcribed copy procedures” adopted during the use of paper CRFs.

Figure 1: EDC data workflow when paper source records are used

The transition to direct data capture and eSource technologies

In 2011, a study evaluated the effects of SDV on the mean and standard deviation values of five variables when using an EDC system together with paper-based source records.

 The protocol was for a new treatment in men with lower urinary tract symptoms associated with benign prostatic hyperplasia. Data were collected from 566 subjects who signed informed consent and from 492 of these same subjects who were subsequently randomized. Results showed that post data validation, there were virtually no differences in mean values of the five key variables and a very minor decrease in the standard deviations.

In 2012, in lieu of initially capturing all source data using paper records, a US-based clinical trial evaluating the pharmacokinetics of a topically applied drug product allowed the clinical site to use direct data entry (DDE) into an EDC system at the time of the clinic visit.

 The clinical research team also implemented a risk-based monitoring (RBM) plan which defined the scope of SDV 1) if paper records were used, 2) the frequency and scope of online data review, 3) the role of centralized statistical monitoring (CSM) and 4) the criteria for when to perform in-person monitoring at the clinical research site. As a result of this novel approach to clinical research operations, there were1) no protocol violations as screening errors were identified prior to randomization, 2) minimal transcription errors as very few paper source records were used, and 3) major reductions in onsite monitoring tasks compared to comparable studies that used paper as source records. As needed, EDC edit checks were modified early in the course of the clinical trial, and compliance issues were identified in real time and corrected. From the safety perspective, there was rapid transparency and detection of safety issues. The clinical research site estimated that just in terms of data entry, it was able to save 70 hours of labor by eliminating paper as the original source records. The article concluded that when DDE and RBM are properly implemented, there could be major increases in productivity for both sponsors, clinical sites, and CROs, as well as reduced times to database lock and the statistical analyses.

Figure 2: Direct data entry/eSource workflow

In terms of the impact on query generation, an article published in 2014 reported the results of study that evaluated the use DDE at the time of the patient encounter on the effect of the query effectiveness ratio (QER), defined as the ratio of the number of queries leading to change divided by the total number of queries issued.

 The study looked at 38,207 forms in a phase 2 study for the treatment of migraine. Results showed that of the 1,397 (3.7%) of forms that were queried, only 782 (2.0%) forms required changes to the database. Of these changes, 294 (37.6%) were in the concomitant medication forms and none of the changes had any impact on the study results.

Traditionally, the major roles of the CRAs were to train the clinical sites on the steps needed to execute the clinical trial and then to assess, via SDV, how the clinical sites could accurately transcribe data from paper source records onto paper CRFs. Data management wrote programs to look at logical errors and inconsistencies among and between forms, and communicated these issues directly to the CRAs or via a query directly to the clinical research site. The number of data entry errors was often used as the main metric to see how well a site was executing the study. As a result, this approach focused more on process conformance rather than on protocol compliance. Unfortunately, due to this manual effort, by the time an error was identified by the CRAs or data management, the same error could have been repeated multiple times due to the time between data entry by the site and monitoring by the CRAs.

Currently with the application of DDE in various software solutions including EDC, Electronic Patient Reported Outcomes (ePRO), Electronic Clinical Outcome Assessment (eCOA), Electronic Informed Consent (eICF), Mobile Medical Devices, etc., there is no need to question the data as the system validation efforts at the time of data capture assure the data integrity. Real-time remote monitoring by CRAs and data managers can now focus on 1) protocol compliance, 2) illogical results/outliers, 3) high risk issues related to safety and efficacy endpoints, and 4) evaluation of potential protocol deviations and violations. As a result, DDE logic errors that are systemic and not user specific can be corrected rapidly and communicated to those performing data entry. It is also now possible to evaluate the performance of the CRAs in detecting issues.

However, if debunking “accuracy” is no longer highly critical in the data validation process as one usually assumes, then what is? The recently published ICH E8(R1) emphases the need for “completeness (lack of missing information)” and “consistency (uniformity of ascertainment over time).”

 Thus, it is reasonable to expect that narrow-focused / accuracy-specific edit checks will play a lesser role over time, while at the same time, the part of aggregate (“across subject”) and complex statistical checks will expand. Data management and clinical operations professionals should not only be ready for this paradigm shift, but lead the transition. This advancement will demand a shift in perception and processes and equally importantly, the development of new skills. Let us now recognize that “Data Science” is no longer exclusively the domain of statisticians, but is now associated with all clinical research operations dealing with data collection, validation, and monitoring.

It is now time to follow the new definition of Data Quality documented by the Clinical Trials Transformation Initiative (CTTI), and redesign the data validation processes from eliminating all errors to the elimination of errors that matter to decision making. It is also time to focus on potential errors which have a meaningful impact on the safety of trial participants, the credibility of the results and the potential impact on the future care of patients.

Technology is providing major contributions to the execution of clinical trials and has assisted in the improvement of DQI. However, there are still major challenges ahead to increase efficiencies, avoid redundancies, and ensure compliance with all aspects associated with data capture, data transfer and final reporting. The creation of lean documented processes and procedures will enable technology and human interactions to expedite clinical development, while at the same time protect the data and the basic principles that govern quality as documented in existing regulations.

With the implementation of DDE, RBM and technology solutions, we can now early in a clinical trial identify potential risks to the patients and to the study itself. Real time risk mitigation strategies must address those risks that are most critical for the evaluation of the safety and efficacy profile of the drug or device under study. Regulators can now shift their focus from how well a data point is transcribed from one medium to another, to issues that could put patients/study participants at risk and compromise the clinical trial outcome, such as lack of protocol compliance, incompetent or unscrupulous investigators and biased sponsors. It is now time to also avoid the profound effects on patients and sponsoring companies where studies fail due to lack of quality rather than lack of efficacy.

., is the President and CEO of THI Pharma Services Inc.

, BS, is the President and CEO of Fasor Technical Services, Inc.

, BE, is the Regulatory Affairs Manager for Videoc Technologies.

, MS., is the Faculty / Program Coordinator / Senior Lecturer, MS in Regulatory Science & Food Safety Regulation Programs, at Johns Hopkins University.

Sr. Director, Data Quality Science and Programming, at BioMarin Pharmaceuticals, Inc.

Mitchel, J., Sugiura, Y., Arnera, V. et al. 

When Should the Audit Trail Begin? Applied Clinical Trials

Papyrus: A Brief History – Dartmouth Ancient Books Lab

Evolution of Clinical Research: A History Before and Beyond James Lind. Perspectives in Clinical Research

Evaluation of Data Entry Errors and Data Changes to an Electronic Data Capture Clinical Trial Database

. Drug Information Journal, Vol. 45, pp. 421–430, 2011.

Mitchel, J., Schloss Markowitz, JM, Hua (Helen) Yin, et

. Lessons Learned from a Direct Data Entry Phase 2 Clinical Trial Under a US Investigational New Drug Application

. Drug Information Journal 46(4) 464-471 2012.

. Query Effectiveness in Light of Risk-based Monitoring (RBM). Data Basics Winter

Time to Change the Clinical Trial Monitoring Paradigm. Applied Clinical Trials

ICH E8 (R1) GENERAL CONSIDERATIONS FOR CLINICAL STUDIES 

https://database.ich.org/sites/default/files/E8-R1_Guideline_Step4_2021_1006.pdf

CTTI’s QbD Project. Recommendations available at: 

https://tracs.unc.edu/docs/regulatory/CTTI_Recommendations-Quality_by_Design.pdf

Articles

New requirements must be put in place to ensure data quality and integrity.

The Transformation of Clinical Trials from Writing on Papyrus to the World of Technology

The FDA and other regulatory bodies have issued multiple guidance documents addressing technology tools that are used in clinical trials. FDA guidance documents include Computerized Systems;

 is also available online. In 2010, the European Medicines Agency (EMA) issued a Reflection Paper on electronic source records in clinical trials,

 and recently, the Medicines and Healthcare Products Regulatory Agency (MHRA)

 issued a document on data integrity. In 2016, the Chinese FDA issued Technical Guidelines on Electronic Data Capture for Clinical Trials.

 issued a document addressing the use of CDISC standards to facilitate the use of electronic source data in clinical trials.
	
	The regulatory push to the paperless clinical trial has occurred despite a pharmaceutical industry that has been risk-averse in adopting modern-day technology tools that could support clinical trials. This risk aversion is in part due to fear by sites and sponsors of receiving a FDA Form 483.

 What if the FDA turns down an application if it discovers that, for example, a patient was born in 1982, when in the study database it is recorded as 1983, even when there is no impact on the study results? The clinical sites are also fearful of losing business as a result of any FDA Form 483 finding, however minor.
	
	The major stakeholders in the clinical trial enterprise include study subjects/patients, sponsors, clinical researchers, regulators and the public at large. Now that the electronic world has entered the clinical trials space, as part of routine pre-approval inspections there will be an assessment of the impact of

 electronic systems on 1) informed consent; 2) data integrity and accountability; 3) protocol compliance; 4) monitoring of the study by the sponsor, including risk-based monitoring; 5) patient safety by assuring that all safety events are accurately collected and reported, and 6) drug/device accountability.
	
	The following are examples of regulated clinical trial software currently being used:

		Electronic patient reported outcomes (ePRO)

		Electronic clinical outcome assessments (eCOA)

		Dedicated tablets collecting and storing data in real time and transmitting data on a schedule to the study database

		Web-based systems collecting data in real time which transmit the data to the study database only after the source record is securely received and stored in an independent eSource storage location under control of the study site

		Mobile-based medical apps whose functionality could pose a risk to a patient’s safety if the app were to not function as intended


	
	It is strongly recommended that a risk-based approach to software validation be assessed for any medical software/app, even those not being used in clinical trials, as there are moral and ethical issues if any software used for medical decision-making does not act as intended.
	
	For software to be “reliable,” it needs to validated. Therefore, assuring that software is in a validated state is a key factor when regulators look at software products used in clinical trials. Fortunately, in 2002, FDA provided validation guidelines for software supporting regulated products.


	
	The following sections address some of the needed steps to assure that software used in clinical trials will be adequate to support regulatory scrutiny during pre-approval inspections and the review of marketing applications.
	
	


	
	Here are some critical questions to be asked concerning the “intended use” of the software, and depending on the answers, appropriate steps must be put in place.
	
	1. Why am I using the software?
	2. How is it being used?
	3. Will the data be used to support decision-making such as:

	4. Will the data be used to support primary and secondary outcomes?
	5. Will the data be used to support subject’s safety?
	6. How will patient privacy and confidentiality be maintained?
	
	


	
	A software product used in clinical trials must be validated and compliant with the Code of Federal Regulations (CFR) incorporated in Title 21 (Food and Drugs), Chapter I, Subchapter A, Part 11.

 21CFR Part 11, as it is commonly referred to, sets forth the criteria under which the FDA considers electronic records, electronic signatures, and handwritten signatures executed to electronic records to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper.
	
	There is a distinction between validation and compliance. Validation is objective evidence which demonstrates that a given piece of software performs as intended against the user needs and the documented specifications and requirements.

	While it is true that software requirements and resulting features within the software can support compliance with 21 CFR Part 11, it is also true that no software or hardware system can be 21 CFR Part 11 compliant on its own. The software implementation, including processes and procedures for use of the system, are what constitute 21 CFR Part 11 compliance.
	
	And validation of the software goes beyond just the software requirements and resulting features necessary for Part 11 compliance. Validation is necessary for the entire software application and should be a defined part of the Software Life Cycle (SLC).
	
	


	
	There should be very clear SLC management documentation that contains the software development process from cradle to grave. In fact, many SLCs can go beyond the development process, to include ongoing support and even consideration for end-of-life for the software.
	
	In terms of popular development methodology choices, there is the Agile development methodology versus the traditional Waterfall approach. For the former, when it is very clear what the deliverable will be, one makes a detailed plan and then implements the plan. In contrast, the Agile approach accepts that there will be changes during software development and has built in flexibility to change direction at a moment’s notice.
	
	The first validation element is the software concept/requirements, or in other words, what is the problem we are trying to solve and why do we need this software? Usually an executive or the person driving the software writes the concept document to convey the mission and rationale of this new endeavor. Next, the “requirements” phase must clearly document what the software intends to do. For example, the software should allow for study subjects to enter study outcome data over the web using any device and any browser.
	
	Once the requirements are buttoned down, there needs to be clarity on how the requirements will be executed, which comes under the heading of “functional specifications.” For example, if the requirement is that “changes to the data base need to tracked,” then the functional specification could say that there needs to be an audit trail which must display the original value, the new value, why the change was made, who made the change, and what was the reason for the change.
	
	The next phase is left to the programmers and architects who create the very technical “design document.” This document basically is in “geek-speak” and we leave that document to the specialists. Once programming begins, the programmers work closely with the “testers” to test and debug the program until at some point the development team says the software is ready for a formal user acceptance test (UAT). In an Agile methodology, the design document, test plan and test scripts are written concurrently with development, iterating as features are refined. In a Waterfall methodology, the test plan and test scripts are often written with the design document, all of which is completed before development even begins.
	
	Once the “test plan and test scripts” are written and signed off, formal testing can usually begin right away. During testing, the testers document the “errors, bugs and fixes,” and when all that is completed, the software is “released” and signed off by those responsible and accountable. Post-release, “version control and change management” kick in. A plan is also put in place for the retirement of the software when it will no longer be supported or commercially available.
	
	


	
	A quality by design (QbD) methodology, which includes a risk assessment and risk mitigation strategies, is a critical exercise when building software. Using a Boeing 787 analogy, a plane has several electrical systems that control the engine and the coffee machine. We should not assign the same risk to the coffee machine not delivering hot coffee as we do to the engine failing on takeoff. Therefore, in a clinical trial, we propose the following for each risk:

		Assign a value of the risk to the study participant if a software failure happens

		Assign a value of the risk to the study itself if a software failure happens

		Assign a severity score if a software failure happens

		Assign a likelihood of detecting the software failure

		Then create a score and a risk-mitigation strategy for each risk


	
	A well-designed and well-written protocol can be viewed as one approach to an ideal quality manual. Therefore, in addition to standard protocol elements that we are all familiar with, the protocol should identify where and when electronic systems will be used, and then explicitly describe at a minimum, 1) all electronic systems being used; 2) the reason for their use; 3) how the systems will be used; 4) how the software will be controlled; 5) the basic elements of the risk assessment and risk mitigation strategies and; 6) data ownership and access controls.

	The inspectors will not usually do an assessment of the protocol design, as that was already done by the clinical team at the regulatory agencies. What the inspectors will do is attempt to determine whether 1) the protocol was followed; 2) there are SOPs supporting the protocol; 3) the site and sponsor followed what was stated in the protocol and SOPs; 4) the electronic case report form (eCRF) was intelligently designed to support protocol compliance; 5) there were system-generated management reports of site communications and protocol deviations/violations; 6) there was an integrated risk management plan and risk mitigation strategy; 7) there was a substantiated and complete root cause analysis of observed issues/events; and 8) there were follow-up corrective and preventative action plans (CAPAs) and that the follow-up actions were resolved as planned.
	
	From the safety perspective, the inspectors will want assurances that there was accurate reporting of safety events by evaluating 1) how safety data were collected; 2) the process for determining the source of original safety data based on a data source agreements; 3) a sample of source data from disparate sources, including review of source records written on pieces of paper, if relevant, and; 4) subject records maintained at the study site.
	
	

Are electronic source data reliable and fit for purpose?


	
	We are asked many times by those committed to paper records-sponsors or regulators-that if the system is fully electronic, how do we know that the data are real if there are no source records with which to compare the electronic record? Ironically, you can ask the same question about paper records; unless there are major controls of paper, notebooks, etc., nothing prevents a piece of paper from being destroyed and replaced by a new one. Bottom line, as some suspicious data trends are impossible to spot through manual review of records, 

fraud and dishonesty can usually only be identified through adaptable, validated checks and the use of informatics and multidisciplinary expertise. Clearly then, misconduct is not dependent of the collection medium of the original record (paper vs. electronic). Consequently, in a double-blind, randomized trial, one effect of fraud, if it occurs, could affect the study results by demonstrating that a potentially effective drug is ineffective from a statistical perspective (Type 2 error). This is where study monitoring techniques and fraud detection programs become vital.
	
	Therefore, when using off-the-shelf or proprietary software systems, the following are some of the quality questions that should be considered:

		Can the software provide any assurances that the study subjects are real?

		Are there validated systems to assure user authentication and control?

		Did the sponsor perform installation qualification (IQ), operational qualification (OQ) and performance qualification (PQ)?

		How was access to the original data controlled?

		Were there date and time stamps for all transactions?

		Were there audit trails that can’t be overwritten?

		How was the data transferred to the study database?

		Can patient privacy and confidentiality be maintained?

Pre-approval inspections of an eSource program


	
	Based on our experience and the experience of others, when the regulators/inspectors arrive, they will need to review:

		Your organizational chart as it relates to the clinical trial

		Copies of relevant contracts/agreements

		Documents supporting vendor qualifications

		Documents supporting software validation/testing

		Corrective actions taken, reasons for actions taken, and the results of any actions

	As of this publication, two regulatory marketing applications have been submitted to FDA where direct data entry at the time of the office visit took place in lieu of maintaining paper source records. From the perspective of the pre-approval inspections at the clinical research sites, there were no Form FDA 483 findings related to study monitoring oversight and any of the electronic systems that were used for data collection. The inspections occurred at three of the five major medical centers that were inspected for the first application, and at five of the 20 outpatient clinical research sites that were inspected for the second application.
	
	The following are the redacted excerpts of the FDA inspection of the CRO (Target Health Inc.) for the first program that received marketing approval where the clinical research sites entered the majority of the data directly into the EDC system at the time of the clinic visit, and all documents were maintained in an electronic Trial Master File (eTMF; target document).
	
	

		This FY 2015 CDRH High-Priority Class II Device Application Inspection was conducted.

		This inspection was limited to the areas of compliance program 7348.810, “Sponsors, clinical research organizations and monitors.”

		Data line listings were provided with the background package, for comparison to the site eCRFs.

		An FDA 483 inspectional observation was not issued to management during this inspection.

 The clinical project manager develops a clinical data monitoring plan and safety monitoring plan for each study, using a company-specific template; these plans are approved by the sponsor before any site monitoring takes place.

How to assure that software used in clinical trials will support regulatory scrutiny during pre-approval inspections and application review.

Regulatory Considerations for 21st Century Paperless Trials

	The informed consent process is based on a trusted interaction between a study subject and the clinical research investigator. In 1991, the “Federal Policy for the Protection of Human Research Subjects,” based on the 

 and known as the “Common Rule,” was issued by 15 federal departments and agencies.

 The last revision of the Common Rule was in 2009.
	
	FDA regulations concerning the protection of human subjects are codified in 21CFR Part 50,

Draft Guidance on Electronic Informed Consent

 The draft guidance was prepared by the Office of Medical Policy (OMP) in the Center for Drug Evaluation and Research (CDER), the Office of Good Clinical Practice (OGCP) in the Office of Medical Products and Tobacco, in coordination with the Center for Biologics Evaluation Research (CBER) and the Center for Devices and Radiological Health (CDRH).
	
	The following are points to consider when designing and using an electronic informed consent form (eICF):
	
	

	If a subject cannot or does not want to sign the informed consent form (ICF) electronically, there should be the option for the clinical research site to document in any electronic data capture (EDC) system that a paper process was used. Likewise, if a stand-alone eICF system is used via a dedicated tablet, for example, there should be ways to seamlessly “inform” the EDC system that the eICF has been signed. For web-based eICF systems that are fully integrated with EDC systems, access to the data entry screens should be controlled until the signature process is finalized. In the world of Bring Your Own Devices (BYOD), access to the eICF should also be browser independent.
	
	

	When using an eICF, the clinical research site and companies sponsoring the clinical trial should clearly define the eICF process as well as the roles and responsibilities of each stakeholder in standard operating procedures (SOPs) and in the study protocol. For example: How is version control managed? How are different languages managed? Who is hosting the eICF software application? How is website access controlled? How are questions created and answered? What happens if the website is down temporarily? Who is managing the user accounts?
	
	

	Attribution, as well as binding of the contents within the eICF to the electronic signature and the signatory, must also allow for the eICF to be compliant with the electronic recordkeeping and electronic signature requirements found in 21 CFR Part 11,

 and the FDA Guidance for Industry on Computerized Systems Used in Clinical Investigations.


	
	When a study subject electronically signs the ICF, there must be confidence that the signature can be attributed to the actual person participating in the clinical trial, and that the subject cannot repudiate the signature once invoked. For example, there must be a verification step to assure that the person signing the eICF is the actual person participating in the clinical trial, and it is not possible to delete the signature. Because technology is always changing and subject authentication can be addressed in multiple ways, wisely, the FDA did not require a specific authentication process in the draft guidance, but left it up to the sponsor to decide. While sponsors could assess technology tools associated with user authentication, especially for remote and virtual clinical trials, costs or complexity should not be added to the ICF process.
	
	

	As we introduce eICF systems, there may be a tendency to make the eICF process more complex than needed. Some may want to create videos to assist in the process or require tests to assess comprehension. While these approaches may add to the success of eICF implementation, we must be aware of unintended consequences especially when the eICF is used in global multicenter studies, when just managing versions and multiple languages within and between sites can be daunting.
	
	

	When Signing the ICF Electronically It is common practice for CRAs and regulators to document that each and every study subject signed the correct version of the ICF. Imagine the time, cost savings and other efficiencies if there was a simple validated online report to track the time and date of each electronic signature.
	
	

	After signing the ICF, the study subject must have access to the eICF and it should be printable. For printing, a .pdf file is preferable as it can be printed with any computerized system supporting an Adobe reader.
	
	

	Since the eICF is a source document, it should be under the control of the clinical site as would be a paper copy. To accomplish this, the eICF should be stored in an electronic repository with user management and access controlled by the study site.

 is Coordinator of the Regulatory Science Graduate Program at Johns Hopkins University.

http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html

		21 CFR Part 50; Protection of Human Subjects. Code of Federal Regulations, Title 21, Volume 1. Revised, April 1, 2015. (

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=50

		FDA Draft Guidance Use of Electronic Informed Consent in Clinical Investigations, March 2015. 

http://www.fda.gov/downloads/drugs/guidancecomplianceregulatory		information/guidances/ucm436811.pdf

		FDA Guidance For Industry. Part 11, Electronic Records; Electronic Signatures - Scope and Application, August 2003. 

http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm125125.pdf

		21 CFR Part 11. Code of Federal Regulations 

http://www.ecfr.gov/cgi-bin/text-idx?SID=c654e9b1077d05f0b0bb0fbdd45f00b8&mc=true&tpl=/		ecfrbrowse/Title21/21cfr11_main_02.tpl

		Guidance for Industry Computerized Systems Used in Clinical Investigations, May 2007 

http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation		/guidances/ucm070266.pdf

When designing and using an electronic informed consent form, there are many factors that should be considered when applying to clinical research.

The Electronic Informed Consent Has Arrived

 What if the FDA turns down an application if it discovers that, for example, a patient was born in 1982, when in the study database it is recorded as 1983, even when there is no impact on the study results? The clinical sites are also fearful of losing business as a result of any FDA Form 483 finding, however minor.
	
	The major stakeholders in the clinical trial enterprise include study subjects/patients, sponsors, clinical researchers, regulators and the public at large. Now that the electronic world has entered the clinical trials space, as part of routine pre-approval inspections there will be an assessment of the impact of electronic systems on 1) informed consent; 2) data integrity and accountability; 3) protocol compliance; 4) monitoring of the study by the sponsor, including risk-based monitoring; 5) patient safety by assuring that all safety events are accurately collected and reported, and 6) drug/device accountability.
	
	The following are examples of regulated clinical trial software currently being used:


	
	We are asked many times by those committed to paper records-sponsors or regulators-that if the system is fully electronic, how do we know that the data are real if there are no source records with which to compare the electronic record? Ironically, you can ask the same question about paper records; unless there are major controls of paper, notebooks, etc., nothing prevents a piece of paper from being destroyed and replaced by a new one. Bottom line, as some suspicious data trends are impossible to spot through manual review of records, fraud and dishonesty can usually only be identified through adaptable, validated checks and the use of informatics and multidisciplinary expertise. Clearly then, misconduct is not dependent of the collection medium of the original record (paper vs. electronic). Consequently, in a double-blind, randomized trial, one effect of fraud, if it occurs, could affect the study results by demonstrating that a potentially effective drug is ineffective from a statistical perspective (Type 2 error). This is where study monitoring techniques and fraud detection programs become vital.
	
	Therefore, when using off-the-shelf or proprietary software systems, the following are some of the quality questions that should be considered:

a) The clinical data monitoring plan for the inspected study was reviewed and obtained.

 This study utilized direct entry of data, by the sites, into the EDC system using the eSource process (Target e*CTR; eClinical Trial Record).

a) This process allows for the original data to be stored in PDF format, in the repository, prior to being transmitted to the EDC database. The PDFs serve as original records.

The time is now for sites and sponsors to put away fears associated with data acquisition and monitoring technologies-and better support the regulatory push in recent years for standardized adoption of paperless trials.

Regulatory Considerations when Designing and Running 21st Century Paperless Clinical Trials

There has been a recent thrust within the pharmaceutical industry to promote the transformation of the clinical trial process. In 2008, the Clinical Trial Transformational Initiative (CTTI), a public-private partnership with the FDA, and Duke University, was established to identify practices that, through broad adoption, will increase the quality and efficiency of clinical trials. A major publication by CTTI of monitoring practices has helped to transform the monitoring of clinical trials.

	According to the Food and Drug Administration (FDA) eSource Guidance of 2013: “Electronic source data are data initially recorded in electronic format. They can include information in original records and certified copies of original records of clinical findings, observations, or other activities captured prior to or during a clinical investigation used for reconstructing and evaluating the investigation.”

 Saying it differently, eSource data are original subject data that are collected digitally without having to record the data on a piece of paper first, and then transcribe it to an electronic data capture (EDC) system.

	It has been shown by many that changes made to the study database as a result of source document verification (SDV) affect neither study results nor the interpretation of those results.

 Therefore, despite the support of regulators, including the FDA

, it is difficult to explain 1) why many of those managing clinical research operations they still respond to a perceived risk, instead of adopting systematic and scientifically substantiated approaches for data management and cleaning, and 2) why there has been tepid adoption of eSource solutions coupled with “intelligent” monitoring of clinical trials. Perhaps failure to appreciate the wisdom contained in the quotes below explains why those doing clinical research spend a lot of their time on non-productive activities.

“Life is really simple, but we insist on making it complicated”

“Simplicity is the ultimate sophistication”

“The art of being wise is the art of knowing what to overlook”

	There is broad agreement across the pharmaceutical industry that it no longer makes sense to collect clinical trial data by writing results and observations first on a piece of paper, then to transcribe those data into an EDC system, and then to save that paper so that a clinical research associate (CRA) can check how precisely a person transcribed information from a piece of paper into the EDC system. Sadly, we continue to see residual, frequent emphasis on checking these transcription activities when electronic systems can easily eliminate this highly inefficient process. Doing the math, moving to electronic systems can provide a dramatic return on investment by 1) reducing labor costs at the clinical research sites and sponsoring companies, 2) reducing travel costs, and 3) providing improvements in data quality.

	In 2006, the Electronic Source Data Interchange (eSDI) Group within CDISC issued a paper entitled “Leveraging the CDISC Standards to Facilitate the use of Electronic Source Data within Clinical Trials.”

 The eSDI document was the product of the CDISC eSDI Initiative, the purpose 

of which was “to investigate the use of electronic technology in the context of existing regulations for the collection of eSource data (including that from eDiaries, EHR, EDC) in clinical trials for regulatory submission by leveraging the power of the CDISC standards, in particular the Operational Data Model (ODM)." The overall goal of the initiative was to make it easier to collect data once in an industry standard format.

	Eight years later, on Jan. 29, 2014, FDA addressed these issues in a webinar reviewing the final guidance for industry titled “Electronic Source Data in Clinical Investigations."

Leonard V. Sacks, associate director, Office of Medical Policy (CDER), Ron Fitzmartin, Office of Strategic Programs (CBER), and Jonathan S. Helfgott, associate director for risk science (acting), Office of Scientific Investigations (CDER).  Clearly, FDA is supporting efforts by industry to adopt eSource solutions.

	Even earlier, however, in 2010, the EMA GCP Inspectors Working Group eSource subteam released the “EMA Reflection Paper On Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials.”

 The reflection paper-which is very similar in nature to an FDA guidance-was followed up by direct exchanges with industry representatives (the Society for Clinical Data Management and the eClinical Forum, among others). Those discussions are captured in a series of questions and answers that are frequently updated on the EMA website.

 of an independent contemporaneous investigator copy (ICIC) of the case report form (CRF) has led to numerous and passionate debates among the professional and industry associations. The basic concept of the ICIC is still far from being consistently understood and implemented in the different geographies; regardless, ICIC has had a considerable impact and has brought about an extension of the concept of ALCOA, that source data should not only be

attributable; legible; contemporaneous; original; and accurate, but also complete; consistent; enduring and available, when needed

As a result of this lack of consensus of the regulatory understanding of how to optimally generate and review electronic source records, apprehension and uncertainty among solutions vendors, contract research operatives, and pharmaceutical company sponsor remains.

 Good Clinical practice states that “The sponsor should ensure that it is specified in the protocol or other written agreement that the investigator(s)/institution(s) will permit trial-related monitoring, audits, IRB/IEC review, and regulatory inspection(s) by providing direct access to source data/documents.” There are no statements that paper records must be kept, just that there needs to be direct access to source data/documents.

	The FDA, in September 2013, issued a final “Guidance for Industry: Electronic Source Data in Clinical Investigations.”

 This guidance is consistent with the EMA “Reflection Paper on Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials.”

	Regulators seem ahead of the game in supporting the transformation of how the pharmaceutical industry manages the performance of clinical trials. The following summarizes aspects of the perspectives of FDA and EMA.

	According to the FDA Guidance, source data include all findings, observations, or other activities in original and certified copies of original records, which are used by regulators to reconstruct and evaluate a clinical trial. All interested parties need to review source data and ensure adequate protection of human clinical trial subjects and the quality and integrity of the clinical data. Source data should be ALCOA and must meet the regulatory requirements for recordkeeping. FDA has proposed that electronically capturing and transmitting source data to the eCRF should:

		Encourage entering source data during a subject’s visit, where appropriate

		Eliminate unnecessary duplication of data entry and transcription errors

		Facilitate remote monitoring of data and promote real-time access for data review

		Facilitate the collection of accurate and complete data

	According to the FDA, data can be entered into the eCRF either manually or electronically using “direct entry of data into the eCRF.”  For example, there are many data elements in a clinical investigation, such as blood pressure, that can be entered directly into the eCRF at the time of the office visit. FDA also indicated that “data elements originating in an electronic medical record (EMR) can be automatically transmitted directly into the eCRF.  Unlike a direct transmission to an eCRF from instruments or medical devices, EMRs can use intervening processes (e.g., algorithms for the selection of the appropriate data elements).” In cases where the EMR serves as the source, the pertinent data for the subjects in the clinical study should be made available for review during regulatory inspections and monitoring visits.

	And although many may know this, the FDA added that the eCRF should be capable of recording metadata (e.g., data about the data identifying who entered or generated the data and when the data were entered or generated).  Also, changes to the data must not obscure the original entry, and must record who made the change, when, and why. Data element identifiers should be attached to each data element as it is entered or transmitted by the originator into the eCRF. Data element identifiers for each subject should identify the:

		Originators of the data element (including those entered manually (e.g., by the clinical investigator(s)) or automatically by devices

		Date and time that the data element was entered into the eCRF

	These data element identifiers allow sponsors, FDA, and other authorized parties to examine the audit trail of the eCRF data and to provide information that will allow FDA to reconstruct and evaluate the clinical investigation.

	The following are from the EMA Reflection Paper

In order to meet the requirements of maintenance of an independent contemporaneous copy of source records, a certified copy of the data should be created before the transfer to the sponsor and retained at the investigator site. The method of transfer should be validated. 

The recording of a clinical observation is made at the same time as when the observation occurred. If this is not possible the chronology of events should be recorded. An acceptable amount of delay should be defined and justified prior to trial recruitment.

The source data and their respective capture methods should be clearly defined prior to trial recruitment (i.e., in the protocol or study-specific source data agreement). The sponsor should describe which data will be transferred, the origin and destination of the data, the parties with access to the transferred data, the timing of the transfer, and any actions that may be triggered by real-time review of those data. There should only be one source defined at any time for any data element.

	According to the EMA, the source data and their respective capture methods should be clearly defined prior to trial recruitment (i.e. in the protocol or study specific source data agreement). The sponsor should describe which data will be transferred, the origin and destination of the data, the parties with access to the transferred data, the timing of the transfer, and any actions that may be triggered by real-time review of those data. There should only be one source defined at any time for any data element. The EMA has focused on the issue of source data control and specifically stated that “the investigator should maintain the original source document or a certified copy (Requirement 5, ICH GCP 2.11, 5.15.1); that source data should only be modified with the knowledge or approval of the investigator (Requirement 6, ICH GCP 4.9.3, 4.9.4 and chapter 8); and the sponsor should not have exclusive control of a source document. (Requirement 10, ICH GCP 8.3.13).”

	The EMA also requires that “all data generated in a clinical trial relevant to patient care must be made available to the investigator at all times during and after the trial, and all data held by the sponsor that has been generated in a clinical trial should be verifiable to a copy not held by the sponsor.”  The EMA also stated that “the requirements above are not met if data are captured in an electronic system and the data are stored on a central server under the sole control of the sponsor.” The EMA requires that a “contemporaneous certified copy of the data should be retained at the investigator site in addition to the record maintained on a central server.”

	In addition, the business requirement of any electronic system can be summarized to include:

		To be a separate legal entity from the sponsor and from the investigator, with a detailed contract defining duties and responsibilities of each party

		An assured mechanism for investigators to be informed about and/or approves of modifications to the data should be clearly established, and this control by the investigator should be demonstrable 

		Any changes to the data should be captured by the audit trail

	The variations in regulatory understanding have confounded many stakeholders. In order to find a practical way forward, industry and professional associations should concentrate efforts on identifying and addressing concerns around clinical data quality. Specific provisions associated with quality assurance and the transparency of data flow should be present in the protocol, protocol-associated plans, and any contractual agreements with the vendors and study sites. Failure to create, implement, and document such risk mitigation associated with source data would be a major shortcoming, as it could:

		Hinder the scientific interpretability of the data-and renders any clinical decision support impossible

	Not surprising, the EMA has directly warned sponsors and contract research organizations (CROs) that documentation, procedural, and systemic shortcomings will compromise (or even block) application package submission to the Committee for Medicinal Products for Human Use (CHMP). Similar reactions in presence of blatant lack of compliance are to be expected in the case of NDAs (CDER), BLAs (CBER) and PMAs (CDRH).

Source records during development process

	Despite the philosophical and historical differences explained earlier, it is possible to implement a clinical data acquisition environment that meets all recognized regulatory expectations. Source records, for example, can be maintained in many formats, including but not limited to paper records (e.g., data files, XML files, and PDF files). Regardless of the medium or the format, compliant maintenance of source records:

		Assures the ability of independent auditors to confirm the accuracy of data presented to regulatory agencies by pharmaceutical and device companies

		Allows for the reconstruction of study data in the event of catastrophic loss of data by the sponsor

	Source data should not be the same “version” of the data that the sponsor (or CRO) “controls.” Confirmation of the accuracy of the data submitted to regulatory agencies can be accomplished by review of the validation of electronic systems used as original data sources, manually reviewing paper records and comparing results submitted by sponsoring companies and/or performing “data compare” of electronic source records to data stored in locked databases. Ideally, evaluating extracts of, or entire datasets against CRF data elements could provide corroborating evidence, and ultimately a measure of confidence, as to the accuracy of reported assessments and endpoints. This, however, would require access to the electronic source data from EMRs, machines used in the clinical trial process (e.g., raw MRI records) and EDC databases prior to transfer to the sponsor of the clinical trial. While all of these tasks can be used to assure precise data (i.e., data elements agree between databases), these processes do not holistically address data accuracy. Sensitivity analyses done on real clinical trial datasets have repeatedly shown that database changes, as a result of SDV and SDR, have little or no impact on final means and standard deviations (SD).

	Therefore, key areas of focus during monitoring, audits or inspections are the following:

		Were the clinical site personnel qualified, as documented by education and experience, to perform the activities for which they were responsible?

		Were the clinical site personnel properly trained on the protocol and the electronic systems used for the trial (e.g., EDC, eSource storage, etc.)?

		Did the clinical site understand and follow the protocol?

		Were there any differences in outcomes between sites, suggesting lack of understanding of the protocol or possible fraud?

		Were basic regulations followed that assure patient safety (e.g., proper informed consent and safety monitoring)?

	The above is certainly not exhaustive-but it is an example of the strict minimums that assure quality.

	Sites may maintain source records in many formats, including electronic data stored by an independent third party prior to the source data arriving at the EDC database. 

	Source records could come from direct data entry at the time of the study visit, from the EMR, from patient reported outcomes (PROs), and from output from machines. The sponsor should have no more access to electronic source records than if given access to a site’s paper source records. One advantage of the source record existing as a PDF file in the cloud, with read-only system privileges, is that it is impossible for registered users to manipulate the file. It is the same as viewing a document on the FDA or EMA website, where one can view but cannot modify the file. PDF files can also be uploaded into most EMR systems and searched if saved as “readable” files. Generating electronic source as .XML can further enhance the ability to search.

How to satisfy regulatory concerns about EDC data integrity and site controls over its source records.