Stephen A. Raymond

The biopharmaceutical industry is, perhaps, the most heavily regulated industry outside of nuclear power and aerospace. For the most part, this is because human safety is at risk. There is a clear mandate to ensure there are adequate protections in place to best safeguard the welfare of those who participate as subjects in clinical studies. To accomplish this, regulations require the establishment of oversight committees, including institutional review boards (IRBs) and independent ethics committees (IECs)—collectively referred to as Ethics Committees (ECs)and other bodies such as Data Safety Monitoring Boards (DSMBs). We will focus on ECs and examine how to optimize their time and effort to efficiently and effectively fulfill their human subjects protection remit.

The elements of these protections are set forth in ICH E6 R2: Guideline on Good Clinical Practices.

, the International Council on Harmonization(ICH) points out that “clinical trials have evolved substantially, with increases in globalization, study complexity, and technological capabilities.” They recommend that “to keep pace with the scale and complexity of clinical trials and to ensure appropriate use of technology we should modernize our approach to GCP to enable implementation of innovative approaches to clinical trial design, management, oversight, conduct, documentation, and reporting that will better ensure human subject protection and data quality.”

Focusing efforts on aspects of clinical trials with the greatest potential impact and/or harm (e.g., misleading or confusing information that might inappropriately influence a patient’s decision-making) would be consistent with the existing standards for risk-based monitoring of clinical studies.

review by a learned committee of every data capture document may protect clinical study subjects from potentially misleading or otherwise deficient content, and including such content together with the trial protocol is in keeping with the mission of ECs. Is there significant additional benefit in supporting review of final screenshots? Obtaining, reviewing, and filing all such screenshots requires a significant commitment of the committee’s time and effort. It should also be considered whether further evaluation of these images is ethically substantive or, rather, more of a quality control task that is a formal responsibility of COA system providers and Sponsor study teams. Does such a review represent an optimum use of the EC time, possibly causing them to spend precious time reviewing screenshots which share content with other data collection documents already under review? Conceivably, this could result in more harm than good—ultimately more of a burden than a clearly beneficial exercise.

Just as with other “protective” activities conducted during the course of a clinical trial, for example, risk-based monitoring, determination of causality associated with adverse events, and, adjustment of acceptable risk according to disease, one must consider the balance of risk of harm to prospective benefit. Thus, the burdensome task of converting from paper formats the representations of all of the screens to be used for data entry on devices must be weighed against the potential patient harm and, not insignificantly, the likelihood of delaying initiation of a clinical study while awaiting EC approval while all electronic versions of COAs are generated in final form for their review.

This issue is at the crux of this discussion—how do we achieve the optimum balance between research subject protection and practicality of the process of bringing new medical products, vaccines, diagnostics, and medical devices to patients in need, as well as advancing healthcare knowledge? If we unnecessarily delay availability of new medicines, we do a disservice to patients who may require innovative therapies.

Perspectives of each of the stakeholders will be considered in evaluating these issues and, in doing so, we will discuss the potential consequences that may result in harm or benefit both to individual research subjects, to the broader population and, indeed, to the research and development enterprise.

Contract research organizations (CROs) and sponsors now often request that data capture forms such as patient diaries, electronic patient-reported outcome (ePRO) assessments and other eCOAs be submitted to the EC. The request seems to be based on a broad interpretation of “written material to be provided to the patient” in ICH E6 R2 guidance rather than justification of relevance to patient safety or burden. From our experience, with names changed and circumstances altered to maintain confidentiality, the following scenario illustrates points in the execution of a study where procedural delay and complexity can occur.

Implementing an eCOA solution in clinical studies requires a substantial amount of activity, as shown in 

Figure 1: general eCOA implementation lead time

During the planning phase, the patient-reported outcome (PRO) measures and clinician-reported outcome (ClinRO) assessments to be programmed into the eCOA system are selected as part of the study protocol. 

During the design phase, the eCOA system provider faithfully migrates these COAs, based on industry migration best practices and on authors’ instructions. This migration process ensures that there are no changes in content, and that participants will interpret and respond the same way regardless of the mode of data collection. Screenshots obtained because of this faithful migration need to be approved by the author and/or an expert screen reviewer. In some instances, additional evidence such as cognitive interviewing and usability testing may be needed to confirm measurement equivalence.

Language implementation activities can begin once faithful migration has been finalized.

Such activities often include translations, and possibly linguistic validation.

The eCOA system software is then developed, validated by the eCOA system provider, and the system then undergoes UAT by the Sponsor or its designee, such as a CRO. Once UAT has been approved by the Sponsor, the electronic system is ready to be deployed in the original language.

The full complement of screenshots only becomes final in each original language when development, faithful migration, and formal evaluation activities have been completed. This is 7-to-14 weeks after the initiation of the planning phase. An additional 8-to-12 weeks needs to be added should cognitive interviewing and usability testing be required. At that time, the final screenshots can be taken and submitted to an EC for review.

Screenshots in targeted languages only become available 4-to-12 weeks after the finalization of the original language, depending on availability as templates, or if linguistic validation is required.

eCOA-related documents required for submission to an ec

Regulatory authorities worldwide have agreed, in most cases, to abide by guidance issued by the International Council on Harmonization (ICH), particularly ICH E6 R21, which applies to clinical studies conducted in the United States (US), European Union (EU), Japan, Canada, and other nations recognizing ICH E6 Good Clinical Practices (GCP) standards. The 2005 European Directive mandates Member States to harmonize processes to the GCP.

 While each national or regional authority has the freedom to issue specific guidance and regulation for studies conducted under their jurisdiction, the predominant underlying standards are those established under ICH. There are national differences regarding protection of patient/subject privacy; however, these will not be addressed in this manuscript.

ICH E6 R2 addresses documents to be provided to ECs, as follows:

3.1.2 The IRB/IEC should obtain the following documents.

Trial protocol(s)/amendment(s), written informed consent form(s) and consent form updates that the investigator proposes for use in the trial, subject recruitment procedures (e.g., advertisements),

(emphasis added) to be provided to subjects, Investigator's Brochure (IB), available safety information, information about payments and compensation available to subjects, the investigator’s current curriculum vitae and/or other documentation evidencing qualifications, and any other documents that the IRB/IEC may require to fulfill its responsibilities.

Note that the provision concerning written information is in the context of consent and recruitment, and does not specifically recommend that ECs obtain ‘screenshot representations’ of diaries, questionnaires, and clinician interviews that will be used to capture information. In contrast to the informed consent form, such scientific content for data capture is not physically provided to subjects to review and retain in conjunction with deciding to participate in an investigation. 

When consent forms are submitted for EC review, the EC will most often apply the approach that FDA recommends regarding EC review of audio or visual advertising, per the FDA Information Sheet “

 This guidance recommends that the EC review the storyboards and proposed text to identify potentially unduly influential language or images prior to incurring the expense of recording the final version. The guidance recommends that the EC then review the final versions of these documents. Using this logic, when an eConsent is submitted to the EC, if early in document development, the EC may ask for a storyboard and proposed text to perform an evaluation on both the initial and final versions.

The GCP Guidance includes, in Section 8, considerations on the Essential Documents for a study to support the review of the study by regulatory authorities. Section 8.2 includes a list of documents associated with “

the period before the clinical phase of the trial commences

should be generated and should be on file before the trial formally starts.

”1It should be noted that there is no specification that evaluation checklists, questionnaires, or diaries be obtained in all languages for review and approval as part of the “planning stage” of a trial. 

Section 8.2.3 applies to the “Information Given to Trial Subject” in the planning stage of a trial, and refers to “any other written information,” clarifying the purpose of such information as “

to document that subjects will be given appropriate written information (content and wording) to support their ability to give fully informed consent.”

Thus, guidance to ECs is narrowly focused on informed consent and written information related to recruitment. Nonetheless, some sponsors, CROs and ECs interpret the GCP Guidance to apply to 

 written information, regardless of relationship to inducement of subjects or whether shown/presented versus literally “provided” to the subjects. In addition, the content of these may be duplicative of other documents provided to the study subjects, adding little, if any, value in terms of useful information or increased comprehension. ICH E6 R2, Sections 3.1.2 and 8.2, instructs ECs to obtain a range of essential documents for review. 

The list does not mandate screenshots of 

that subjects will see during participation in a trial where eCOA is used to capture eSource records. Requiring screenshots of 

of all diaries and questionnaires into local languages is an expansive reading of ICH E6 Guidance.

For example, “Other written information to be provided to the subject” is a phrase used throughout ICH E6 R2 Guidance (Sections 3.1.2; 3.1.9; 4.4.1; 4.8.1-2; 4.8.9-11; 5.11.c; 5.11.2; 5.18.2; 8.2.3; 8.2.7; 8.3.2-3).1 The context is always limited to informed consent or incentives to subjects in recruitment, and never refers to the diaries, questionnaires, or other forms to be used for data collection, or to device instructions.

The provisions in the ICH GCP Guidance that identify “any other written information to be provided to the subject(s)” do not specifically mandate review of diaries and questionnaires.

 Nonetheless, providing the content and wording of such eCOA content to ECs in English is useful in determining the nature of the data that will be collected, its relevance to the objective of the trial (Section 2.5), and the possible burden on the subjects who will complete the questionnaires (Section 2.2).

 The scientists on the EC boards may also be interested in assessing the pertinence of any sensitive information that may be sought (Sections 2.2; 2.5; 3.1.2).

Specific national laws affirm the concept that it is not necessary to provide any COA instruments to the EC. For example, in French law there is no requirement that the CRF or any COA instruments be part of the dossier provided. The guidance is also silent as to whether the research documents are provided on paper or in electronic format.

The EC is primarily concerned with the protection of the rights, safety and/or well-being of the subjects (Section 3.1.5). Thus, an EC may request more information be given to subjects than is outlined in Paragraph 4.8.10 when, in the judgment of the EC, the additional information would add meaningfully to these protections.

Regulatory perspective (FDA’s office of GCP/EMAs IWG GCP)

GCP Inspectors do cover ECdocumentation during GCP-inspections and often request the documentation that was provided to the ECs and designated to be given to trial subjects.

Several inspectors have indicated that the ECs can decide whether screenshots should be sent separately or as part of the clinical trial protocol. However, the opinion of inspectors is that details about diaries belong in the protocol (appendix) or a related document – in which case it would also be provided to the EC and the competent authorities. We are in agreement with this process and believe that a paper version of the content of a diary or other COA should be part of the information submitted with the clinical trial application.

To our knowledge, after hundreds of inspections and audits of sites, sponsors, eCOA/ ePRO system providers, the provision of eCOAs only in English and only on paper as part of a protocol has never been identified as inadequate provision of “written materials.”


Based on the guidance from regulators and ECs, our understanding of the purpose of ethical review, as well as previous examples describing potential consequences of requiring a plethora of eCOA documentation, we are proposing the following framework for documentation to be provided to ECs, including both local and central IRBs:

Paper documents that convey the content and wording (but no need for final screenshots showing layout and appearance) of measures the subject is to complete as well as any COA forms to be entered by clinicians on the eCOA system; include a description of the Time-and-Event schedule, and planned schedule for the

 Often these capture instruments themselves and the scientific rationale for collecting associated data are already included in the protocol; however, if absent from the protocol, they should be listed in the EC submission. 

Descriptions of data capture devices to be used in the study, including any regulatory approvals (e.g., for medical devices such as glucometers and peak expiratory flow meters); summary of training for the subject on the devices; privacy and security protections; and summary of administrative functions required to implement the eSource data capture forms for entry, transmission, maintenance, and retention in the provider’s system

. This would provide a better sense of what the subjects would experience in the study and how the electronic systems would maintain security and privacy of data. It would supplant the need to show and explain every administrative screen, and their translations. This then obviates the need to generate and review potentially thousands of pages of documentation for global studies. Even if screenshots are reviewed, they do not typically include discussion of how the implementation of the system for capture may contribute to the scientific and ethical merit of the study.

The description of the processes followed to ensure appropriate translations are being obtained

. Certificates of translations may be provided, if requested, when completed. 

Screenshots of eCOA questionnaires as they appear on the devices, together with their translations. Requiring screenshots would be redundant, as eCOA system providers do not change the content and meaning of data collection forms when implementing electronic capture. Further, eCOA system providers perform “faithful migration” of the COA content and follow requirements of the COA instrument developer if the content is already developed for use electronically. To require screenshots – and translations - puts undue pressure on up-front timelines for study start-up, while not providing further information of value to ECs for their review.

Administrative screens that may appear before, or during, the COAs and screens that appear to direct the subject to complete items or advance to the next screen; rules for moving from one screen to the next, or how to transmit a questionnaire after it is completed. Administrative screens do not change the scientific nature or the meaning of the questionnaires. To include these screens requires much of the up-front programming and translations to be completed, again placing undue pressure on already compressed study start-up timelines. 

Actual devices for review by the EC. If well-described summaries of the descriptions of the devices are provided, this should be adequate.

With recent Patient-Focused Drug Development deliberations and the passing of the 21stCentury Cures Act

 in the US, there has been more focus on risk-based, patient-centric drug development to help patients benefit from newly developed drugs sooner. Creating documentation that may not protect clinical trial subjects, is time-consuming and costly to produce, and that ultimately does not benefit subjects should be discouraged. After review of applicable regulations and guidance documents, along with many discussions among stakeholders across the clinical development process, we believe our proposal for a simplified way to look at EC submissions for eCOA documentation reflects best practices. For pharmaceutical sponsors, our proposal would reduce the complexity, costs, and timelines associated with such submissions without increasing potential risk or compromising subject protection. For eCOA system providers, it reduces unnecessary and time-consuming paperwork, and alleviates the timeline pressure to generate screenshots before the electronic implementation of the study is completed. For ECs, our proposal would reduce paperwork submissions for review that are not essential and may not even be reviewed. Ultimately, our justification and recommendations stand to support subjects through reducing timelines and costs of drug development. If we unnecessarily delay innovative medicines, we do a disservice to patients-in-need. Limitations to this proposal include instances of regional/cultural differences or therapeutic areas that may require additional documentation.

Based on our review of the guidances that govern submissions to ECs, we find no evidence that there is a specific requirement to submit for review the actual devices that will be used during the trial; screenshots of electronically implemented questionnaires as they appear on the devices; administrative screens that may appear before or during the entry of eCOA data, nor corresponding translations. Further, we believe requests for these screenshots would be an expansive interpretation of the ICH guidance. Such requests lead to unnecessary delays in the drug development process, as well as increased costs for sponsors and, ultimately, consumers. They do not provide additional human subject protection for clinical trial participants. As ICH GCP guidelines are reviewed, clarification on this matter would provide further support for simpler and more streamlined EC submissions of eCOA questionnaires to be used in a clinical trial.

, MBA, MAis the Executive Director of patient centered at

, DrPH, JD, LLM, is the Public Health & Outcomes Project Director at the

International Drug Development Institute.

MA is Associate Director of Critical Path Institute’s Patient-Reported Outcome Consortium and Acting Director of the Electronic Patient-Reported Outcome Consortium.

Views expressed in this manuscript do not necessarily reflect the official policies of the Department of Health and Human Services nor does any mention of trade names, commercial practices, or organizations imply endorsement by the United States Government. Support for the Electronic Patient-Reported Outcome (ePRO) Consortium comes from membership fees paid by members of the 

Bhatt A. International Council for Harmonisation E6 (R2) addendum: Challenges of implementation. Perspectives in Clinical Research. 2017; 8:162.

International Council for Harmonization (ICH) Steering Committee. Final Concept Paper Addendum for ICH E6: Guideline for Good Clinical Practice. 2014. Available at 

https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2_Concept_Paper_July_2014.pdf. Accessed 8/7/19

Commission of the European Communities. Commission Directive 2005/28/EC of 8 April 2005 laying down principles and detailed guidelines for good clinical practice as regards investigational medicinal products for human use, as well as the requirements for authorization of the manufacturing or importation of such products. Official Journal of the European Union. 2005;91:13-9.

U. S. Food and Drug Administration. Office of Good Clinical Practice. (1998). Recruiting Study Subjects Guidance for Institutional Review Boards and Clinical Investigators. Washington, DC.

Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force Report. Value in Health. 2009; 12:419–29

Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient‐reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value in health. 2005; 8:94-104.

Wild D, Eremenco S, Mear I, et al. Multinational trials—recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR patient-reported outcomes translation and linguistic validation good research practices task force report. Value in Health. 2009; 12:430-40.

Raymond S, Gertel A, Barrasso B, Hackett A. Optimizing Electronic Clinical Outcome Assessment Materials Required for IRB/EC Review. Medical Research Law & Policy Report. 2016; 15 MRLR 159.

AFSP1635538A, Order of December 2, 2016 setting the content, the format and the modalities of presentation of the file of request of opinion to the committee of protection of the people on a research project mentioned in 1 ° of the article L. 1121-1 of the code of Public Health Concerning a Drug for Human Use. Available at 

https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000033545400&categorieLien=id

21stCentury Cures Act. United States PUBLIC LAW 114–255—DEC. 13, 2016. 

https://www.congress.gov/114/plaws/publ255/PLAW-114publ255.pdf

Articles

Examining how to optimizing the time and effort of Ethics Committees to efficiently and effectively fulfill their human subjects protection remit. 

Demystifying Submissions of eCOA Documentation for Ethics Review: Are We Making Submissions More Difficult than Necessary?

Life is defined by our achievements and failures. Making goals and striving to reach them are how we categorize our day-to-day lives. This is true in our careers, our finances, our relationships, and also our health. Particularly for those living with a chronic condition, health goals must be agreed, set, and rigorously maintained. In some cases, the goals are defined by clinical parameters; for example, in diabetes, where control of blood glucose to within-recommended targets is the goal for most people. Achieving this goal helps to reduce the likelihood of micro- and macrovascular complications. In other cases, the goals are defined by our behavior; a month of no alcohol consumption among people taking blood-thinning therapy, for example. In line with the principles of behavioral and developmental psychology, it is important to reward successful outcomes where goals have been met. These rewards can be intrinsic or external. However, if we treat the outcomes in a simple binary fashion, where they are met or not, we miss important and rich information on those goals. 

	In order to better understand goal attainment, data is commonly collected using continuous interval data, so as a researcher can measure the achievements of an individual in terms of their status change (and the magnitude thereof), rather than simply whether a pre-defined goal was achieved. This measurement strategy is the cornerstone of in-vivo clinical research for the development of pharmacotherapy, and is the basis on which pharmacotherapeutic medications are developed, licenced, regulated, and made available within the healthcare system. Specifically, pivotal trials aim to answer two questions: (1) “did the experimental therapy achieve clinical benefit in patients who enrolled in the study”, and if so; (2) “is the benefit greater with the experimental therapy than with standard of care?”

	The answer to this question can, at its simplest, be achieved by the analysis of data at two timepoints; baseline (prior to initiation of therapy) and one additional timepoint, some days, weeks or months later, when the pharmacokinetics and

 pharmacodynamics of the drug indicate it should have reached peak efficacy. In reality, longer-term trials are usually carried out to ensure maintenance of an effect over time, with data collection visits occurring at regular intervals (normally one to six months apart). In Figure 1 at right, a graph shows the effect of a hypothetical experimental medication (medication A) compared to two active comparator medications (medications B and C) on physical functioning in rheumatoid arthritis. Assessment of physical functioning is made using a well-established patient-reported outcome (PRO) questionnaire, the SF-36.

	Items on this questionnaire are measured and scored using interval data, and lower scores are indicative of less impairment. In this trial, all three medications are efficacious, in that they have improved physical functioning over the three months of the study. They have done so to a “similar” (non-inferior) degree. In response to the two questions above, one must conclude based on the data presented in Figure 1, that (1) yes, there was a significant improvement in physical functioning from baseline to trial endpoint (3 months) with the experimental medication, but (2) no, the benefit was not greater than with standard of care (using traditional inferential statistical analysis).

	However, the apparent similarities in the impact of these medications on physical functioning may be a function of the simplicity of the study design; more specifically, 

the infrequent measurement of a concept which may change frequently may be masking important differences between the medications. Imagine that, in fact, the medications behaved as per Figure 2 at left, captured in the same trial using twice-weekly diary data rather than just once at the end of the trial. This shows clear differences between the medications. Although our answer to the first question (“did the experimental therapy achieve clinical benefit in patients who enrolled in the study?”) remains yes, the answer to the second question (“is the benefit greater than with standard of care?”) is now not so clear. Utilizing area-under-the-curve analyses (in addition to change-from-baseline analyses) would suggest medication A differs significantly to medication B, but not to medication C. However, the data now provides a basis for hypothesizing that these three drugs are fundamentally different. The differences are in the patient experience.

		A person who is cautious of doing too much, gets anxious about changes, but desires increased physical functioning would benefit most from medication A.

		A person who was physically active prior to the rapid development of rheumatoid arthritis will want to achieve maximal therapeutic benefit quickly so as to return as much physical functioning as possible as early as possible. For this person, medication B is the right choice. 

		A person who needs to engage in peak physical performance but is able to rest for some days afterwards would benefit from medication C (assuming that the peaks and troughs are predictable).

	It is too simplistic to suggest that more data points are more informative. Rather, the frequency of data capture must relate to the dependent variable (outcome) of interest, and be captured in a manner that reduces measurement error. For example, daily collection of “quality-of-life” data may decrease the variability in the measurement owing to participant burden and generalizations by participants when completing the scales. This would reduce the sensitivity of the assessment to identify a meaningful change from baseline within and between cohorts.

	On the other hand, researchers have shown reduced error variance by collecting episodic data in trials of overactive bladder, largely as a function of reduced recall bias and heterogeneity within participants over the course of a trial. The timing of assessment is also key; while random momentary assessment may be appropriate and beneficial for capturing “in-the-moment” fatigue in multiple sclerosis (which is known to follow a diurnal pattern), it is appropriate to standardize collection of pain and stiffness data once during a 24-hour period among rheumatoid arthritis patients. 

	Data collected on an appropriate assessment schedule gives power not only to answer the two questions regularly posed in pivotal trials, but also to explore individual responders to therapy. Figure 3 below provides population-level data from a trial of a hypothetical medication for insomnia.

	Although there is no comparator in this trial, we are able to conclude that there was a significant improvement in sleep time from baseline to trial endpoint (36 weeks) with 

the medication. Figure 4 at left shows data from four participants in the trial with very different experiences. The frequency of data capture allows us to define these as responders (for whom the medication worked) and non-responders. Participant 1, 2, and 4 could all be classified as responders, but for different reasons: 

		When participant 1 sleeps, (s)he sleeps a similar number of minutes/hours per night following initiation of medication as before medication. However, (s)he sleeps more frequently (every night) following initiation, opposed to before initiation (57% of nights).

		Participant 2 sleeps longer since initiation of medication, and has less variability in total time spent sleeping per night.

		Participant 4 has been afforded a consistent addition of one hour sleep per night since initiation of the medication.

		These individual differences in medication action and patient experience do not show on a plot of the average effect (See Figure 3).  

	The temptation in clinical research is to collect some data on a more frequent basis than others. A good example of this is treatment satisfaction, an endpoint utilized in clinical trials as a partial proxy for adherence (anticipated medication-taking behavior outside of the trial environment), but collected less frequently (in general) than clinical data. Satisfaction is a latent variable likely to be a function of expectations and experience of efficacy (positive clinical benefit) and adverse effects of medication (negative side-effects). Some combination of these will contribute to a person’s stated satisfaction with 

treatment, which can change day-to-day, in line with the perceived benefits and detriments of treatment on that day. Regular collection of all three types of data (efficacy, treatment satisfaction, side-effects) allows the researchers to overlay the observed data and understand the main contributing factors to whether high levels of satisfaction are achieved within-and between-patients. 

	As the medical community moves toward individualized and tailored therapeutics, and pharmaceutical companies are increasingly opting into risk-sharing agreements with payers, it is important to understand the patient journey with a medication, as well as the goal achievement of that medication. Collecting this data in pharmacotherapy development programs allows an understanding of the true impact of the medication in that context. This allows for more targeted prescribing, but also for appropriate expectation setting by treating physicians in clinical practice; if the patient journey does not match expectations, then adherence is likely to be poor. In addition to the use of frequent data collection to understand and prescribe medication optimally, there may be other hypotheses that can be generated from the data which can prompt future research for all stakeholders. For example:

 Is there any long-term (>1 year) clinical benefit to medication C over medication A and B (See Figure 2)? The rapid achievement of a goal and maintenance of that goal may predict future outcomes more than simply achieving the goal at three months. Key audience: regulators, payers, healthcare professionals.

 The “positive feedback loop” is a very important psychological construct in medication taking. The best predictor of positive outcomes is good response and, therefore, one could hypothesize that if patients see good outcome early (like with medication B in Figure 2), they are more likely to put up with side-effects and adhere (and, therefore, persist) better. However, what about medications A and C? How do they compare on treatment satisfaction and adherence variables? Key audience: payers, healthcare professionals, patients.

 Can a timepoint be established on Figure 2 at which an “unlikely responder” can be identified with reasonable specificity and sensitivity? Key audience: payers, healthcare professionals, patients.

	Answering these questions, and other clinically relevant questions, requires routine and regular collection of outcome data according to a schedule which allows for maximum understanding of the pharmacotherapy under investigation. In an era of “real-world evidence”, this data can support healthcare tools to enhance medication-taking behavior and clinical communication at appropriate and relevant timepoints, potentially increasing both the quality of life of patients and quality of care they receive.

, M.Sc., was Sr. Scientific Consultant, ERT, at time of research; 

, PhD, is Chief Scientific & Quality Officer, ERT

Measuring clinically relevant response requires routine and regular collection of outcomes data.

Therapeutic Success or Failure: A Journey, Not Just a Destination

Although many constituencies want clinical trials to be accomplished more quickly, the combined actions of three parties determine how well and how quickly clinical trials can be accomplishedsponsors, regulatory agencies, and technology providers. FDA did its part to facilitate development and adoption of technology several years ago. With participation from both sponsors and technology vendors, FDA personnel developed regulations that were approved and issued on 21 March 1997. These were designed to facilitate the use of computer systems and data processing technology by both sponsors and the FDA. 

FDAs intent is stated in the preamble to 21 CFR 11 (The Rule on Electronic Records and Signatures) as follows: 
These regulations, which apply to all FDA program areas, are intended to permit the widest possible use of electronic technology.[21 CFR 11 Summary] 

The purpose of this rule is to permit the use of a technology that was not contemplated when most existing FDA regulations were written.[21 CFR 11  XVI A, Objectives] 

Subsequent public statements by the authors of this rule emphasized that these regulations were intended to be flexible and facilitate the continued and further development of new technology. 

FDAs action was a boon to providers of electronic systems for data capture, correction, review, and archiving in clinical trials intended for submission to the agency. These providers now had a covenant: if their system complied with 21 CFR 11, then the FDA would accept electronic records and electronic signatures in place of paper records, and sponsors could be confident enough to execute their important clinical trials with electronic systems. So far, so good.

Technology providers interpret regulations


When developing an electronic clinical trial system, a technology provider must interpret regulations in detail. Its part of the job. Otherwise FDA or other regulators would have to invent and specify everything, and technology providers would simply build to order. 

When it comes to converting regulatory provisions to system specifications, programmers do not have one luxury that lawyers have. Lawyers, like politicians, write for people and can thus presume, rightly or wrongly, a degree of common sense. Programmers write for machines, and their regulatory interpretations tend to reach a deep level of detail. They cannot ask FDA to certify each judgment they make, nor could the agency respond to hundreds of such questions. Therefore, system developers must somehow address the needs of sponsors, regulators, and site personnel who will use the system to conduct many different types of clinical studies ranging in complexity and size from single site/single patient trials to trials with several thousand patients and hundreds of sites. The system has to comply with FDA (and international) regulations. This is actually fairly challenging. Any semantic or functional contradictions in the regulations and guidance must be resolved. And the specificity of a particular solution will amplify each element of the regulations in the sense of expanding the depth of detail while it will simultaneously limit the generality of the regulatory requirement in the sense of arriving at a specific realized implementation. 

Immersed in the various urgencies of system development, product sales, trial execution, and competition, a system designer at a particular provider may find it difficult to remember or imagine that various implementations might meet a general regulatory objective. This very difficulty shows the value of FDAs statements concerning its desire to preserve flexibility and thus fertilize invention, novelty, and ultimately the optimization of clinical research. At any point in time it may well be that the combination of individual requirements that must simultaneously be met will substantially narrow the set of acceptable technical implementations. Multiple objectives in conjunction with the state of the art may imply only one solution, or even, dreadfully, none. The technical challenge is akin to the many-body problem in physics, and this multi-problem problem for technology providers in clinical research can, by its sheer size and difficulty, lead to a kind of provincialism. A particular solution may appear to be the only solution.

The providers thus become regulatory experts of a certain sort. Because of their detailed immersion in regulations, their viewpoints can help clarify and extend regulatory understanding. However, as described above, structural attributes lead providers to include details or language in their extended interpretations that are not explicitly provided for in the regulations themselves (amplification and/or limitation). 

Centralized storage of data was questioned


We want to comment on a particular example of such an interpretation and try to draw some general lessons. In his May 2001 Technology Viewpoint column (e-source: To e or not to e, Applied Clinical Trials, May 2001, p. 36), Paul Bleicher suggested that data maintained at a central location, with no local archive at the investigator site, would not comply with FDA requirements for electronic source (e-source) documents. This idea has also been presented at various conferences. In our opinion, his views interpret and reword (amplify and limit) regulations to arrive at a narrow view that is neither conservative nor correct. Unless countered, this kind of interpretation may delay or prevent adoption of technology for e-source in clinical trials, which would be unfortunate and ironic given the original intent stated by the regulators themselves. 

We wish to present a view in counterpoint that the integrity of data stored centrally can fulfill what regulations actually say concerning the obligations of the site and e-source. We think that centralized systems using digital and handwritten signatures linked to e-source documents lessen the burden for sites, and move the industry toward a transforming acceleration of research much greater than the incremental improvement that has come from using the Web for electronic case report forms (eCRFs) transcribed from paper source documents.


A primary obstacle to the adoption of e-source, according to Dr. Bleichers column, was the requirement that a site maintain archival control of data. He also wrote: If the electronic CRF is the primary source document, the investigator must maintain control of that CRF to ensure attributability. He suggested a possible technical approach: To establish control, the investigator can create the electronic record on a computer that is physically located at his or her site or institution. Then he amplified and limited the regulations by writing: In any case, the electronic record cannot be stored with the sponsor or the sponsors representativesuch as a CRO or technology vendorbecause attributability would be lost.

The article went on to note that Web-based data entry was difficult to reconcile with primary source data entry and to advise that with central-server systems it was always advisable to have separate source documents and that a site without physical ownership and control of its database does not meet the GCP requirement for investigator control of the data...

Similarly for handheld devices: If it is primary source data the site must maintain an archival copymaking it necessary to archive the memory card or a certified copy at the site.

And for interactive voice response (IVR) systems: No obvious workaround is available to secure investigator control of IVR generated data. 

Ultimately the article constituted an interpretation of regulations and guidance that seemed to stipulate reliance on local storage systems and to restrict compliant capture of e-source to offline RDE with a consistent method of local storage. For diary data, which is a familiar form of e-source, the article amplifies regulations to require physical storage devices at the location of a site so that such data can be kept for archival purposes. While the article acknowledges that Web-based direct entry of e-source into eCRFs under digital signatures could address attributability, such an approach is faulted on the grounds that the sponsor could delete the electronic record and signature. The point of local storage would be to support manual authentication processes including field-by-field source document verification in the same manner now done with paper to prove that centralized data agrees with data at the site. This would continue the most expensive and burdensome chore required to generate integrity in paper systems. 


In support of an alternative view, consider which regulations bear on this issue and what they actually say. (See the accompanying sidebar.)

Taken together, the regulations and guidance bearing on electronic records and e-source in clinical trials concern data integrity. Both the Rule on Electronic Records and Signatures and Guidance for Industry: Computer Systems Used in Clinical Trials (CSUCT) emphasize the agencys intent to set forth the steps needed for computer systems to ensure that the pedigree of the data with electronic records would be as trustworthy as with paper records. The predicate rules spell out that data and record integrity are important, and that case history data, including source data and CRFs, are to be prepared, maintained, and retained by the investigator. 

A system compliant with 21 CFR Part 11 that is designed to provide the sites with ready access and the capability to enter, edit, review, authenticate, and maintain all of their electronic records for a clinical trial might well meet the regulatory requirements and guidance currently issued. To fulfill the criteria for data quality mentioned in the Guidance, such a system should ensure that electronic records are attributable, legible, contemporaneous, original, and accurate (ALCOA) and that they are protected in such a way as to enable their accurate and ready retrieval throughout the record retention period.

In addition to what the regulations do say, consider what they dont say. Many of the terms used in Dr. Bleichers column, including the phrases controlled certified copy, investigator controlled data, physical ownership and control, local control, local copy, local archival control, and archival control of data do not appear anywhere in the regulations. Only the terms prepare, maintain, and retain appear in the regulations.

Control without local storage is possible


Sponsors executing clinical trials with electronic data collection (EDC) technology providers regularly stipulate that the EDC systems used by the sites specifically provide controlled access by the site to its records. They set forth in their requirements specifications exactly who shall have the capability to enter electronic records, edit and review them, and make changes in the data. In preparing a particular trial, the EDC system applications are validated to ensure that only such designated personnel (usually site personnel and investigators who have signed the 1572) can control (prepare, maintain) the data during the trial. Physical access to the devices is not a necessary component of maintaining the electronic record. Instead, access to the information is needed. If an investigator has all of the components of an electronic record in a secure form on his or her desktop at will, then the case for data integrity is good and one can legitimately say that the site has maintained and retained that record.

Precedents with paper records that fulfill the records retention requirement include the storage of the records in a central facility (on or off-site), and the use of contractors, such as Iron Mountain, to store records in secure warehouses. Laboratory data is often held offsite in computerized systems that are under contract to the sponsor. Regulations indicate that the site should have ready access to such data, and specifically that FDA inspectors should have access to such data for purposes of review at the site. But nowhere does it appear that the only satisfactory solution to meeting such requirements is for the site to physically possess local hardware on which its electronic records are stored.

Paper and electronic records are different


It is useful to consider how different e-records are from paper records. Our habits of thought concerning physical possession of paper, which confers access to information on the paper, should not obscure the much weaker link between physical possession of digital storage media and access to that data. A look at a paper CRF page reveals the logically related fields, the data in the fields, handwritten commentary and marginalia, and any signatures. By physically preventing people from looking at the paper, one can do a pretty good job of protecting the information. However, a look at a hard drive is not the key to access. By cryptographically linking data fields, audit trails, and electronic or handwritten signatures on electronic records, access to electronic records can be controlled by software running in the devices needed to interpret the altered bits on the storage media. Electronic records can be accessed remotely as quickly as they can locally, and possibly more securely given the potential exposure of computer systems at local sites to environmental hazards, maladroit IT management, and cyber-attack. What matters is the full story on data integrity and security, not physical possession.


In a nutshell, the notions concerning local copies emerge from a reasonable desire to ensure that data captured at a site and sent to a sponsor or sponsor representative is not subsequently altered by the sponsor. This is the point of source document verification and database verification. It is why an FDA inspector during an audit must be able to compare the values in a locked database against some local accessible values maintained by the site and presumed to have an unimpeachable pedigree. The process of holding a local copy of source data (and a local copy of CRFs sent to sponsors) all makes sense for paper, where the physical properties of paper are relied on to hold the data on certain fields togetherand where handwritten signatures can be linked to data fields and auditable changes to data only by the physical properties of the paper. 

However, the regulations simply state that investigators shall prepare and maintain case histories (source data and CRFs) during a trial and retain them afterwards for a specified period. Various requirements are intended to ensure data security and integrity for electronic records, but none of them specifically mandates a controlled certified copy. The records of sessions that FDA has with vendors carry a notice that they reflect only the views of individuals present at the meetings and not of FDA. So one looks primarily to the regulations and the guidance. These do constitute what FDA in the full force of its deliberations wants. Such wants include technical flexibility and best solutions. The Rule on Electronic Records and Signatures and CSUCT are specific about the objectives for data integrity, system controls, authority controls, and data quality. 

Below are some of the features and approaches that can be combined to make the case for integrity and security of electronic source and compliance with regulations without using local storage and without requiring field-by-field source document verification. The example is that of handhelds that send diary data directly to a centralized Web server. During the trial the site can access the data for review and approval (maintenance) via the Web:

1.Data is attributable because it is captured on devices that are assigned to subjects and electronically registered to a specific trial.

2.Data can be authorized and confirmed to be from a subject by using handwritten signatures executed on the handhelds for each report and linked cryptographically to that report in a centralized store.

3.Such tamper-proof electronic records can be protected from deletion at the central server by technical security and processes that require the willing collaboration of at least two individuals. The effectiveness of such protection can be validated.

4.Sites prepare diaries as part of their case histories by teaching the subjects how to use the handhelds and by making sure the subjects comply with schedules and constraints for completing and transmitting diary records. 

5. Sites maintain diary data by reviewing it using secure Web sessions and by virtue of the constraints built into the system that prevent anyone but duly trained and authorized site staff from entering or editing subject data. Such constraints are part of the requirements specifications and are confirmed by system validation.

6. System validation also takes the place of field-by-field source document verification, since data entered on the handheld is validated to be accurately and completely transmitted and parsed into the datastore and/or database.

7. Sites retain diary data after the trial by receiving a certified CD-ROM or other nonvolatile record that includes all the data, metadata, and trial documentation necessary for an audit of the trial at the site. The CD-ROM can handle confidential data (with password protection), support computer generated audit trails, and permit reconstruction of the trial as it occurred at each site.

8. Such diary data in tandem with electronic source data signed with digital signatures (or handwritten signatures attached to electronic records) by site personnel can provide a nearly paperless system that exceeds the quality of paper data and is also more convenient to use.


Last Mays column mentioned two disadvantages of e-source that we question. The first disadvantage was that electronic capture complicates the interaction between a doctor and a subject more than paper does. Therefore the clinical investigators wont use it. Certainly this is simply a technical matter. Whatever the method of capture is, it must in no way make it more complicated for the doctor or study coordinator to use than paper. This is not a conclusive disadvantage, just an opinion about technologies with which the author was familiar.

The second disadvantage mentioned was that e-source brings increased risk of data entry errors. This is an unproven assertion. It may be true that data typed into a keyboard or entered on a PDA is more prone to error than data written to paper, but the proof offered (the common error of misdialing a phone number) is not very compelling evidence. People can make errors of the hand: I think Im writing 180, but my hand writes 108. But with electronic capture, such entry errors may in fact be dramatically minimized using real-time confirmation and other accuracy-enhancing techniques, including numerical selection instead of writing numbers. The idea that anyone would automatically know whether 108 mm Hg or 180 mm Hg was correct during source document verification (SDV) is questionable. 

In many audits and reviews of audits done of clinical studies, one of the authors (GM) has fomed the strong view that electronic source data does not bring with it an increased risk of daa entry errors. It has been his experience that manually entered and transcribed data, lab values, CRF observations, and hospital records are all significantly more accurate when recorded electronically. Entry errors are not the unique province of e-source, and they certainly are not prevented by the use of paper source documents and SDV. The incidence of getting data values right from the start could probably be improved if data entry personnel were spared the additional burdens of double or triple transcription and could focus a bit more attention on the moment of original entry. In any case, this disadvantage is not established.

Certified copies at sites are not essential 


While a sites electronic records can certainly be stored locally to support e-source in a compliant way, the regulations do not specify that it is essential for hardware at a site to hold a certified copy. The original record can be centralized as long as the site retains access to it at will, and as long as the data remains protected and stable and editable only by the site in files of the site. 

One must note that FDAs good clinical practice regulations (GCPs) recognize that sponsors may need to make changes in clinical data originating from the site, and section 4.9.3 of the Guideline for Good Clinical Practice specifically sets forth that such changes are OK provided that any such changes are documented, are necessary, and are endorsed by the investigator. The GCP also states that the records pertaining to such changes as well as all the audit trail records be retained.  As above, this guidance appears to be met both in letter and spirit if such records can be displayed at will at the site and are protected and maintained so that their integrity is preserved. EDC systems compliant with 21 CFR Part 11 have been implemented so that that claim is met.

Trusted third parties may provide centralized storage


The idea that centralized servers maintained by persons other than site personnel can bring improved efficiency and reliability to clinical trials is probably correct. However, it is neither implied nor required by regulations that such trusted third parties be outside the pharmaceutical area or that they have contractual arrangements with the sites. As written in paragraph 41 of the preamble to 21 CFR Part 11: 
The agency does not believe it is necessary to codify the basis or criteria for authorizing system access, such as existence of a fiduciary responsibility or contractual relationship. By being silent on such criteria, the rule affords maximum flexibility to organizations by permitting them to determine those criteria for themselves.

Centralization is one of many possibilities


We do not believe centralized systems are the only acceptable method. We do believe that the objectives of the regulations can be achieved through such site client systems and that they have some advantages. 

We believe that data integrity is best preserved when the trial data is centralized, protected, backed up, and restorable by a fully professional staff and by validated procedures. Some systems centralize the data onto servers that are physically protected (under lock and key) at network operating centers. The centers ensure reserve power (in the event of main power failure from city utilities) and proper, fireproof, and air-conditioned places for computer and telecom equipment needed for each study. Providers then ensure in a validated fashion that the sites can prepare, maintain, and retain the data on clinical subjects, and that they can also retain access to laboratory data and other clinical data streams from other collection paths. This ready access and review, provided electronically, corresponds to the review enabled by physical possession of paper records. The review capacity and familiarity with the data would not necessarily be enhanced, however, by the sites having physical possession of the storage devices on which the altered bits representing the electronic record repose. Although technology providers are paid by sponsors, the conditions of payment can specify that they build, validate and provide EDC tools for studies that ensure data content is under control of the site. They can and often do act as third parties at the request of the sponsor to provide the secure tools for the sites to use to prepare, maintain and retain their clinical research data. This strategy brings the benefits (central administration, rapid implementation of validated mid-study corrections, restoration capability, and the existence of a single canonical electronic record in one place) of centralized data storage, but preserves the key regulatory aspects that support approval and execution of actions on data as a responsibility of the investigator.


The technology for supporting data capture and management in its full range of complexity may have arrived. Its transforming benefits will emerge with fuller use of e-source and direct data capture, and we have presented a case that the approach of centralizing such data without local storage should not be dismissed presumptively.

While we have criticized Dr. Bleichers article about e-source for amplifying regulatory barriers, we think that commentary from technology providers about regulations, including that article, has been and will be useful. We have mentioned the structural constraints that shape such commentary (including ours) and readily reserve to regulatory agencies the last and most authoritative word. 

During the past year, it has been reassuring to see sponsors rise to the required level of courage to assess the case for integrity of centralized data in spite of comments that fan the flames of concern. Such action on the part of FDA, sponsors, and technology providers seems to be the only way to move ahead.            

The authors contrast two views on the regulations concerning electronic source data.

Stephen A. Raymond

Articles

Demystifying Submissions of eCOA Documentation for Ethics Review: Are We Making Submissions More Difficult than Necessary?

Therapeutic Success or Failure: A Journey, Not Just a Destination

Interpretation of Regulatory Requirements by Technology Providers: The Case for Electronic Source Data