Critical Path Institute’s Electronic Clinical Outcome Assessment (eCOA) Consortium

On May 26, 2021, the EU began fully implementing the Medical Device Regulation (EU) 2017/745 (MDR).

 This regulation was developed to ensure medical devices are marketed consistently across the EU member states, ensure minimum quality and safety standards for patients and users, and ensure that data generated from such devices in clinical investigations are reliable and robust.

Members of Critical Path Institute’s Electronic Clinical Outcome Assessment (eCOA) Consortium have been required by their clients to go through the process of obtaining CE (“Conformité Européene”) certification for eCOA applications when they do not in fact fall in scope of MDR. By requiring this certification, which would result in the CE mark being applied to the app, clients are equating the function of the eCOA application with being a medical device, or more specifically Software as a Medical Device (SaMD). If the regulation is examined in depth, it becomes evident that while there may be boundary cases where this may indeed apply to an eCOA app, it is clearly exceptional rather than typical.

To date, there is no known published article or publicly accessible paper that examines the regulations in the context of eCOA systems that would support an informed discussion between sponsors and eCOA system providers. To address this, the eCOA Consortium formed a team in early 2022 tasked with performing an in-depth analysis of the regulations generated by the EU as well as international collaborations such as the International Medical Device Regulators Forum (IMDRF). The analysis resulted in the generation of 

, a self-published white paper that offers guidance to the medical product development community for when eCOA systems may or may not fall in scope for CE certification. The paper also outlines at a high level the steps required to attain CE certification.

The concept of intended use is at the heart of MDR. The European Medicines Agency (EMA) regulations place the burden of proof upon “manufacturers.” In the context of eCOA systems, the manufacturer would be the software developer (eCOA provider) that generates a digitized form of a licensable and/or publicly available COA when contracted to do so. MDR, Article 2, defines specific purposes for which a system may be considered to be a medical device. Of those specific purposes, those that an eCOA app could be considered to support are “diagnosis, prevention, monitoring, prediction, prognosis, treatment, or alleviation of disease.”

 Typical eCOA systems may be reasonably considered to monitor disease, or more accurately the health status of a trial participant. This is certainly a consideration for sensor-based wearable technologies that are becoming more common in clinical trials, but those devices would themselves be required to attain CE certification.

It is important to understand that the intent of an eCOA system is generally to enable recording of responses to questions that may be part of a standardized assessment and for those responses to be transmitted to a clinical database as source data. In general, eCOAs are not implemented for medical purposes in clinical trials. Sites do not implement medical treatments based solely on these data; the data inform a statistically driven assessment of the effect of a given study intervention or interventions. Commonly, eCOA software does not implement medical treatments based on the data generated either, if it did, it would then be classified as SaMD and require the CE mark.

The regulation does leave room for interpretation, and as such creates the opportunity for misclassification of an eCOA system. To support that assessment, the eCOA Consortium’s white paper incorporates a decision tree that may serve as a guide to support the assessment of whether an eCOA app could require CE certification.

The paper is available for download from the eCOA Consortium’s webpage: 

https://c-path.org/wp-content/uploads/2023/03/eCOA-Systems-and-CE-Certification.pdf

Authored on behalf of Critical Path Institute’s eCOA Consortium by Scottie Kern, Executive Director, eCOA Consortium; Ian Pallett, AD Sensor Solutions, Parexel; and Rinah Yamamoto, Principal Scientist, Clinical Ink

 Legislative Acts: Regulation (EU) 2017/745 of the European Parliament and of the Council 

https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32017R0745

MDCG 2021-24: Guidance on Classification of Medical Devices 

https://health.ec.europa.eu/system/files/2021-10/mdcg_2021-24_en_0.pdf

Articles

EU Medical Device Regulation must be followed carefully as it leaves room for interpretation.

eCOA Systems and the CE Mark: Navigating the Regulations

The well-recognized potential of sensor-based digital health technologies (DHTs) as drug development tools has not led to their widespread adoption by drug developers as originally anticipated.

 Part of the problem is the lack of consensus on regulatory terminology and the corresponding regulatory pathways that would enable the effective leveraging of these technologies in medical product development.

FDA defines a DHT as “a system that uses computing platforms, connectivity, software, and/or sensors for healthcare and related uses. These technologies span a wide range of uses, from applications in general wellness to applications as a medical device. They include technologies intended for use as a medical product, in a medical product, or as an adjunct to other medical products (devices, drugs, and biologics). They may also be used to develop or study medical products.”

Although the European Medicines Agency (EMA) does not provide a precise definition of DHT, its description of the digital technologies covered by its remit is virtually the same.

 Hence, the dizzyingly broad definition of DHTs encompasses a vast array of tools and applications, while the uses of the various DHTs in clinical investigations come with different implications and require different regulatory considerations.

The use of body-worn sensors, combined with data processing algorithms in drug development, is particularly challenging. The data generated can be a biomarker or a clinical outcome assessment (COA), but the distinction between the two is, at times, blurred by conflicting definitions. One area of confusion is when these sensors (e.g., accelerometers) are used to collect data on the wearer’s real-world behavior or activity. They can be COAs, but the passive nature of these measurements does not fit within the four types of COAs defined by FDA.

 These have been called “DHT-passive monitoring COAs” within FDA’s COA qualification program

 but remained undefined in FDA’s recent draft guidance titled “Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments.”

As mentioned, wearable sensor data can also be a biomarker as defined by FDA5 and the EMA.

 It is important to note, however, the definitions from these agencies are not entirely congruent. While FDA clearly defines biomarker categories and characteristics as well as the distinctions from COAs, EMA provides a specific definition for digital biomarkers that, among other things, blurs the distinction between COAs and biomarkers. In addition, the emergence of the term "digital biomarker" has led to further conflation of the definitions of biomarkers and COAs in the scientific literature.

Why do these definitions matter? If there is no consensus on the definitions of and distinctions between COAs and biomarkers in the context of sensor-based DHTs, tribal language within scientific communities emerges and produces inefficiencies. Since drug development is multi-national and multi-jurisdictional, common terminology is essential but unattained. To further illustrate this point, the Mobilise-D project funded by the Innovative Medicines Initiative in Europe uses the following definition in relation to wearable sensor-derived measures of mobility: “A mobility outcome is a biomarker that provides a quantification of each patient’s mobility.”

 This definition looks odd to those of us who would consider mobility a COA concept based on FDA’s biomarker and COA definitions, but it highlights the fact that whether a measure derived from wearable sensors is considered a biomarker or a COA can depend on the jurisdiction. That is a problem.

Similar problems have existed and have been solved through collaboration. Critical Path Institute’s (C-Path’s) eCOA Consortium and Patient-Reported Outcome Consortium have a joint “Lexicon” project aimed at ensuring that eCOA system providers and clinical trial sponsors speak a common eCOA language. It’s time to expand beyond C-Path to engage other stakeholders, such as the European Federation of Pharmaceutical Industries and Associations (EFPIA), Foundation for the National Institutes of Health, FDA, and EMA, to reach consensus on conflicting DHT definitions and terminology in the regulatory context. To help fulfill the potential of DHTs in medical product development, the eCOA Consortium is seeking a productive path toward doing just that.

EFPIA is building on this and other sources of established terminology in an upcoming paper that calls for alignment on DHT-related concepts and terminology. C-Path looks forward to engaging with EFPIA, FDA, EMA, and other stakeholders to help reach the full potential of sensor-based DHTs in drug development through consensus on a common language that is essential for success.

Authored on behalf of Critical Path Institute’s eCOA Consortium by Elena S Izmailova, Chief Scientific Officer, Koneksa Health; Christine Guo, Chief Scientific Officer, ActiGraph; and Stephen Joel Coons, Senior Vice President, Clinical Outcome Assessment Program, Critical Path Institute

Izmailova, E.S.; Wagner, J.A.; Perakslis, E.D. Wearable Devices in Clinical Trials: Hype and Hypothesis. 

https://pubmed.ncbi.nlm.nih.gov/29205294/

Digital Health Technologies in Drug Development: Accelerating the Momentum in Europe. EFPIA, May 19, 2022, 

https://www.efpia.eu/news-events/the-efpia-view/blog-articles/digital-health-technologies-in-drug-development/

Digital Health Technologies for Remote Data Acquisition in Clinical Investigations: Guidance for Industry, Investigators, and Other Stakeholders (Draft)

https://www.fda.gov/media/155022/download

Questions and Answers: Qualification of Digital Technology-based Methodologies to Support Approval of Medicinal Products 

https://www.ema.europa.eu/en/documents/other/questions-answers-qualification-digital-technology-based-methodologies-support-approval-medicinal_en.pdf

(National Center for Biotechnology Information, December 2020). 

https://www.ncbi.nlm.nih.gov/books/NBK566059/

Clinical Outcome Assessments (COA) Qualification Program Submissions 

https://www.fda.gov/drugs/clinical-outcome-assessment-coa-qualification-program/clinical-outcome-assessments-coa-qualification-program-submissions

Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments (Draft) 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-selecting-developing-or-modifying-fit-purpose-clinical-outcome

Vasudevan, S.; Saha, A.; Tarver, M.E.; Patel, B. Digital Biomarkers: Convergence of Digital Health Technologies and Biomarkers. 

https://pubmed.ncbi.nlm.nih.gov/35338234/

Viceconti, M.; Hernandez Penna S.; Dartee, W.; et al. Toward a Regulatory Qualification of Real-World Mobility Performance Biomarkers in Parkinson’s Patients Using Digital Mobility Outcomes. 

https://www.mdpi.com/1424-8220/20/20/5920

Despite high potential of DHTs, regulatory picture remains unclear.

Conflicting Terminology in Digital Health Space: A Call for Consensus

The eCOA Consortium has served as co-sponsor of the Drug Information Association’s (DIA) Digital Technology in Clinical Trials Conference for the event’s first three years. All have been held virtually so far, delivering insightful and inspirational content that was deserving of a wider audience. This year, the central theme was how patients and sites are engaged in the context of the expanding use of digital health technologies in clinical development.

The committee tasked with developing the conference program aligned on the mindset of the content being progressive, yet honestly practical. Collectively, we aligned on two long-standing challenges—true patient engagement and the technology impact on clinical sites. Building on this, the conference explored the site perspective on digital technology, patient centricity in decentralized trials (DCTs), and regulatory implications for digital health technologies. Sessions also explored the use of AI and machine learning, and offered an intriguing exploration of how patient ownership of their own real-world data might impact clinical development.

Among the wealth of strong material, the final presentation, titled “We are All Patients”—and delivered by my eCOA Consortium colleague, Elena Izmailova, chief scientific officer at Koneksa—left a lasting impact. Izmailova, shared the story of her work utilizing digital technologies to track and support her recovery from cancer treatment.

Izmailova was diagnosed during the COVID-19 pandemic, which presented a series of challenges. Her knowledge drove her to construct an 

 technology kit that not only enabled an informed patient voice, but also delivered insights on her functional status to share with treating physicians in a remote setting. All post-surgery follow-up visits were virtual, and having digital technology-derived data allowed Izmailova to pressure-test sensor-based data collection as a DCT approach. She also used a non-structured patient diary to track symptoms of importance and relevance to her; a complaint among trial participants using electronic patient-reported outcome (ePRO) systems is repeated questioning on the same symptom across different copyrighted measures.

Noting a reduction in post-surgery step count, Izmailova was cognizant of how critical physical activity tracking is to patients, as described in the Ohri 

 that suggested step count may serve as a predictor of short-term hospitalization risk during chemoradiotherapy. Izmailova took ownership of tracking her activity and used the data to support consultations with the medical team. In discussing her elevated heart rate while walking, the need for an iron infusion was identified to address post-surgical anemia; having objective data to share with physicians was central to the decision to proceed with the infusion.

In essence, Izmailova created an individual trial, an N-of-1, defining her own assessment schedule based on known risks to patients with similar conditions. And there were broader learnings, such as the challenge of sharing data from multiple sources. Systems integration via portals has always focused on the site, but what about the patient? Izmailova’s N-of-1 study suggests the potential value of tracking vital signs and physical activity and provides an example of a methodology to assess data generalizability.1

Izmailova’s story highlights two key concepts; the value of technology and the critical need for meaningful patient engagement. The balance we must strike between delivering on working objectives and the realities of being a patient is a delicate one. We have talked often within the eCOA Consortium about how eCOA systems must adapt to the realities of managing serious disease. Talking the talk of true patient centricity doesn’t solve the realities of invasive methods and patient burden, but we have a duty to empower patients and trial participants with strategies and tools to help them manage their conditions and treatment.

Thankfully, Izmailova has fully recovered and continues to provide outstanding contributions to eCOA Consortium’s work. Beyond that,she reminds us that we are all patients and have a duty to think laterally and progressively to ensure patients are central to their care.

Izmailova, E; Ellis, D. “When Work Hits Home: The Cancer-Treatment Journey of a Clinical Scientist Driving Digital Medicine.” 

Ohri, N; Halmos, B; Bodner, WR; Cheng, A; Guha, C; Kalnicki, S; Garg, M. “Daily Step Counts: A New Prognostic Factor in Locally Advanced Non-Small Cell Lung Cancer?” 

International Journal of Radiation Oncology, Biology, Physics.

Final presentation at DIA reminds: "we are all patients."

First-hand Perspectives on Achieving True Digital Health Engagement

Because the numeric rating scale (NRS) and visual analog scale (VAS) are frequently used in clinical outcome assessments (COAs), these (along with the verbal rating scale, which is not addressed here) are commonly migrated to electronic devices for use by trial participants to collect endpoint data. Despite their prevalence and the existence of migration best practice recommendations,

 we frequently see suboptimal implementation of these response scale types that could compromise a measure’s psychometric properties and trial outcomes. This article is a primer on the implementation of the NRS and VAS on eCOA systems and includes tips for ensuring your eCOA provider can accommodate best practice recommendations.

An NRS consists of numbers representing an ordered continuum of some attribute. The NRS typically includes verbal descriptors/anchors at the ends. Respondents select the number that reflects their perceived state.

FDA’s patient-reported outcomes (PRO) guidance

 defines the VAS as a line of a fixed length (on paper, typically 100mm; eCOA uses 101 distinct response points, regardless of line length), with verbal anchors at the extreme ends, and no descriptors of the scale’s intermediate positions. Typically, no numbers are presented at the anchor positions. Respondents are asked to place a mark on the line corresponding to their perceived state.

While both NRS and VAS have been used in endpoint measures to support labeling claims,

 suggests superiority of NRS over VAS in pain measures due to ease of administration and scoring, higher acceptability, and psychometric property retention. Further, while FDA guidance suggested VRS, VAS, and NRS are all suitable to measure pain,

 states the NRS is the most appropriate measure of pain intensity. In fact, FDA’s Patient-Focused Drug Development draft guidance

 states that FDA does not recommend the use of a VAS.

 These regulatory recommendations, as well as concern about VAS comprehension, are reasons why novel instrument developers may select other response scales over the VAS. Finally, the Critical Path Institute’s (C-Path’s) PRO Consortium recommends avoiding the VAS when developing 

 PRO measures and encourages use of other response scales appropriate to the measure’s content and the patient population.

Regardless, because both NRS and VAS are frequently used in COAs and need to be implemented in eCOA solutions, it is imperative to follow best practice recommendations for faithful migration to retain a scale’s measurement properties. Figure 1 below shows examples of an NRS and VAS that align with best practice recommendations.

Can your eCOA provider follow best practice recommendations?

Although VAS and NRS are common, some eCOA providers struggle to implement them according to best practice recommendations and offer unacceptable suggestions and/or formatting (e.g., NRS response options wrapped over multiple lines), incorrectly insisting that VAS lines must be 100mm long despite extensive comparability evidence,

 and displaying numeric values associated with a VAS response. In part, eCOA providers should be assessed based on the provider’s study-design capabilities.

Sponsors are urged to choose providers who demonstrate their system’s ability to accommodate complex designs, conditional logic, a user-friendly interface, and faithfully migrated measures. Sponsors are cautioned to scrutinize the capabilities of providers with no proven experience in eCOA data collection in clinical trials who propose non-clinical, generic data capture systems such as commercial online survey tools to capture eCOA data. The functionality required to display scales according to best practice recommendations, let alone complex solution design or custom report generation, is often not available in those generic systems. Those basic limitations could lead to deviations from eCOA best practice recommendations and result in low-quality data that compromises trial outcomes.

Ensuring your eCOA provider is aware of and has experience in implementing eCOA best practice recommendations is pivotal to collecting data that appropriately measures endpoints and withstands regulatory scrutiny.

Authored on behalf of Critical Path Institute’s eCOA Consortium by 

, Vice President, eCOA Science, YPrime; and 

ePRO Consortium. Best Practices for Electronic Implementation of Response Scales for Patient-Reported Outcome Measures. Available at 

https://c-path.org/programs/ecoac/epro-overview/best-practice-documents

Electronic Patient-Reported Outcome Measures: An Implementation Handbook For Clinical Research

Mowlem, F.D.; Elash, C.A.; Dumais, K.M.; Haenel, E.; Olt, J.; Kalpadakis-Smith, A.V.; James, B.; O’Donohoe, P.; Balestrieri, G.; Becker, K.;. Newara, M.C.; Kern, S. on behalf of the Electronic Clinical Outcome Assessment (eCOA) Consortium. Best Practices for Paper to Electronic Migration of Patient-Reported Outcome Measures. Manuscript in preparation.

Haefeli, M.; Elfering, A. Pain assessment. 

 15, Suppl 1(Suppl 1): S17-24 (2006). doi: 10.1007/s00586-005-1044-x. Epub 2005 Dec 1. PMID: 16320034; PMCID: PMC3454549.

U.S. Food and Drug Administration. Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. 2009. Available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims

Gnanasakthy, A.; Mordin, M.; Clark, M. A Review of Patient-Reported Outcome Labels in the United States: 2006 to 2010. 

Gnanasakthy, A.; Mordin, M.; Evans E. A Review of Patient-Reported Outcome Labeling in the United States (2011–2015). 

Gnanasakthy, A.; Norcross, L.; DeMuro Romano, C.; Carson RT. A Review of Patient-Reported Outcome Labeling of FDA-Approved New Drugs (2016-2020): Counts, Categories, and Comprehensibility. 

 25(4), 647-655 (2022). doi: 10.1016/j.jval.2021.10.006. Epub 2021 Dec 24. PMID: 35365309.

Safikhani, S.; Gries, K.S.; Trudeau, J.J; Reasner, D.; Rudell, K.; Coons, S.J.; Bush, E.N.; Hanlon, J.; Abraham, L.; Vernon, M. Response scale selection in adult pain measures: Results from a literature review. 

https://doi.org/ 10.1186/s41687-018-0053-6

U.S. Food and Drug Administration. Guidance for Industry Analgesic Indications: Developing Drug and Biological Products. Draft Guidance. 2014. Available at 

https://www.fdanews.com/ext/resources/files/02/02-05-14-Analgesic.pdf

U.S. Food and Drug Administration. Development of Non-Opioid Analgesics for Acute Pain; Draft Guidance for Industry. 2022. Available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/development-non-opioid-analgesics-acute-pain-draft-guidance-industry

Celeste A. Elash. Personal communication.

U.S. Food and Drug Administration. Patient-Focused Drug Development: Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments. 2022. Available at 

Byrom, B.; Elash, C. A.; Eremenco, S.; Bodart, S.; Muehlhausen, W.; Platko, J. V.; Watson, C; Howry, C. Measurement Comparability of Electronic and Paper Administration of Visual Analogue Scales: A Review of Published Studies. 

Therapeutic Innovation & Regulatory Science

https://doi.org/10.1007/s43441-022-00376-2

Despite prevalent use in electronic devices, NRS and VAS are often implemented suboptimally. 

Implementing Numeric Rating and Visual Analog Scales in an eCOA Solution

The following is the second installment of a two-part series evaluating the use of backup solutions for electronic patient-report outcomes (ePRO) systems. Here, C-Path’s Electronic Clinical Outcomes Assessment (eCOA) Consortium presents results from a survey issued to 15 sponsors who are members of C-Path’s PRO Consortium and have a specific interest in eCOA systems. Twelve responses were received. The survey investigated backup solution adoption, justifications for solution selection, reasons why paper persists, and how backup data are evaluated.

General approach to backup solutions and justification for selection

When asked about their general approach to backup solutions:

4 sponsors include backup solution(s) in every trial.

2 sponsors include backup solution(s) for a specific subset of trials based on endpoint and/or indication.

2 sponsors provide backup solution(s) for only some trials but didn’t specify the criteria.

2 sponsors have internal guidelines for backup procedures.

1 sponsor does not have any internal guidelines and delegates the decision to the clinical teams.

The primary electronic backup solution of choice for those that employ them was a web backup. Paper remains prevalent and is often used in combination with a web backup.

7 respondents selected web as the primary back-up (but some commented they use multiple backup solutions within a trial).

Interestingly, web backups are not used in isolation in our sample. Multiple respondents indicated that web backup is not always accessible; limited connectivity or device availability could be factors, and even if accessible, the web-based solution may still be challenging for specific participant populations and/or assessments. In such cases, sponsors have used alternative methods such as bring your own device (BYOD), interviewer administration of assessments, replacement devices, and/or paper. Early discussions with clinical teams and sites to identify situations that necessitate a more flexible or tiered backup approach is advised (e.g., the combination of web and BYOD).

As indicated, paper backup is used by some sponsors as a primary backup solution. Based on the survey, paper backup is mostly used to support site-based ePRO collection. The principal reasons for use were:

Site activation prior to eCOA availability, specifically in localized languages.

Feasibility for specific therapeutic areas.

Most respondents agreed data captured via backup methods should be identified as such in the dataset. It was unclear from responses if sponsors uniformly conducted sensitivity analyses prior to pooling these data. Further discussion on backup data is necessary; it is important to know if sponsors see differences in the data and how they assess the relative level of quality between the data. It is evident from the survey that respondents consider it to be critical to reduce the amount of missing data; and employ a variety of backup solutions to ensure data completeness.

Although not generalizable beyond the survey respondents, the results indicate that there is no universal solution. Strategies may adapt to the trial’s endpoint hierarchy, therapeutic area, and accessibility, and there is some desire for multiple options within a trial. Part 1 of this series suggested that with more flexible eCOA solutions, we may reduce the need for backups, specifically paper. Following the survey, we continue to champion the concept of flexibility. Respondents also acknowledged that what is planned is not always executed within the study. Clinical teams need immediate access to options such as web-based backup, BYOD, or replacement devices in order to pivot as needed, but this demands continued evolution of the eCOA systems used, inclusive of innovation in science, technology, device management, and technical support.

Authored on behalf of Critical Path Institute’s eCOA Consortium by Lisa Charlton, Senior Director, eCOA, Science 37; Shelly Steele, Senior Scientific Advisor, eCOA, WCG Clinical Endpoint Solutions; Celeste Elash, Vice President, eCOA Science, YPrime; and Scottie Kern, Executive Director, eCOA Consortium

Survey results emphasize need for flexibility in strategies.

Survey Investigates Backup Solution Adoption for ePRO Systems

, paper backups to electronic patient-reported outcome (ePRO) systems are still requested and used. Web-based backup is often proposed as a practical solution with significant advantages over paper, yet given the frequency of requests for paper backup in lieu of or in addition to web-based backup, the web-based approach has evidently not sufficiently alleviated concerns. Here, in Part 1 of a two-part series, we offer a retrospective examination of the effectiveness, usability, and challenges with web-based electronic clinical outcome assessment (eCOA) backups. Part 2 will present results from a survey of sponsor organizations evaluating the frequency of adoption, the justifications for selecting specific backup solutions, reasons for the persistence of paper, and how backup data are evaluated. 

The benefits of web-based backup over paper backup for ePRO data collection are well established.

 Web-based backup not only aligns with regulatory guidance pertaining to ALCOA+ (attributable, legible, contemporaneous, original, accurate, complete, consistent, enduring, and available) principles

, but also incorporates the design benefits of electronic methods like logical branching, forced ordering, and timestamps. Given the prevalence of internet availability, it could be reasonably assumed that web backups present the ideal solution to acquire PRO data in the field that is of superior quality when compared to paper. Web-based backup is not without its challenges.

The solution does require (minimal) additional training for trial participants and sites and may require alternative login information, which is generally unappealing to users. In addition, there may be issues with browser compatibility with personal devices or access to suitable web-enabled devices for both trial participants and site staff. Depending on the eCOA provider, a web-based backup could require additional cost and add operational demands that may impact trial timelines. We hypothesize that these challenges are the principal reasons paper backup persists.

Web-based backup was introduced as one of many solutions to mitigate device loss or failure at home and/or at sites. According to an informal poll of the eCOA provider member firms of the eCOA Consortium, device failure is attributed to less than 1% of devices used in the field. As such, web-based backup historically has been of limited need in clinical trials.

While web-based approaches have certainly been used as the primary data capture method in clinical trials, this approach gained significant additional interest and adoption as a result of the COVID-19 pandemic. The pandemic not only disrupted participants’ ability to visit a site, but it also impacted supply chains and associated distribution hubs and pathways for management of provisioned devices and accessories. FDA encouraged clinical trial sponsors to embrace alternative methods of data collection, including remote eCOA data capture.

 In response to that guidance, the eCOA Consortium offered web-based eCOA as one of the viable options to mitigate risk and reduce missing data

, and some eCOA providers repurposed their web-based backup solutions to provide this alternative primary solution.

The primary differences between web-based backup (pre-pandemic) and web-based eCOA (mid-pandemic) were the removal of managed or limited access to the field-based web system and replacement of site-based tablet ePRO with remote web-based ePRO.

The increased use of web-based backup during the pandemic was unanticipated but should lead to reconsideration of backup solutions overall. Given how web-based backup is built to mimic app-based solutions and the compelling amount of data supporting measurement comparability when best practices are implemented, there is sufficient evidence to support a more flexible, device- and location-agnostic approach to ePRO; web-based ePRO is a trusted approach already. 

Instead of relegating web-based solutions to backup only, we should consider them as an integral component of a flexible bring your own device (BYOD) model with greater applications to support traditional, hybrid, and fully decentralized clinical trials. We hypothesize that by allowing greater flexibility in the eCOA technology solution we will reduce the need for backups, specifically paper.

Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient non-compliance with paper diaries. 

Shields AL, Shiffman S, Stone A. Patient compliance in an ePRO environment: methods for consistent compliance management, measurement, and reporting. In: Byrom B, Tiplady B, eds. ePRO: 

Electronic Solutions for Patient-Reported Data.

Ganser AL, Raymond SA, Pearson JD. Data Quality and Power in Clinical Trials: A Comparison of ePRO and paper in a randomized clinical trial. In: Byrom B, Tiplady B, eds. ePRO: 

Howry C, Elash CA, Crescioni M, Eremenco S, O’Donohoe P, Rothrock T. Best practices for avoiding paper backup when implementing electronic approaches to patient-reported outcome data collection in clinical trials. 

Food and Drug Administration. E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) – Guidance for Industry. March 2018.

Food and Drug Administration. Conduct of Clinical Trials of Medical Products During the COVID-19 Public Health Emergency Guidance for Industry, Investigators, and Institutional Review Boards. March 2020; updated on August 30 2021.

Critical Path Institute’s PRO Consortium and ePRO Consortium. Coronavirus Disease 2019 (COVID-19): Risk Assessment and Mitigation Strategies for the Collection of Patient-Reported Outcome Data through Clinical Sites. 

https://c-path.org/wp-content/uploads/2020/06/COVID-19_CPath_v3.0_2020JUN5.pdf

Greater flexibility in eCOA solution could reduce need for backups.