Using patients’ own mobile devices to collect self-reported outcomes data (referred to as electronic patient-reported outcome [ePRO] or electronic clinical outcome assessment [eCOA]) is an industry hot topic. Limited use of “Bring Your Own Device,” or BYOD, in regulatory studies to date is mainly due to industry concerns spanning two areas. The first is a concern that different device sizes and operating systems might affect the measurement properties of a PRO instrument. When employing eCOA on a single device type, the measurement properties can be assessed fully by usability testing, cognitive debrief, or quantitative equivalence studies. In a BYOD setting, performing validation studies to cover all possible device types and sizes would be impossible. The second area is concern around the technical and practical aspects of using a patient’s own hardware. Concerns, for example, include the effect of a subject changing their device mid-study, upgrading their operating system, or having insufficient storage space available to store eCOA data due to other apps, data, pictures, and music.

	Between August and October 2015, we conducted a research survey to identify and assess the perceived barriers and challenges with the use of BYOD for eCOA in clinical trials. Our aim is to provide information helpful in devising future strategies for BYOD adoption, and to help identify popular perceived challenges that perhaps are more myth than reality. In preparing the survey questions, we supplemented our own knowledge of commonly considered challenges and issues with information gathered during telephone interviews of five respected industry eCOA experts.  

	Ninety-eight individuals accessed our survey which was promoted primarily through LinkedIn connections and groups. Of these, 19 individuals answered only the first question, a mandatory question measuring employment type, but did not answer any of the BYOD-specific questions.

 We excluded these respondents, leaving a sample of 79 respondents, and assume that the individuals answering only the initial question did so to proceed but then realized that they would be unable or unwilling to answer the technical questions that followed. 

	Of the 79 respondents, 14 (18%) were employed at biopharmaceutical companies, 18 (23%) at contract research organizations (CROs), and 27 at eCOA vendors (34%) (see Figure 1 above; click to enlarge). For confidentiality reasons we do not report the individual organizations represented, but note that in all categories companies ranged from large to small organizations, and each contained a number of household names. The four respondents in the “Other” category included a patient advocate employed by a number of charities, a psychometrics expert, and two individuals from research institutions.

	In all cases, responses collected represented personal views and not necessarily those of the respondents’ employing organizations.

Attitudes toward equivalence requirements

	While the challenge of proving equivalence across multiple device types seems to dominate the public discussion regarding BYOD, our respondents seemed significantly less deterred by the equivalence challenge:

		Overall, 44% agreed or strongly agreed that equivalence should be demonstrated on all possible devices used in a BYOD study (see Figure 2 below; click to enlarge). 

		Only 30% of respondents disagreed or strongly disagreed that demonstration of equivalence on a single device was acceptable if access using devices of a smaller screen size or resolution could be prevented. 

		In addition, 68% of respondents agreed or strongly agreed that showing equivalence only on a single device would be acceptable if the strategy was agreed a priori with the regulatory bodies. 

		Few saw distinction between primary and secondary data-only 22% agreeing or strongly agreeing that demonstrating equivalence on a single device was necessary only if the data represented secondary endpoints. 

		Seventeen percent of the respondents disagreed or strongly disagreed that no further equivalence testing would be needed if a similar equivalence study had already been conducted and reported.

Few respondents disagreed that scale author agreement would be necessary if using an existing instrument in a BYOD setting-only 17% and 5% disagreeing and strongly disagreeing, respectively (see Figure 2b). The majority of respondents agreed or strongly agreed that ensuring minimum screen size would be sufficient for valid implementation of a visual analogue scale (41% and 27%, respectively), and that differences in font sizes between devices was unimportant (44% and 26%, respectively).

	There was some evidence of trends indicating differing strength of agreement based on the employment type of the respondents, although the sample was not considered large enough to assess this formally. In comparison to CROs and eCOA vendors, biopharmaceutical company respondents generally saw a greater need for equivalence demonstration across all device types, with 72% agreeing or strongly agreeing, compared to 48% among CRO respondents and 34% among eCOA vendor respondents. That said, these sponsor respondents were supportive that equivalence demonstration on a single device was acceptable if usage could be limited to devices of at least that screen resolution and size (79% of sponsor respondents agreed or strongly agreed, compared to 67% and 35% for eCOA vendors and CROs, respectively).

Concern over perceived BYOD practical or technical challenges or issues

	Of the 21 perceived practical/technical challenges and issues associated with BYOD use for eCOA that we considered, few appeared of significant concern to the respondents in this survey (see Figure 3 below; click to enlarge). Over 75% of respondents identified they were “not at all concerned” or “a little concerned” about the following perceived challenges:

		The subject could delete the app during the study.

		The subject may fail to download an updated version.

		The subject may upgrade their device operating system.

		The subject may not be permitted to use their device at work.

		There may be insufficient free storage capacity on the device.

		The subject may become distracted by other things on the device during ePRO completion.

		Some personal identifiable data may need to be collected

		Data on the device could be accessed by a hacker.

		It may be complicated to compensate subjects due to differences in individual data plans.

		Patient setup and training may be more complicated for site staff.

		It may be more difficult to identify that the app is working correctly on the subject’s own device.

		Logging into the app in addition to their device may be inconvenient for the subject.

	Fifty-three percent of respondents were “not at all concerned” or “a little concerned” that the subject may be able to turn off in-app notifications, such as diary reminders, using their phone settings. 

	There was little concern about subjects changing their phone during a study. Seventy-four percent of respondents were “not at all concerned” or “a little concerned” about subjects changing device mid-study, 67% that subjects may discontinue their contract, and 71% that subjects may lose their device during the study.

	Respondents were generally not greatly concerned about perceived security issues with using subjects’ own devices. Sixty-seven percent (67%) were “not at all concerned” or “a little concerned” that eCOA data could be accessed by other apps on the subject’s device, and 83% that data could be accessed by a hacker.

	Almost 20% of respondents were very concerned or extremely concerned that subjects without a suitable device would be ineligible to participate in the study. Thirty-two percent of respondents indicated they were very concerned or extremely concerned that a subject’s device may not pair with a Bluetooth device if used in the study, with 59% “not at all concerned” or “a little concerned.”

	There was moderate concern around training and support of study participants. Twenty-seven percent were very concerned or extremely concerned about the potential training burden on sites in a BYOD study, with 33% of respondents expressing the same degree of concern that site staff may be unable to troubleshoot more technical problems associated with using an eCOA app over multiple device types.

	Again, while the numbers per group prohibited formal analysis, we noted some possible trends that may indicate differing strength of concern over certain perceived issues based on the employment type of the respondents. In comparison to CROs and eCOA vendors, biopharmaceutical company respondents generally appeared more concerned about subjects deleting their ePRO app during the study, subjects discontinuing their device contract during the study, subjects losing their device, data being accessible to other apps on the subject’s device or being accessed by a hacker, and the subject’s device being unable to be paired with a provided Bluetooth device.

	When it comes to demonstrating measurement equivalence across all devices in BYOD settings, over half of the respondents in our survey neither agreed nor strongly agreed that testing was required on all possible devices; and over half agreed or strongly agreed that demonstrating equivalence on a single device was acceptable if all subjects could be guaranteed to use a device of at least that minimum screen resolution and size. Would that strength of feeling translate into the use of BYOD to deliver eCOA instruments in a regulatory study today? Perhaps, but maybe that’s unlikely. However, as we see more and more evidence that electronic devices of all shapes and sizes do not adversely affect the measurement properties of eCOA instruments across different study contexts and patient populations, this position may relax.

	There are already positive signals that measurement equivalence across modalities is less problematic than previously thought, especially if ePRO design best practices are employed (see the C-Path institute’s ePRO consortium white paper for example).

 One of these signals is the growing evidence of paper and electronic equivalence. One might argue that the magnitude of change is far greater from paper to an electronic device, than from device to device. A recent meta-analysis by Muehlhausen and colleagues provides strong evidence of the equivalence of paper and electronic over multiple instruments, patient populations and electronic media.

 This study also included two studies in which the equivalence of two electronic formats was assessed. If patients respond consistently with PRO instruments, whether in paper or electronic form, then it seems a reasonable inference that the subtle changes across different mobile phones should not present an equivalence challenge.

	This isn’t the first meta-analysis we’ve seen exploring this topic. Gwaltney and colleagues published a meta-analysis of 46 equivalence studies conducted up to 2006.

 This analysis reported a pooled correlation of paper to electronic scores of 0.90 with a 95% confidence interval from 0.87 to 0.92. This is above the correlation threshold of 0.75 or 0.8 considered to represent acceptable reliability.

	Muehlhausen et al.’s meta-analysis considered new equivalence studies published from 2007 to 2013. Significantly, these studies were reported after the publication of the ISPOR ePRO Task Force recommendations on the design and analysis of equivalence studies and many, therefore, adhered to the task force recommendations. This new meta-analysis included 72 equivalence studies from 23 different patient groups and included a wide range of electronic modalities including PC, tablet, handheld device/smartphone, and interactive voice response system (IVRS). Their conclusions were in line with Gwaltney and colleagues-a pooled correlation of 0.875 (CI: 0.867-0.884). 

	These two important studies provide extensive evidence that paper and electronically administered PROs are equivalent-across many different PRO instruments, patient populations, and electronic modes of administration. While none of these studies were conducted in a BYOD setting, it seems that device type does not affect equivalence to paper-so we might gain encouragement that device-to-device differences are likely to be similarly insignificant in affecting the way in which patients respond to ePRO instruments if the design of the questionnaire follows the ePRO Consortium white paper design guidelines.

	Should measurement equivalence concerns be assigned to the category of myth? We argue that the body of evidence collected to date strongly suggests this. The above pieces of work, and others actively being conducted, provide a positive signal on the way to greater acceptance of BYOD as a valid approach that protects eCOA instruments’ measurement properties when applied appropriately.

	As we have seen in our survey, however, perceived issues and challenges with BYOD for eCOA are not confined to considerations of measurement equivalence. There are perceived practical and technical concerns with the use of a subject’s own mobile device to collect submission data.

	Some of these concerns are tangible situations that could happen in a clinical trial. Subjects, with full control over the contents and operation of their mobile device, could indeed delete the eCOA app, prevent notifications appearing on their home screen outside the app, may upgrade their operating system during a study or change their device or mobile contract during a trial. Can these risks be mitigated and what is their potential impact on the measurement of the PRO? Certainly some could be limited through training, and additional information from system-based monitoring of the app can help to present issues for patient follow-up by sites. This might include regular receipt of information on the device operating system and version of the app being used to identify when changes have occurred, and flagging when push notification tokens indicate that notifications have been disabled on the patient’s device. 

	However, eliminating the possibility of the user upgrading his or her device or turning off notifications in a BYOD setting will be hard to eliminate completely. Do these risks outweigh the potential advantages of BYOD? We argue the potential benefits of BYOD are greater than these concerns. While the cost of provisioning mobile devices in current trials is high, we do not believe that BYOD will necessarily result in significant cost savings. Some provisioning may be needed to enable inclusion of patients without compatible hardware, and provisioning savings may be balanced by a higher support cost when patients use their own mobile devices to operate study eCOA solutions. 

	It is hoped, however, that BYOD brings with it greater patient convenience and centricity-enabling subjects to utilize their own smartphone to maintain their symptom diaries and instrument entries using the device they already carry with them and refer to over the course of each day. With BYOD, subjects will use a device they are familiar with and know how to use. They will also not be required to carry and keep charged a separate device solely for the purposes of their eCOA entries. Previous patient preference studies have shown that the majority of patients prefer to have a single device, and this can only benefit PRO convenience, completion, and compliance.

	Some perceived issues and challenges, however, can likely be dismissed as myth-at least in the sense that they do not apply specially to BYOD and in fact apply equally to other approaches to PRO collection. Subjects can equally lose a provisioned device or 

paper diary as opposed to their own mobile device, and subjects may be equally unable to use a study device as opposed to a personal device in a working environment. In an unsupervised setting, subjects may be equally distracted by their own smartphone while completing a paper diary or a diary on a dedicated study device; and the same requirements for collection of personal identifiable data apply to both BYOD and provisioned device studies.

	Other issues fall into the category of surmountable technical considerations, which should be addressed by good mobile app design. Security, for example, while an important concern, has not limited the prevalence of online and mobile banking services. There is no reason why we cannot learn from the application of technology solutions in other industries to gain confidence and develop solutions that are appropriate for healthcare and clinical trials. While the banking industry has the benefit of large investment, leveraging their R&D may be less expensive, and online banking has already had an impact on user behavior and acceptance of online solutions to manage sensitive information.

	We believe that the time for BYOD is upon us. With that we see greater potential to apply eCOA to study protocols where paper data collection remains quite popular despite its well-known limitations. As an industry, we should continue to investigate the use of BYOD and share our findings, positive and negative, so that as a collective we can provide sufficient evidence to turn the tide. 

	FDA recently requested public input from a broad group of stakeholders on the scope and direction of the use of technologies and innovative methods in the conduct of clinical investigations.

 This docket includes a specific question for comment: “What are the challenges presented when data are collected using the Bring Your Own Device (BYOD) model?” This is a positive signal from the regulators that we welcome and one that can only help to ultimately provide better understanding of FDAs position and any gaps in evidence required to make BYOD a fully endorsed approach.

 is Senior Director, Product Innovation, ICON Clinical Research Ltd.; 

 is VP, Chief of Staff, Medidata Solutions; 

 is Senior Director, Product Management, Medidata Solutions; 

 is Director of Product Innovation, ICON Clinical Research; 

 is Vice President, Head of Innovation, ICON Clinical Research

	1. C-PATH Institute ePRO Consortium (2014). Best Practices for Electronic Implementation of Patient-Reported Outcome Response Scale Options.

 http://c-path.org/wp-content/uploads/2014/05/BestPracticesForElectronicImplementationOfPROResponseScaleOptions.pdf

	2. Muehlhausen W. et al. (2015). Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013.

http://hqlo.biomedcentral.com/articles/10.1186/s12955-015-0362-x

	3. Gwaltney C. et al. (2008). Equivalence of Electronic and Paper-and-Pencil Administration of Patient-Reported Outcome Measures: A Meta-Analytic Review. 

	4. Federal Register (2015). Using Technologies and Innovative Methods to Conduct Food and Drug Administration-Regulated Clinical Investigations of Investigational Drugs; Establishment of a Public Docket. 

https://www.federalregister.gov/articles/2015/10/29/2015-27581/using-technologies-and-innovative-methods-to-conduct-food-and-drug-administration-regulated-clinical

Articles

Feature Article

Survey uncovers the challenges, myths, and potential useful strategies associated with BYOD adoption.

Bring Your Own Device for Trial Outcome Assessment

	FDA’s breakthrough drug initiative is much more successful than ever anticipated, attracting hundreds of applications and speeding dozens of life-saving and important new therapies to patients. More than 45 new drugs and biologics with the breakthrough therapy (BT) designation have come to market since the program was established in 2012, many benefitting from special assistance from FDA experts. 

	Even more notable is that the program actually appears to be accelerating clinical development of promising therapies-as opposed to just speeding  up the agency’s approval process. An 

 by Friends of Cancer Research (FOCR) finds that pre-market development time for breakthrough-designated drugs is 2.2 years shorter than for those without the BT imprimatur. These gains arise from more streamlined clinical testing protocols negotiated early in development by FDA reviewers and sponsors; a single arm trial with fairly few patients may suffice if treatment effect appears very large. But the BT program involves a lot of work for FDA:  the agency has vetted more than 300 breakthrough designation requests (BTDRs) and granted BT status to about one-third of these therapies.

	The program’s success also has created some confusion in the healthcare community about the significance of drugs receiving BT status. More than half of physicians believe that BTs have been proven safe and effective in randomized trials, according to a recent study in the 

Journal of the American Medical Association

 (April 12, 2016), and consumers have high expectations for new drugs described as “breakthroughs,” says another

	FDA seeks to address these overly optimistic expectations for BTs by clarifying its terms and improving BT program operations. Although these drugs may “look really good” in early trials, many may “fail to deliver on the promise” initially shown, explained Richard Moscicki, deputy director of the Center for Drug Evaluation and Research (CDER) in a recent 

. He noted that BT drugs must treat serious conditions and present evidence of their potential for providing “substantial improvement” over available therapies, a term that FDA is looking to define more clearly. While a head-to-head comparison of a new drug to standard of care may be ideal for documenting results, evidence that a BT reduces progression of disease may suffice when more extensive data is not available. 

	To be able to provide assistance to sponsors of true BTs, FDA also seeks to limit the volume of inappropriate BTDR requests. A new “preliminary BTDR advice” request process asks sponsors to file a short form with basic information on its BT candidate, namely whether the indication is serious and life-threatening, other available drugs, and early clinical evidence (trial design, endpoints, number of subjects enrolled). The relevant FDA review division will discuss the request in a teleconference and offer nonbinding advice on whether to file the full BTDR at that time. 

	Meanwhile, BT therapy development is leading to more “seamless” clinical research programs, particularly for oncology treatments. Instead of three distinct clinical trial phases, FDA is encouraging studies that can rapidly enroll patients based on preliminary evidence of effectiveness. Some patient advocates, though, have objected that such approaches increase risks to patients. FDA officials commented in a recent journal article that well-designed protocols, clear rationales for expanding patient cohorts, and updated informed consent can address these issues. FDA also 

 to limit the seamless trial model to drugs receiving the BT designation and thus more intensive oversight and communication with agency experts through product development. 

	Another concern is whether expedited development programs can fully incorporate patient experience in clinical trials. Although sponsors increasingly are tapping patient groups for input on study design and key endpoints, accelerated studies may conclude before they can collect and assess patient-reported outcomes and other measures. One approach supported by FDA leaders is to encourage more standing clinical trials able to investigate innovative therapies more efficiently by reducing the need to identify sites, recruit patients, negotiate research agreements, and enlist CROs for every new promising treatment. 

	FDA and the research community also are examining whether expedited development programs can efficiently evaluate combination therapies and diagnostics needed to identify patients likely to benefit from a personalized breakthrough medicine. Patient advocates led by FOCR have proposed that FDA establish a “virtual” Oncology Center of Excellence that would better coordinate scientists and reviewers in separate FDA Centers to achieve a more integrated approach to regulating combination cancer therapies. FDA has offered strategies that will improve the ability of its Office of Combination Products to accomplish similar goals, but may be willing to devise a pilot program to test this new “disease-oriented” regulatory approach. 

News

FDA's breakthrough drug initiative has proven successful to date, but challenges remain in addressing the expectations and patient concerns surrounding candidates in this program.

‘Breakthrough’ Program Generates Optimism — and Concerns

	Antimicrobial resistance is hitting the world headlines at present for several very good reasons-and the clinical trial community in particular may benefit from all this new attention to an old problem.

	The problem of resistance is almost as old as antibiotics, since it is in the nature of microbes that they adapt and evolve. But the new attention is a response not only to the increasingly familiar and terrifying predictions of the risks of the coming decades, or to the release in May of the long-awaited O’Neill 

 “Tackling drug-resistant infections globally,” or to the 

 at the World Health Assembly at the end of May on its “Global action plan on antimicrobial resistance,” or the discussions among world leaders at the G7 summit in Japan. The real driver is the painful awareness that only a trickle of new antibiotics are emerging to compensate for the erosion of the existing arsenal (even the last-resort carbapenems), and that the world’s entire R&D pipeline contains less than 50 candidate products.

	The momentum now gathering at the international level to tackle the threats more effectively is throwing what may prove to be some welcome light into some of the corners of clinical trial regulation that have until now been too esoteric and obscure to capture public interest.

	 One of the main points that the O’Neill report makes is that antibiotic research and development is hindered by regulatory challenges-which it follows up with a fact that is well known in the world of clinical trials, but not so well known in the wider world: “The greatest cost associated with new antibiotic development is that of running clinical trials-particularly during the later stages of testing.”

	 O’Neill goes on to remark that on average more than 80% of the costs of bringing an antibiotic to market are related to clinical trials. It is right, the report concedes, that testing and approval processes should be robust to prevent unsafe or ineffective drugs coming to market, but it then explains -in a way that is capable of changing public perception-that antibiotics face particular challenges, and that those have contributed to the progressive decline of R&D efforts.

	 As an example, it cites the cases of antibiotics that are intended for use as back-up defences for current generics to which resistance is rising. These need, in principle, to demonstrate clinical “superiority” versus the existing treatment, it points out, and “identifying and enrolling large enough groups of patients with drug-resistant infections can be a technical and logistical challenge.” Furthermore, companies and health technology assessment agencies must consider how to establish fair assessments of value for money and cost-effectiveness for new products approved via adapted non-inferiority trial processes.

	 Its recommendation is to form clinical trial networks for antibiotics, not only to reduce the time taken to get each trial up and running, but also to speed up the trials themselves and reduce costs. “Sharing ‘control arms’ between trials, the overall size of each trial could be reduced by more than 40%,” O’Neill speculates, adding that regulators could also benefit from all trials undertaken using the same protocol.

	 The report complements the efforts by the US FDA, the Japanese Pharmaceutical and Medical Devices Agency, and the EMA in Europe for their alertness to these concerns and the steps they have taken to address them. But, it goes on, “more can be done to support antibiotic development by improving the regulatory process.”

	 Because, the report points out, “even when regulators manage to harmonize and simplify requirements for developers to bring new antibiotics to the market that are effective for those patients with a resistant infection, there remains a separate challenge, which is the question of how to price these antibiotics.” The demands from healthcare providers and price-setting authorities for robust clinical evidence of the value of new drugs against comparators are legitimate, the report says-so “there is no escaping relatively large clinical trials.”

	One way of reducing these costs, argues O’Neill, is for regulatory agencies to work more closely together to improve the global harmonization of regulatory pathways for new antibiotics, and explore the possibilities for mutual recognition of regulatory approval across multiple jurisdictions. If pharmaceutical companies, regulators, and healthcare system leaders cooperate, they could set up national and regional “clinical trial networks” for antibiotics, to streamline the clinical trial process and reduce the costs and duration of antibiotic development.

	"With the exception of EU member states, almost every country in the world requires new antimicrobials to be registered individually with them," the O'Neill report points out. It underlines that the consequent task of filing registrations and paying fees in up to 170 different jurisdictions in order to achieve global market access for a new product is "an expensive and slow process."

	What has been for years an obvious answer in the clinical trial community is now gaining traction in public consciousness: "Harmonizing regulatory procedures can have a high impact in reducing costs and improving access," says O'Neill, which notes the steps already taken by the EMA, FDA, and PMDA to align their processes for new drug approval more closely. The steps win a ringing endorsement: "These efforts should continue and aim to go further-for instance, to explore opportunities for mutual recognition of market approvals-with input from partner agencies in other parts of the world to achieve streamlined approval processes that work to benefit both patients and product developers."

	In other words, O’Neill is chipping away in a very public fashion at some of the barriers that have for years bedevilled drug development-and not just in the field of antibiotics. The current alarm over the risks of exhaustion of the antibacterial armamentarium is giving more exposure to some of the underlying challenges of drug innovation that have long remained remote from public attention. The unsung heroes of clinical trials have battled misunderstanding and indifference for decades as they pointed to the inflexibility of clinical trial rules, the perverse diversity of national regulation, and the persistent failure by regulators and payers to perceive the efficiency gains from a comprehensive approach to evidence generation.

	Now, blown on the turbulent wind of AMR, many of their concerns are getting an airing that could help advance solutions that might extend across a much broader field than antibacterials. But, as O'Neill concludes, "so much more remains to be done over the rest of this year and the following years." How alert is the broader clinical trial community to the opportunity that is now open?

Peter O'Donnell is a freelance journalist who specializes in European health affairs and is based in Brussels, Belgium

View from Brussels

Renewed attention on ways to tackle this age-old scourge is building in Europe and beyond.

Antibiotic-Resistance Focus Could Benefit Clinical Trials

	The European Forum for Good Clinical Practice (EFGCP) is stepping up its efforts to involve more older people in clinical research by focusing on two central questions: What can be done about it? What are the problems and the solutions?

	There’s an urgent need to obtain better evidence for future treatment of older people (i.e., over 75) and for providing them with optimal care, according to Florian von Raison, MD, and Laurence Hugonot-Diener, MD, the co-chairpersons of the EFGCP’s Geriatrics Medicine Working Party in Brussels.

	“There is only weak evidence-based guidance coming from often poor and insufficient clinical research, and this vulnerable population is often underrepresented in research and the development of drugs,” they noted in a statement issued in advance of the late September workshop on the topic, to be held in Nice, France. “If study participants do not represent the average patient in daily practice, the consequence is that many medical decisions for older people are extrapolated from clinical trial data derived from younger population.”

	Drugs are often prescribed to older patients with very little idea of efficacy, dosage, or adverse effects, they believe. Global evaluation of their needs and frailtiesâ including co-morbidity, polypharmacy, disability, and cognitive impairment, is necessary in order to reduce the risk of a drug’s inappropriate use and to evaluate the chances of benefit, the risks of harm, and the cost-effectiveness of treatments.

	Three years on from the EFGCP’s publication of detailed guidelines on older people and research, the organizers are keen for everybody to take a fresh look at this subject. Raison thinks several key areas of ethical concern in geriatrics remain:

		Different reaction to medicines from other adults

		Differences in pharmacokinetics and pharmacodynamics 

	âThe aim of the Nice event is to clarify definitions, identify issues, and find possible solutions in order to allow a comprehensive geriatric assessment in future research and regulatory measures. This will include the contributions from up to 10 emerging researchers. Following the postponement of the EFGCP workshop about “Excluded Older People from Clinical Research,” which was to be held in March 2015 in Barcelona, Spain, the organizers want to regain lost momentum.

	The Geriatrics Medicine Working Party comprises a multi-stakeholder forum involving European members from academia, regulatory bodies, and the pharmaceutical industry and patient groups. It focuses specifically on social aspects (present and future healthcare of elderly populations), medical issues (the need for appropriately studied and labelled medicines, with particular attention to avoiding iatrogenic problems), and ethical concerns (ensuring appropriate healthcare and research protection for older people).

	The group’s prime goal is to examine issues in the development of geriatric medicines that include approaches taken by ethics committees, information on medicines, and medical research provided to elderly patients, informed consent in elderly populations, new regulatory approaches, specific pharmaceutical formulations, and strategic approaches to educating society.

	The EFGCP workshop will take place on September 27 at the Institut Claude Pompidou in Nice, and is being organized in partnership with Innovation Alzheimer.

European forum tackles the problem of limited elderly patients involved in clinical trials.

EFGCP Renews Campaign on Older People in Research

	In January 2014, the FDA presented a webinar “Promoting eSource Data Capture,” to explain its Final Guidance on Electronic Source Data in Clinical Investigations. In a slide, the FDA said the worst case for source data capture is “transcription of data from paper or electronic sources to the eCRF (electronic case report form).” The reason being that there was no intervening paper step needed, less confusion on what is source and the decreased likelihood of data errors.

	In April of this year, we conducted a survey with SCORR Marketing to evaluate the paperless trial. In this survey, of 143 respondents, 25% said 0%-10% of their data collection in clinical trials was paperless. The next highest was 22%, who said that greater than 75% of their trials were paperless. And 15% said between 51% and 75% were paperless. So the majority fall in the middle of paperless processes, leaving room for greater adoption in the coming years.

	In fact, the majority of respondents-27%-said that in the next three years, 75% to 99% of the processes for clinical trials would be paperless. With the plethora of new technologies surrounding data aggregation, mining, and analytics in all critical areas of study conduct, we can only see that our respondents are correct in their view.

	For more information, download SCORR’s free report 

Survey reveals mixed adoption of paperless data collection in clinical trials, pointing to the need for greater alignment of new eClinical technologies and study conduct.

Paperless Processes on the Way Out?

	Now that the era of digital health is upon us in the U.S. and other countries, it’s time for a fresh look at how we collect research data. What if we could make a great leap forward by completely changing our approach through electronic health record (EHR) eSource? Would we dare to try?

	“Source” is the initial recording of data for a clinical study.  When the original recording is on digital media rather than paper, it’s “eSource.” Clinical trials have used eSource for years in ECG readings, lab results, and other measurements.  The FDA eSource Guidance describes different ways to transmit eSource data (from direct capture, devices, transcription, EHRs, or patient-reported outcome instruments) to an electronic case report form (eCRF) system, and several approaches have been proposed for trying to feed EHR data into our existing EDC-based processes. Last year the FDA even asked for demonstration projects to explore such approaches. 

	But a more recent draft Guidance on Use of EHR Data in Clinical Investigations offers another take entirely with explicit goals to “facilitate the use of EHR data in clinical investigations” and to “promote the interoperability of EHRs” with clinical research systems.  The guidance recognizes that the ONC Health IT Certification Program can indicate the readiness of EHRs to support research. And ONC’s Advancing Care Information initiative relies heavily on leveraging application program interfaces (APIs) to make health data more timely and accessible to patients and caregivers.

	This is where HL7’s FHIR platform standard comes in:

		FHIR’s Data Access Framework will provide a universal API to EHR systems that can be used to populate much of a casebook in a clinical database. 

		The Smart on FHIR specification demonstrates how patients can grant researchers access to their data through electronic informed consent, as well as input outcome data through smartphones and browsers-data that can be directed to an EHR or a trusted third-party cloud-based research repository simply by selecting the appropriate target FHIR server.

		Since EHR data is eSource, FHIR can also provide authorized access to remote study monitors.

		And since FHIR can update as well as read data, it can also support the processing of data clarification transactions, thus making it possible to synchronize EHR records with clinical databases, improving transparency and traceability for both monitors and regulatory inspectors.

		FHIR makes it possible for regulatory reviewers to delve into the full EHR database to explore, for example, serious adverse events, in more depth than was ever possible before.

	In the future, it may not be necessary to have an eCRF system in the middle of the process. That would enable using the EHR data directly to feed our analysis so that all health data could potentially be reused as research data. 

Wayne Kubick, is Chief Technology Officer, Health Level Seven International, and a member of Applied Clinical Trials’ Editorial Advisory Board 

Recent FDA draft guidance pushes for the use of electronic health record data in clinical investigations-and synching EHRs with research systems.

EHR eSource: Sword of Change?

	In December 2015, the ICH updated its E14 Guideline Q&A to define an alternative path for identifying the cardiac safety issue of QT prolongation in non-cardiac drugs. This is the most fundamental revision to the Q&A of “The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrhythmic Potential for Non-Antiarrhythmic Drugs” since its implementation in 2005.

	This alternative path-call it “concentration effect modeling”-relies on intensive, high quality ECG analysis and the use of exposure response modeling to determine the extent of QTc prolongation. Proof of its validation is FDA’s acceptance of this approach in lieu of a TQT study earlier in 2015.

 spoke with Norman Stockbridge, MD, PhD, Director of the Division of Cardiovascular and Renal Products in CDER’s Office of New Drugs, about this development. Stockbridge noted that it has always been a known that TQT studies are inefficient. He said, “It led to sponsors doing a separate study instead of piggybacking onto an existing study. Then because it was a special study, it was only conducted late in development. Also, the previous by-time-point analysis doesn’t use all the information; it just uses the information where the QT interval is the worst.” 

	However, getting to a new approach that also had an acceptable confidence level for the older TQT approach, took time In the end, Stockbridge says, FDA asked for a trial that formally compared the two approaches. The Cardiac Safety Research Consortium (CSRC) and IQPharma members initiated the IQ-CRSC study, of which the FDA was heavily involved in the design. 

	For more information, download the free report 

CDER official comments on the significance of ICH's new alternative path for identifying the cardiac safety issue of QT prolongation in non-cardiac drugs.

Critical Update to Cardiac Safety Assessments

	FDA defines electronic source (eSource) data as data initially recorded in electronic format. Now that direct data entry (DDE) at the time of the clinic visit, mobile devices, telemedicine, and remote monitoring of clinical trials are all realities, and are no longer “pie-in-the-sky” concepts, it is time for a transformational change in our processes as we plan, design, and execute trials, as well as how we will collect, analyze, and use the potentially huge amount of data coming off mobile devices. 

	While the pharmaceutical industry has always used and analyzed data from multiple sources, such as central laboratories, ECG and diagnostic devices, MRIs, etc., the “mobile apps” that are currently flooding the market, if used in the clinical trial space, will provide challenges to data quality as we deal with the large volumes of data collected and with the different operating systems and hardware. Therefore, the current eSource transformation is fraught with challenges that must be overcome, including regulatory compliance, cooperation, and acceptance by the patients and clinical research sites, who are the customers of our 

trials, and strict approaches to validation of these electronic tools. We must also separate the “toys from the tools.” Just having a “neat app” won’t fly in the regulated ecosystem we all must live in.

	Fortunately, the regulators have beat us to it. There are now FDA and EMA guidance on eSource, eInformed consent, mobile devices, and risk-based monitoring, with more to come. In terms of data integration, there are also active programs to integrate the electronic medical record (EMR) with clinical trial electronic data capture (EDC) systems, and companies are starting to include the use of FDA-approved mobile devices such as glucometers, blood pressure monitors, and ECG devices as part of their clinical trials. 

	It’s critical to involve regulators when the use of these devices impacts the management of subject safety and the evaluation of primary and secondary endpoints. Also, as potentially non-regulated devices are used or regulated devices are used “off-label,” we must be ever-vigilant on the software validation process, the impact of app performance when used with various operating systems and devices, as well as the impact of system upgrades on the validated state of software.

	As we collect continuous data, such as ECG readings, number of steps taken during wake time, or number of awakenings when participating in a nocturia trial, the software should have validated algorithms to summarize the outcomes rather than requiring the pharma or device company to do the analysis from scratch. This will be one of the main advantages of using FDA-cleared devices presenting with validated software. For example, the Vicor PD2i Analyzer is a software algorithm for recording heart rate variability (HRV) using the point correlation dimension algorithm (PD2i). The intended use of the tool is to display and analyze electrocardiographic information and to measure HRV. 

	The following quality measures were applied to the development of the system and given to FDA as part of the approval process: 

	This is an exciting time in drug and device development, where the goal is to improve processes and reduce the time to get safe and effective drugs and devices to the patients who need them. However, we must make sure that as we “do more with less” (the words first coined by visionary Buckminster Fuller), we do not add more complexity to an already complex system.

, MBA, PhD, is President, Target health inc. 

A Closing Thought

The need to involve regulators is crucial when the use of electronic data devices impacts the management of patient safety and evaluation of trial endpoints.

Regulatory Considerations are Key in eSource Data Integration

Prior to a patient’s enrollment in a clinical trial, the prospective participant is often handed a document that is 15 to 25 pages long. Known as the informed consent form (ICF), the document is largely comprised of a combination of medical terminology and “legalese” that even members of the pharmaceutical community would have difficulty comprehending in full. To further complicate matters, important information on the benefits, and risks of the clinical trial experience that patients are about to undergo is often only a small part of the document. This practice is outdated and is not consistent with current calls for increased patient engagement in the clinical trial process. 

	For many clinicians, informed consent is thought of as a signature on a document, but it should be a process that leads to greater understanding of what lies ahead for patients participating in a clinical trial. Though it took our industry 20 years to embrace electronic data capture (EDC), and at least 10 years to embrace electronic reported outcomes/electronic clinical outcome assessments (ePRO/eCOA), both now play an increasingly crucial role in new drug development. Electronic trial master files (eTMF) are also making significant gains. Still, even though “patient centricity” has become the new buzzword, our industry has not yet implemented an electronic informed consent (eIC) process that would benefit our patient partners in clinical trials.

	The FDA published guidance on informed consent in 2014

 and followed up with guidance on eIC in 2015.

 The guidance document defines eIC as using electronic systems and processes that may employ multiple electronic media to convey information related to the study and to obtain and document informed consent. The general principles of each are similar to the following recommendations for obtaining patient agreement:

		Description of the clinical investigation

		Compensation and medical treatment in the event of injury

		Patient understanding that participation is voluntary

		The total number of patients in the study

	In addition to the above, the patient should understand what will happen if the study, or their participation in the study, is terminated either involuntarily or voluntarily, and the patient must be made aware that new, unanticipated adverse events may occur during the course of the trial. Importantly, proof of comprehension is not a requirement. Consequently, information about whether the patient truly understands what they have signed can be sparse. This should be of great concern as informed consent is, by its intent, established to ensure that the patient is both freely participating in a process and making an informed decision based on a complete understanding of the benefits, risks and alternatives.

How effective is the current paper process?

	Although the current adoption rate for eIC has been slow both in clinical trials and in hospitals, there have been several studies conducted in medical centers proving the value of computer-assisted learning and consent that date back over 20 years. 

	Many studies have shown that most patients are unable to recall or do not understand most of the information that is presented to them in the informed consent process. In 2009, Finch et al. presented 100 patients scheduled for transurethral prostatectomies. When the patients were interviewed three hours after consent was obtained, less than 50% of them could accurately recall the risks or potential complications.

 In another study, only 18% of guardians who provided consent for surgery in pediatric patients could recall the specifics of the procedure two to four weeks after the surgery.

 Krupp et al. showed that 65% of patients who consented for neurosurgery were unable to remember more

 than two of six major risks associated with their surgery only two hours after informed consent was obtained.

	In 2007, an article published in the American Heart Journal revealed that among 633 patients who were to undergo coronary artery bypass grafting or percutaneous coronary interventions, there was poor agreement between what patients expected as symptom benefits versus what they were told by their physicians. On the important outcome of mortality, there was no correlation between what physicians and patients expected because patients believed there would be an improvement in survival even when physicians did not describe this as a benefit.

	Cassileth et al.7 had 200 cancer patients complete a test of their recall within one day of signing consent forms for chemotherapy, radiation therapy, or surgery. Only 60% understood the purpose and nature of the procedure, and only 55% correctly listed even one major risk or complication. Further, a mere 40% of the patients stated that they had read the form “carefully.” Although most thought that consent forms were necessary, comprehensible, and contained worthwhile information, the legalistic connotations of the forms appeared to lead to cursory reading and inadequate recall. In general, most believed that consent forms were primarily meant to “protect the physician’s rights.”

	Although these studies were conducted in a healthcare instead of a pharmaceutical clinical trial environment, it is reasonable to assume that the patients in each environment would be similar. Of greater concern is that a typical informed consent for a hospital procedure is generally one to three pages, but many clinical trial ICFs are 20 or more pages. Naturally, this significantly increases the problem of comprehension and retention.

	In 2013, The Center for Information and Study on Clinical Research Participation (CISCRP) conducted a Perception and Insights Study that included 724 patients who had participated in clinical trials. Those that participated in North America were at least somewhat satisfied that their questions were answered with the current process-85% of the time. However, approximately 33% of patients outside North America were dissatisfied in regard to the answering of their questions. Importantly, as an indicator of what is to come in the future, 28% of respondents ages 18 to 34 were not satisfied that their questions were answered. Younger generations are used to receiving a lot more information than their elders. As they continue to age, we can expect that the level of dissatisfaction with the current process will grow exponentially. 

	The same CISCRP study also measured the impact of dissatisfaction with the informed consent process on patient retention. The results of the study (see Table 1 below; click to enlarge) showed that patients who were not satisfied with or did not understand the informed consent process were much more likely to drop out of a trial than those that were satisfied with the process. As patient recruitment becomes more difficult and costly, it is essential to retain those patients that have entered a clinical trial. There is a substantial cost when a patient is lost after passing through the screening process, and the cost is even greater if they have also started treatment and undergone a significant amount of testing. Additionally, there is no potential benefit in regard to proving the primary endpoints needed for drug approval. Time is also lost as new patients now need to be recruited to fill the void.

What is the impact of the patient’s native language?

	The importance of language on patient understanding of informed consent is often overlooked, especially if English is not the patient’s native language and he or she is in an English speaking country. Patients who have a poor understanding of English are less likely to receive adequate informed consent. For example, when the charts of 74 Spanish- and Chinese-speaking patients who underwent thoracentesis, paracentesis, or lumbar puncture at a hospital with well-trained interpreters in Spanish and Chinese were compared with those of 74 English speakers, only 28% of foreign language-speaking patients had well documented informed consent, versus 53% of the English speakers.

	Trials of video educational tools have shown greater patient understanding. For example, a randomized controlled trial of an informational video for women considering laparoscopic tubal ligation showed that women who watched the video, in addition to standard consent procedures, demonstrated 56% higher knowledge scores than women who were engaged in standard consent procedures alone (p<0.001).

	In another example, English- and Spanish-speaking patients receiving intravenous contrast for computed tomography who viewed a video in their preferred language showed 20% higher knowledge and 10% higher satisfaction scores than those who did not watch the video. This result was consistent for Spanish and English speakers. Importantly, Spanish and English speakers in the video group had similar post-consent knowledge scores while Spanish speakers in the non-video group had significantly lower scores than English-speaking controls.

 completed a research study with a presentation of the same information using paper versus an iPad device. Among patients (n = 55), iPad subjects had significantly higher test scores than standard paper consent subjects (mean correct = 75% vs 58%, p < .001). The total time spent reviewing the paper consent was 13.2 minutes, significantly less than the average of 22.7 minutes total on the three components to be reviewed using the iPad (introductory video, interactive quiz, consent form). Overall, the study demonstrated that combining multimedia videos, interactive assessment questions, and standard consent language on a tablet-based system improved comprehension of research study procedures and risks.

What is the electronic informed consent process?

	Electronic informed consent can be obtained at clinical sites via a computer or tablet (preferred) or can be obtained over the web as long as the user provides sufficient information that the person signing the document is actually the patient. If the information for consent is obtained via the web and the patient still has questions, the site clinician needs to have a procedure in place to ensure that the patient’s questions are answered before the form is signed. 

	The process should begin with a secure login by the site staff or by the patient if consent will be obtained remotely. User groups should be defined, by role, to determine their access to information. The patient can then watch videos that will explain the clinical trial process and contain the elements described earlier that should be required of the informed consent process. During or after the video segment, patients may be asked questions to assess their retention of the information imparted. To reinforce the learning process, any incorrect answers a patient may provide should be visually corrected on screen, showing them the correct answer. Once the video and assessment questions are complete, the patient will then have access to a digital screen copy of the ICF for review. 

	Tablets provide an additional opportunity to use built-in dictionary functionality on the device to allow patients to look up definitions of terms used within the ICF. Patients can note and record any questions they have to discuss with the investigator. The patient or site should be able to print the consent form at any time. Upon completion of the process, a digital signature can be obtained for the patient and/or guardian, the investigator, and a witness. The entire process will be entered and recorded in an audit trail.

Why use electronic informed consent versus paper?

	The eICF process allows for improved patient engagement and understanding from the outset of the clinical trial process. The interaction with the site is enhanced by using a system that enables patients to self-educate in their native language and prompts them to ask questions. At the same time, institutional review board (IRB) and independent ethics committee (IEC) review is enhanced by allowing them to review the script of the patient video alongside the ICF via a web-based portal. They can also possess a greater certainty that all sites will communicate the needed information in a consistent manner. 

	Sponsors can monitor regulatory issues by viewing a complete audit trail, which will show precisely when patients viewed the information, how long they took to review the video and the ICF, what additional questions they had, and their degree of retention of the information given. This process also allows for adjustments to the videos and/or ICF if a large number of patients are unable to answer certain assessment questions correctly. Importantly, it is currently difficult to monitor what version of an ICF a patient has signed if the study has protocol amendments containing adverse events that were not previously noted. eICF systems can be used to notify both the clinical sites and the patients for follow-up in these situations. Finally, from a site monitoring standpoint, digitally stored documents are easily stored and easy to review via a web portal. The benefits just mentioned are outlined in Table 2 below (click to enlarge).

	As has been the case with most new technology, the greatest challenges to adoption are inertia and cost. As paper has served the purpose of obtaining consent, and it is viewed as easy and inexpensive, the challenge is convincing sponsors that there is a significant enough benefit to outweigh the additional expense and learning curve of adopting a new technology. This can be difficult as the costs of paper processes are often not well quantified. Studies with few patients and multiple languages can result in considerable additional cost primarily due to the fees for translation and voice-over of the videos produced. Larger studies with fewer languages can be much more cost efficient. 

	Several pharmaceutical sponsors view the eIC as part of their patient recruitment process, especially for rare diseases where a consistent well-scripted message can lead to increased enrollment as well as retention. In these cases, cost justification is easier with the value proposition becoming part of the decision-making process of the clinical team.

	Finally, there is still concern about IRB and, particularly, EC adoption of both the educational videos and the eICF forms. There are certainly individual reviewers who are still not completely comfortable with this process but, in general, we have found most IRBs and ECs to be very receptive, as they recognize that this method is well controlled and that the message received by patients is standardized.

	Patient engagement in clinical trials should start at the beginning and continue throughout. The best way to start a process that is patient-centric is to ensure that the patient is fully aware of what her or she is signing up for. Many studies conducted in the healthcare and clinical trial setting have shown that we are currently falling short when it comes to providing our 

patients with the information they need to truly give informed consent. Providing patients with video educational tools and retention aids that explain information in layman’s terms and in their native language improves both patient recall and comprehension. 

	eICF has been used worldwide and is accepted by IRBs and ECs, but the overall adoption rate has been slow. The regulatory safeguards for documentation and re-consenting, the ease of use for ICF approval on a local level, and the benefits to the patient will hopefully move the ICF from being one of the last remaining bastions of the paper process into the modern digital era.

, MD, is a co-founder of Patient Genesis; email: jeffrey.litwin@patientgenesis.com

	1. Informed Consent Information Sheet, Guidance for IRBs, Clinical Investigators, and Sponsors July 2014, 

http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM405006.pdf

	2. Use of Electronic Informed Consent in Clinical Investigations Questions and Answers Guidance for Industry, March 2015; 

http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm436811.pdf

Why electronic informed consent is key to supporting today’s patient-centric mantra in clinical trials.

Applied Clinical Trials-06-01-2016

Bring Your Own Device for Trial Outcome Assessment

‘Breakthrough’ Program Generates Optimism — and Concerns

Antibiotic-Resistance Focus Could Benefit Clinical Trials

EFGCP Renews Campaign on Older People in Research

Paperless Processes on the Way Out?

EHR eSource: Sword of Change?

Critical Update to Cardiac Safety Assessments

Regulatory Considerations are Key in eSource Data Integration

Engagement Shift: Informed Consent in the Digital Era

Emphasizing ‘Solutions’ in eClinical Tools for Sites

Quality Remote Monitoring: The Tools of the Game

Applied Clinical Trials, June/July 2016 Issue (PDF)