When is EDC not ePRO?

Mea Culpa-or “that wasn’t what I meant at all…” Around a year ago I followed a piece in Applied Clinical Trials entitled “EDC: Caught in a Trap” with a longer article in The Monitor entitled “Beyond the Value of EDC-the eClinical Paradigm Shift.” Underlying the base premise of these works was the idea that a non-integrated-a “homogenous” system-was capable of many things that went a long way toward being a true eClinical system: that is, a system that automates or facilitates all or nearly all study activities. What was not emphasized adequately, is that the desired end result must drive selection of the technology-to not do so, and assume that a given technology can perform one function well simply because it does others adequately is a road to frustration and failure. Thus, as always, beginning with the end in mind is appropriate: asking the question of yourself and your project “what am I trying to achieve?” and basing decisions fundamentally on the answer, is a very wise practice. To assume a single system can do everything is...silly. Let’s consider, to prove the point, the idea of ePRO (electronic patient reported outcomes). There are some EDC vendors out there who maintain that their offerings can do pretty much anything, and that includes ePRO. Is this true? Certainly there are EDC systems that can allow patients to access PRO Web pages and record responses. Is this adequate? Is it credible data? Well, that depends. ePRO has been given particular credence by researchers since the publication in the prestigious British Medical Journal (BMJ) in 2002 [1] of evidence that strongly suggested patient non-compliance with paper diaries was very high, and the data capture thereby was suspect at best-in contrast, ePRO was almost the polar opposite. Further work published in June 2004 in Applied Clinical Trials bore out the results of the BMJ paper.[2] For any computerized system’s data to pass muster with the FDA, it must have certain characteristics. These can be neatly summed up in the acronym “ALCOA,” that is, Attributable, Legible, Contemporaneous, Original & Accurate. These are appropriate features of any and all data really-thus it makes sense that it is prime amongst the FDA’s requirements for electronic data.[3] You might wonder about “contemporaneous,” but as we’ll see shortly, there’s a very good reason for it being present, as it has a definite effect on the last element mentioned-Accurate. Let’s examine the idea briefly of a fundamentally EDC based system and how it might try and effect ePRO by relating the elements of the ALCOA acronym to the notion and see what pops out the other end. For the sake of the argument, we’ll assume the EDC system is web-based, although it really doesn’t matter. For the comparator, we’ll use a standard handheld ePRO device that either uploads data via a modem or wirelessly. Attributable. This is largely managed via the use of passwords and user names. For this element, both EDC and ePRO are about on a par: both depend on what you know, although one could argue that a handheld device in a patient’s pocket is going to be somewhat more secure than the family PC with little Johnnie the nascent computer whiz around. Legible. Both systems shine over their low-tech predecessors here: unruly handwriting with illegible margin notes simply don’t exist anymore. Contemporaneous. Here we hit the first snag for EDC-based PRO systems. The Oxford English dictionary defines the word as “an adjective, meaning existing at or occurring in the same period of time.”[4] Now, the Internet may be becoming increasingly ubiquitous, but very few people have access to it every minute of the day. What that means is that access to a PC to record, say, medication adverse events, or ability to undertake light exercise (e.g. walking) will necessarily take place after the event and thus not, in fact meet the definition of contemporaneous. Dedicated ePRO devices by contrast, are designed to be portable, pocket sized devices that are highly likely to be on the person of the patient at the time of any event-and that is more important than it might otherwise appear, as we’ll see in the discussion on accuracy. Original. Both Web-based EDC and dedicated ePRO are on equal footing here and require no further discussion. Accurate. Herein lies the problem, or at least one of them, for EDC-based PRO systems. Most people when they consider this element think of automatic edit checks and other programming techniques to reduce erroneous data capture. These elements are important to be sure, but address accuracy of input against other criteria, not the input in and of itself. To examine the problem we must use research from a variety of disciplines, and as we are examining PATIENT (not robot) reported outcomes, it is apropos that the behavioral sciences are considered. To elaborate, if we accept as a premise that people walking around with full blown PC’s in their pockets (or perhaps in their briefcases) are in the minority of the population, then the data will necessarily NOT be contemporaneous-here’s why the “C” part of the acronym is so important. People have poor memories. Behavioral science tells us that even short term memories in persons with no signs of dementia (that’d be you, and hopefully me) are subject to all sorts of inaccurate reconstruction after the event. To quote McKenzie’s 2004 work on the topic with respect to ePRO: “Memory relies on a variety of mental shortcuts, or heuristic strategies, to reconstruct past events. This retrospective reconstruction is imperfect and vulnerable to a range of biases.”[5] Other work has demonstrated McKenzie’s assertion to be well-founded, and that recall biases, that is, erroneous recollection of events after the fact, can and do skew data that may be critical to the clinical trial/study involved.[6,7] Thus, if your end goal is accurate data, collection must necessarily be contemporaneous to be accurate. If the “soft” sciences aren’t your cup of tea, just ask any police officer or public prosecutor about the reliability of eyewitness testimony sometime. One could advance the argument that patients could access the Internet via mobile devices (cell phones primarily) and enter their PRO data in that fashion at time of event. Putting aside the issues of squeezing a screen designed for a 17 inch wide-screen LCD monitor onto a 3.5-inch mobile phone or similar, and that the fundamental navigation of Web-based EDC’s is designed for clinicians (if you’re lucky) or geeks (if you’re not), and not patients, mobile Internet access is surprisingly low at around 3.4% for Europe and 7.5% for the United States [8]-and that's for subscribers, not the general population. So mobile Internet access is unlikely to work either for a Web-based EDC PRO system. So, mea culpa: I should have emphasized this point more clearly, as m any other examples exist beyond the ePRO example -ad hoc reporting, financial management, resource allocation and utilization and more. One should always consider the desired end result in examining any technology, and closely examine how each individual element can best achieve that result: sometimes a specialist offering is superior to a generalist solution. Timothy Pratt, PhD, is a member of the Applied Clinical Trials Editorial Advisory Board, and is currently an independent clinical and business consultant in Minneapolis/St Paul MN. References 1. Stone A, Shiffman S, Schwartz J, Boderick J, Hufford M, "Subject Non-Compliance with Paper Diaries," British Medical Journal, 324, 1193-1194 (2002). 2. McKenzie S, Proving the eDiary Dividend, Applied Clinical Trials June 2004 3. FDA Guidance for Industry Computerized Systems Used in Clinical Investigations May 2007. See http://www.fda.gov/cder/guidance/7359fnl.htm 4. http://www.askoxford.com/concise_oed/contemporaneous?view=uk 5. McKenzie S, op. cit. 6. Redelmeier D, and Kahneman D, "Patients' Memories of Painful Medical Treatments: Real-Time and Retrospective Evaluations of Two Minimally Invasive Procedures," Pain, 66, 3-8 (1996). 7. Ross M, "Relation of Implicit Theories to the Construction of Personal Histories," Psychological Review, 96, 341-357 (1989). 8. http://www.comscore.com/press/release.asp?press=2345