Clinical outcomes assessments—often referred to as COAs or eCOAs to describe their electronic presentation—offer an additional view of a patient’s current health or change in health over time. The four categories of COAs frequently used in trials today are patient-reported outcomes (PROs), clinician-reported outcomes (ClinROs), observer-reported outcomes (ObsROs), and performance outcomes (PerfOs). Data generated from each type of eCOA allows for a complete understanding of the safety and efficacy of an investigational product in clinical development and postmarketing studies.

The following technology trends allow COAs a more streamlined data-collection process through digitizing their presentation and availability on tablets, smartphones, or other handheld devices, providing many benefits to clinical development without risking data integrity issues presented by differences in modality.

 can be expensive for sponsors to provision a device for trial participants. However, with the massive adoption of smartphones globally, study subjects can utilize their own devices through bring your own device (BYOD). Instead of being responsible for charging and carrying an additional device to upload their PRO responses, the patient enters their data into an app on a device they are already accustomed to using. While some risks are still associated with this process, such as the patient turning off eCOA notifications or updating their device operating system, BYOD has potential to significantly reduce eCOA spend on hardware and increase overall compliance.

, using audio and video data sources, promise to increase overall objectivity and data integrity. Being able to analyze patient speech or behavioral patterns captured in audio and video files allows researchers to document disease progression or improvement more accurately than possibly using eCOA scale data alone. Multimedia eCOA data is held to the same, if not more stringent, data integrity standards than typical clinical trial sources, as these eCOAs pose a more significant risk of revealing patient-identifying information. Audio and video files must undergo data-obscuring processes, such as encryption, to ensure subject anonymity—a step that must be accounted for when study timelines are projected.

 can be used gather a more objective understanding of an investigational product’s effect on patient health by collecting real-time health data through devices such as digital watches. This might include information on a patient’s sleep patterns, heart rate, and blood sugar levels. Wearable devices can also offer insight into a patient’s ability to complete a specific task, the performance of which may be hindered by their diagnosis. Much like implementing multimedia eCOAs, however, special considerations must be made to assess whether wearable technology used in a clinical trial is fit for purpose and meets regulatory and validation requirements.

Sponsors must consider whether the target patient population could efficiently operate the device and readily implement it into their lives for the duration of their time in the trial. There must also be an explicit agreement between sponsors and device vendors detailing the ownership and use of the patient data gathered.

 can allow timely nudges and notifications to remind patients to complete their daily diaries, which can also personalize their trial experience. To capture patients’ attention while guiding them to complete study procedures, ePROs can use gamification to turn the trial process into a “game” where participants must complete specific tasks or assess their current health to earn points. Gamification can also include “competing” anonymously with other subjects moving up the ranks by remaining compliant with their study activities.

 algorithms are being developed to analyze eCOA/ePRO data and identify trends, patterns, and outliers, which can provide valuable insights into treatment efficacy, patient behavior, and disease progression.

As a result of technological advancements, patient data acquisition has evolved, moving to encompass more data sources than ever before. However, these tools must be carefully considered as fit for purpose before implementation. Also, patient privacy and less burden must remain a top priority across the pharmaceutical industry.

Articles

A Closing Thought

Even with exciting advancements, stakeholders must remain diligent in choosing data platforms.

Modern Technology is Shaping the Face of COA

Arthritis is a general term categorized by acute or chronic inflammation of the joints. The condition may cause a variety of symptoms, most commonly pain, stiffness, decreased range of motion, and joint deformities. The pathology of arthritis is not limited to the joint itself and can extend to the surrounding connective tissues and structures throughout the body having a global impact on ones’ quality of life.

Clinical research in arthritis often includes clinical outcome assessments (COAs), such as patient-reported outcomes (PROs), clinician-reported outcomes (ClinROs) and observer-related outcomes (ObsROs). While objective measures are a necessity, the extent of the patient experience cannot be captured simply with a physical exam, lab tests, or imaging. Only including these types of assessments would provide a limited understanding of the overall condition, which could lead to reduced therapeutic advancements, treatments that do not fully manage patient symptoms, and restricted scope of research that is not targeted to patient needs. PROs allow patients to be active participants in the research conducted and contribute their insight as it relates to their health condition. Additionally, regulatory bodies are increasingly looking at COAs to guide patient-focused research and decision making.

Of the many PROs used in arthritis research, some questionnaires are centered around specific symptoms such as fatigue, pain, and perceived physical function. Others are more specific to the subset of arthritis being studied. Collecting and assessing PROs allows the full breadth of symptoms and disease impact to be quantified. Of most importance, PROs allow for research to focus on what matters most to patients with the condition being studied.

Fatigue is a common symptom expressed by patients diagnosed with arthritis often accompanied with an inflammatory disease process and/or chronic pain. The Functional Assessment of Chronic Illness Therapy-Fatigue Scale (FACIT-Fatigue) aims to assess patients’ self-report regarding their level of fatigue and the impact that their activities of daily living. This 13-item questionnaire is used in patients diagnosed with rheumatoid arthritis, psoriatic arthritis, and osteoarthritis, as well as a multitude of other chronic health conditions.

 The visual analog scale (VAS) offers a way for patients to quantify their pain using a horizontal line rating their pain from no pain on the left to the worst their pain can be on the right. This simple method is transferrable to different locations of pain and different disease types. For example, knee pain VAS may be used in a patient with osteoarthritis while back pain VAS categorizes the intensity of inflammatory back pain that can present in arthritis types such as juvenile idiopathic, reactive, and psoriatic arthritis. The VAS is often included under other PROs when measuring the extent of disease impact. For instance, the patient global assessment for psoriatic arthritis pain uses several VAS scales for musculoskeletal and dermatological manifestations of psoriatic arthritis.

Furthermore, research has shown a link between pain severity and mental health in patients with arthritis. High pain levels are correlated with increased depression and anxiety. Additionally, anxiety and depression can lower one’s threshold for pain, creating a cycle of poor health, pain, and a negative impact on mental health. By using PROs to examine pain, researchers can not only examine pain levels and how those levels change in response to treatment but can also use that information to inform care focusing on comprehensive management and overall well-being. 

 As indicated, a key focus of arthritis research is assessing patients’ perception of their own health and their quality of life. The Health Assessment Questionnaire-Disability Index (HAQ-DI) assesses activities such as walking, eating, dressing, grip, and reach to allow patients to explain how their disease is impacting activities of daily living. The Short Form-36 (SF-36) is another PRO that is widely used to analyze health-related quality of life. Both have been used in rheumatoid arthritis and a variety of other chronic health conditions to assess changes in quality of life. Use of PROs that analyze quality of life and how patients are experiencing their disease is vital to provide patients with a voice to explain their symptoms, as well as how their disease impacts their ability to live a fulfilled life.

Some PROs allow for a more comprehensive collection of data related to the patients’ disease activity. The Clinical Disease Activity Index (CDAI) is a composite index that assesses disease activity based on joint swelling, tenderness, and the patient and physician’s global assessment. The GRAppa Composite ScorE (GRACE) is another composite score specific to psoriatic arthritis that looks at joints, skin, function, pain, and quality of life. The data collected from multiple variables to form the composite score allows researchers to view different aspects of the disease objectively and subjectively, depending on the patients’ insight to evaluate their disease progression and response to treatment. Using these types of PROs across a trial can provide valuable information on symptom range and related trends.

 In arthritis and other inflammatory and immune-mediated conditions, symptoms can vary not only from patient to patient, but can change day to day in the same patient. In certain types of arthritis, there may be periods where symptoms are more intense, and times where symptoms are less noticeable. Due to this fluctuation, it can be difficult for patients to accurately recount their symptoms. A diary format takes this into consideration by allowing for data to be collected in real time. Many of the PROs referenced can be formatted as eDiaries to be accessed by patients remotely. For example, PROs focused on fatigue or pain can evaluate the change in these symptoms overtime, how well patients respond to intervention, and ultimately lead to more individualized care as these symptoms are tracked, assessed, and analyzed.

eCOA systems have been shown to enhance legibility, logic, attributability, accuracy, and timeliness of data capture, as well as allow for both on- and off-site data collection. Remote data collection can decrease geographic barriers and add flexibility for study participants. Symptoms of arthritis, such as pain, fatigue, and physical limitations, can be disruptive and difficult for patients to manage. As such, these symptoms can greatly interfere with patients’ quality of life and ability to perform activities of daily living, including traveling on-site for a study visit. In the event the patient is unable to come in for their visit due to these barriers, there is the adaptability for electronic data collection to be done remotely. This increases the likelihood of data to be collected within the visit window, prevents data loss, and supports patients continued participation in the trial.

Availability of complex assessments in a straightforward system eases the burden of data collection for sites and patients. ePROs can be programmed according to the needs of the trial per protocol, and of the individual patient. Some researchers may choose to study multiple disease subsets within the same trial design. In this case, the ePROs can be specified with programmed triggers to allow for completion of the measures depending on the patients’ pathology. For example, if a trial is to include both patients with rheumatoid and psoriatic arthritis and using the FACIT-Fatigue and GRACE questionnaires mentioned, the eCOA platform should be programmed such that only patients with psoriatic arthritis should be presented with the option to complete the GRACE. Patients with psoriatic arthritis and rheumatoid arthritis should both have the FACIT-Fatigue available for completion.

Certain PROs are used in arthritis research, such as the CDAI and other disease activity scores (DAS) that weigh factors such as patient feedback, lab results, and joint swelling and tenderness, include score calculations to measure disease activity and response. These scores can be calculated automatically in an electronic solution and be available in real time for sites to review and monitor the data.

Depending on the trial design, eDiaries may be utilized to capture data directly from patients. eDiaries can be programmed with patient engagement tools such as reminder alert systems and allow for data capture in real time. This is especially important in the arthritis types with patterns of flare-ups and remission to assure the diary data is being collected as timely and accurately as possible.

As symptoms change over time, patients may not accurately recall their experience later. The use of daily electronic PRO diaries in arthritis has been shown to reduce recall bias and improve the quality of PRO data collection. Electronic diaries and ePROs can be implemented according to a pre-defined schedule, with only the expected questionnaire opening at the determined time slot. For instance, if the recall period is one week prior to the next study visit, the eDiary can be programmed to only open seven days prior to the defined visit date. By programming the ePROs and eDiary so only the correct questionnaires are available at the correct time, there is a decreased risk of capturing unwanted data. 

The use of ePROs in arthritis involves careful consideration and appropriate management to ensure success throughout the duration of the trial. When implementing ePROs, the device design and interface should be user friendly, programmed as specified in the protocol, and the patient experience must be prioritized.

Arthritis management focuses on reducing pain, minimizing joint damage, and improving function and quality of life. Therefore, the ePRO design needs to support those goals. The ePROs should be concise and easy to use with minimal scrolling and typing required by the patient. Features such as interview mode can be added to allow for ePRO completion through dictation, which can minimize joint strain.

Best practice includes user acceptance testing (UAT) for sponsors to review that the ePROs meet the needs of the specific trial. Study teams, clinical sites, and patients must be properly trained in using the technology and training materials should be readily available to refer to as needed. Site teams should be familiar with assistive devices commonly used and able to support the patients’ use of these tools when using a provisioned device or smartphone. Site teams must also be trained in the full capabilities of the ePROs that have been designed specifically for this patient population.

Examples of training options include investigator meeting presentations and demonstrations for clinical staff, online training using material provided by the eCOA solution providers, and videos to be reviewed by relevant staff. It is recommended that refresher trainings are conducted, as recruitment can occur several weeks after the initial training sessions or site staff can change during the course of the study.

Project management teams with clinical experience and eCOA expertise can support the implementation, training, and management of these ePROs. Clinical project managers successfully meet client needs due to their familiarity with medical terminology and arthritis pathology, research requirements, and protocol adherence. These teams are necessary to assess, mitigate, and manage trial-related risks, anticipate challenges sites and patients may face throughout the trial, and provide guidance and solutions.

Clinical project managers work directly with sites to provide strategies to support arthritis patients, such as reinforcing options for remote data collection or interview mode and listening to patient feedback. By having a direct link to the sites, project management teams can understand the arthritis patients’ experience and are able to develop specific strategies to support and improve their experience. These strategies are aimed to support the goals of arthritis treatment by improving functionality of the ePROs to reduce patient stress and minimize any impact on the patients’ joints. Clinical project managers are also essential in assuring timelines are met and support in inspection readiness and in reaching key trial milestones.

Data management activities must be conducted throughout the trial to assure data quality. Full audit trails should be available for review, relevant edit checks implemented, and the data cleaned to account for any inconsistencies. As the data from ePROs and eDiaries is becoming more relevant for clinical trials in arthritis, and even becoming a primary endpoint in a rising number of trials, it is vital the data is comprehensive and its integrity and quality are maintained. In an electronic portal, the data can be easily viewed to support clinical trial monitoring and query management. The electronic portal should also be able to integrate with interactive response technology (IRT) systems for efficiency, decreased costs, and increased data accuracy.

Global research in arthritis continues to focus on developing trials to provide new drugs and advancements to current treatment options. PROs are increasingly being reviewed to determine efficacy and exploratory outcomes within the research context.

However, the content of PROs used in both the clinical and research setting must be continually reviewed and improved upon. New PROs must be developed to ensure they are asking the right questions and addressing topics that matter most to patients with arthritis. Implementing new PROs is a lengthy process, but certain initiatives, such as the NIH funded Patient-Reported Outcomes Measurement System (PROMIS), are actively working to develop and validate PROs for research and clinical practice.

Technology must also advance alongside these developments by continuing to be patient-focused. This includes ongoing integration of ePROs and eDiaries in clinical protocols and the addition of other functions, such as interview mode and televisit capabilities, that will reduce the burden and, as such, improve the patient’s trial journey. Technology adapted to focus on patient engagement, support study teams, and provide quality data will lead to more efficient patient-centered research, better outcomes, and improved care for arthritis patients.

Feature Articles

The value of electronic patient-reported outcomes data in this disease setting.

Implementing ePROs in Arthritis Clinical Research

Earlier this year, a panel consisting of leaders from the clinical research industry convened for a virtual roundtable to discuss the emerging uses of real-world data (RWD) drug development.

The group discussed a variety of topics, including patient centricity, regulatory concerns, underrepresented populations in clinical trials, and their own experiences with using and/or managing RWD.

What is the top RWD use you have seen in practice in clinical trials this past year?

 From my experience, the use of RWD has been very important and more popular now in two areas.

The first one is the use in external control arms; the ability to use RWD to accelerate the clinical trial process using medical care data or data coming from the real world to be able to simulate the control arm.

And the second one, which is very exciting—but not a lot of people are talking about—is all the new software and algorithms coming out for biosimulation. What we use to train these systems and algorithms is RWD. It is electronic health record (EHR) data. And that allows the development of systems and algorithms that can help develop clinical trials or biosimulation for PK/PD (pharmacokinetics/pharmacodynamics). So that’s exciting, and I see more and more of that in the market right now.

Amy Cramer, Steering Committee Board Co-chair, HL7 FHIR Accelerator Vulcan:

 What I’m really excited about is an organization called Vulcan. It’s the HL7 FHIR (Fast Healthcare Interoperability Resources) Accelerator dedicated to both translational and clinical research. Vulcan is made up of 40-plus international members representing various stakeholders from the research community, including patients, sites, sponsors, regulators, vendors, and more. In order for RWD to be functional, there must be consistent use of standards because data mapping and harmonization is not scalable. Vulcan’s members and collaboration with other organizations is exciting to me because standardization of data is how we’re going to optimize efficiencies out of all the data that’s available.

Jill Pellegrino, Vice President of Recruitment and Real-World Evidence (RWE), CVS Health Clinical Trial Services: 

At CVS, we have access to millions of lives in our data, so access to pharmacy records and claims information on these individuals. And we’ve been using that data to reach out to the patients to invite them to participate directly into research and to trials. That’s been powerful because a lot of what I see in RWE and use of it is taking data sets from different sources and bringing them together to glean insights from it. But we’ve seen a lot of success in engaging the patient directly as a participant in that. It allows us to collect more information and then also to bring their voice into the research, which has been powerful. Further, it’s allowed us to bring people to trials and research studies that they wouldn’t have had access to before and help with enrollment challenges that we see in the industry.

Craig Serra, Head, Commercial Solutions and Partnerships (clinical research), Flatiron Health:

 The top use case that I’ve seen over the last year is figuring out what kind of patients those sites have historically seen and if there are current patients eligible for a trial at those sites. Then, when you do that last mile connection, you’re starting to see magic happen between patients and sites within the clinical research world.

Alecia Clary, Founder and CEO, Evidence to Practice: 

I’m most excited about watching people embrace remote trials, hub and spoke trials, and using RWD to identify not only where patients are but who are the patient populations that most need the therapeutic or the tool, and making sure that recruitment efforts align with those populations. This includes efforts to incorporate patients and data from community centers and other places where they patients may not traditionally have an opportunity to participate in trials.

What challenges and/or successes have you seen in some of these RWD practices?

 From a pure RWD perspective, the timeliness of the data and the contemporaneous nature of it, are we getting to patients soon enough? Do we have enough “real-time data?”

We have a lot of partnerships with patient advocacy groups, who are really the ones helping to build that trust with the patient. But sometimes it’s not just the method of educating them on the importance to be part of the study. It’s educating them on how to get access on their data. For example, at Alira we believe that the patient is the carrier of the longitudinal information. Why don’t we focus on the patient and not on the provider?

Right now there is a new regulation from the [21st Century] Cures Act that patients can have access to their electronic medical records (EMRs), but not everybody understands what an electronic medical record is. Not everybody knows how to go on a patient portal and download it. Not everybody is willing to acquire it.

Also, EMRs are not available to everyone because many patient portals have not yet been updated to allow downloading of [electronic health records] and EMRs.

There’s also a knowledge barrier of where can I get my information, and how can I download my information? Now hopefully that will change in the next couple of years. But it’s a process for them; it’s a process for us, but an exciting one because the changes in the regulations are allowing patients to be able to get access to that very valuable information.

One of the challenges that we’re working on is how to get more in-depth or robust data in populations with more complex disease states. Areas like oncology and immunology, you need a lot more data where you can reach out to patients and make sure the trial is relevant for them at that point in time.

 One of the most pressing challenges for the work that I’m doing right now, is missing data.

We’ve found that, for example, race and ethnicity data are missing in 30% to 70% of clinical RWD sets. So, if you’re interested in doing a targeted outreach to a potential patient pool, there may be several patients that you’re missing. Also, are patients for whom race and ethnicity data are missing systematically different from patients who are willing to report that data, for example?

The only people who show up in these clinical data sets are people that have access to care at these sites. There are large amounts of patients who are probably not represented in these data sets and could benefit from and be interested in participating in clinical trials or having their data leveraged for the generation of RWE.

Global differences in data collection, including variability in the documentation of data domains and some data coming from paper sources, as well as the inability or inconsistency of data interoperability are challenges to optimizing RWD. Country-specific and regional variances in regulations, data access, patient trust, and more creates different operating landscapes that are not always easily comparable. To enable more effective data comparisons and integration globally, we need to establish and adhere to a single set of standards across both clinical care and
clinical research.

How does RWD help advance approaches in patient centricity?

 Using RWD really does allow more people to benefit from participating in research than people who typically participate in prospective clinical trials. And so, consequently, they also get to participate in the benefits of research.

Postmarketing safety studies are a well-established and robust example of how real-world data can facilitate our ability to understand the real-world performance of therapeutics for people, again, who have access to them. But, as several of my colleagues have mentioned, there really is an opportunity to improve our ability to link data sets; increasing interoperability is really an opportunity for us to not only link some of our clinical data sets but also to go beyond that and link more nontraditional data sets to begin to incorporate additional data about these patients.

When I think about patient centricity in clinical research, I’m thinking about studies designed and executed in a way where the sample represents the population with a particular disease. RWD is tremendously effective at helping us accomplish that, and inherently when a population uses that drug, you have exponentially more confidence that your safety and efficacy evidence is the highest quality possible. I also think about patients who may be eligible and want to be in a study, but the trial burden is too much; RWD is able to help that aspect tremendously as well.

I found incorporating patient centricity is very hard to do on a global or protocol level. It’s best to make the experience and the options for patients most flexible. At CVS, we are bringing trial sites to our MinuteClinic footprint across the US. And the strategy and goal there is to provide an environment and an opportunity for more patients to participate in trials in a way that’s more flexible and convenient for them. In the data we’re collecting at the site level, there are insights that we can act on at that local level to help improve patient experience and increase the level of engagement.

 The promise of RWD allows for flexibility in study design such as decentralized clinical trials, local care models, or claims/clinical data in an interoperable way. When we look at protocol development, there must be options for patients in how they want to participate in clinical research. Some people enjoy coming into the site and having that interaction. But where is the site? Are participants willing to travel a long distance, or do they want to travel to a closer location?

If we’re really going to be patient-centric, we have to give flexibility at the level of patient interaction. It’s going to make it more nuanced, yes. But I think in the end, this approach will allow sponsors to provide a better patient experience, the study will be more inclusive because we can offer access to patients from underrepresented communities, and we’ll likely help boost retention in the study. RWD is an enabler to allow that flexibility.

That also brings to mind opportunities for remote patient monitoring, for example, using some digital tools. There are opportunities to collect some data from the patient’s home so that they don’t have to travel to the trial site no matter how far it is.

What kind of information can I give back? What type of technology can I give to them (patients) that will help them manage their condition while they’re going through a collection or they’re providing data for clinical research? That is the right approach because if you use them only as, “Oh I need them to collect data,” we’re not going to get their compliance. So thinking about what can you deploy as a technology for them that can help them also manage that period of their life, their emotions, their conditions, that’s the key to get compliance, to get participation. That’s what companies should focus on.

How does RWD help improve clinical trials among underrepresented populations?

 The goal at CVS was to bring trials to underrepresented populations that don’t always get a chance to participate. We rely on RWD to inform that strategy in terms of the locations and patients with disease areas that would benefit from trials.

With the advantage of having an oncology-specific EHR embedded in hundreds of community practices, of which there’s a good portion of them that also practice research, we’re simply able to get to where patients actually get care. And many of these patients simply can’t make the trek to a big-named university, academic medical center, or large hospital system that’s a couple hours from them, despite being diagnosed with cancer. Logistics aside, our historical study design and execution ecosystem vastly overestimates capabilities and knowledge of clinical research that your average patient has. With RWD, we meet patients where they are at.

What are the steps we need to take to unify data sets?

 I believe there are three main components. One, we need to agree on standards because right now clinical research and clinical care don’t typically use the same standards. Second, we need to reduce variability in the way that data is collected—semantic interoperability. What I mean is when we collect data, we need to have the same definition of what we’re collecting. What does a rescue drug mean when I’m a nurse as opposed to what does a rescue drug mean when I’m in a clinical trial? We need alignment, and we need to make sure that we agree on the way data is collected. How many years have you been a diabetic vs. what was the year that you became a diabetic? It’s a small thing, but it makes a big difference in the end. We also need to collect data at the point of care so we’re not requiring another step for patients to complete.

Lastly, we must enable multidirectional data flow. Data should come from the physician to the [principal investigator], to the sponsor, to the patient, and to the regulator—flowing in multiple directions in near real-time. Imagine, a triangle with the site, sponsor, and patient at each corner with data flowing in multiple directions and regulators positioned in the center of the triangle.

The subtext to the conversation about linking data sets and interoperability is also the data holder’s knowledge or interpretation about existing laws and regulations governing it. My experience with multi-sponsored studies suggests that data partners may participate in the same study but interpret laws and regulations differently even when they’re operating within the same jurisdiction. While we’re aligning on definitions and the data elements that we’re collecting, we also need to align on how we interpret laws and regulations that governs data sharing.

What do you think is lacking to further industry adoption of RWD?

I think it’s just time and change management. We have the pieces of the puzzle, and what isn’t mature yet is being developed. We have stakeholders across the research community who are interested in making RWD the industry norm. What I think is really interesting and exciting is that we’re taking learnings and best practices from recent experiences like the COVID-19 pandemic and driving toward a standards-based interoperable exchange of RWD. The next step is change management and incentives to drive adoption.

 I share Amy’s optimism and enthusiasm and if we can keep that same energy up around the use of RWD as a whole and clinical research as a whole—having more cross-industry conversation about what works and what doesn’t work; having conversations about what is acceptable; and where we have some work to do in terms of acceptability—there are a large amount of opportunities for stakeholders.

I think we’ve seen a huge increase in pharmaceutical companies that are trying to use real-world data to make clinical trials more efficient to capture greater insights. There’s a lot of momentum and motivation to do this. It’s just a matter of working through the challenges and trying to navigate the data options that are available and finding sufficient data to do those analyses.

Right now in pharma, the amount of money spent in real-world evidence research matched the actual amount of money spent in clinical research in 2021, which is great. The amount of money spent on real-world evidence will be more than the amount spent in clinical research. The importance of proven effectiveness and not just safety and efficacy, that has become a very important point through COVID. I share the positivity and the trend is there. You see pharma companies are spending a lot more money in this particular discipline. That’s a key factor for sure.

Senior leaders gather to share insights on the increased investment in real-world data to boost the quality and efficiency of drug development—as the industry begins to more widely embrace evidence of proven outcomes in aiding approaches in clinical trial design, patient engagement, regulatory decision-making, and other areas.

Executive Roundtable: The Rise of RWD in Clinical Research

Recent FDA draft guidance on core patient-reported outcomes (PROs) in oncology clinical trials

 lays out a framework for effective patient-reported outcome measure (PROM) selection and implementation. This may be important in increasing the use of PRO data in oncology medication labeling in the US, where, historically, labeling claims based on PRO data have rarely been included. The reporting of PROM endpoints on drug labeling is valuable to both the patient and the prescribing physician in informing risk/benefit discussions beyond tumor response and survival endpoints. The lack of PRO-related labeling is clearly illustrated by the fact that only three out of 85 drug approvals in oncology submitted to FDA between 2010 and 2016 included PROM-related labeling claims.

While this may be related to trial design limitations (e.g., single-arm and open-label studies), it may also reflect weaknesses in clinical outcome assessment (COA) measurement strategies typically used in oncology research.

FDA’s draft guidance addresses two areas of potential weakness in current COA measurement strategies for oncology trials.

In oncology trials, PROMs are often implemented at clinic visits just ahead of starting each cycle of treatment. While administratively convenient, this is the time at which patients are feeling well enough to receive the next cycle of treatment, so this strategy may fail to measure the impact of treatment in the earlier stages of each cycle when treatment-related side effects are likely to be most pronounced. Just measuring at cycle start may provide a biased view of treatment toxicity.

The draft guidance stresses the importance of selecting measures that are highly specific so that treatment effects can be understood more granularly. FDA identifies five core measurement domains of interest: disease-related symptoms, symptomatic adverse events, an overall side-effect impact measure (single item), physical function (aspects such as walking, lifting, and reaching that are considered important for independent functioning), and role function (impact of a treatment on the ability to work and carry out daily activities). Not all commonly used instruments provide scores or subscores that provide the level of specificity required to separately assess these core areas.

In this article, we explore these two areas in more detail.

The draft guidance recommends more frequent measurement during initial cycles, with fewer later on (see Figure 1 below). In particular, FDA indicates that symptomatic adverse events, the overall impact of side effects, and physical function should be measured frequently at the start of treatment, before moving to less frequent cadences later in the treatment and subsequent follow-up periods. This schedule helps to mitigate chances of missing the incidence and severity of items associated with these domains, as may be the case if measuring only at the start of new treatment cycles.

This increased frequency of assessments raises several practical questions:

If these domains are measured by subscales of existing instruments, is it valid to use only the subscale items at these more frequent timepoints?

To enable this more frequent measurement schedule, patients would need to complete PROMs at home. Is this practical when patients are experiencing challenging treatment-related symptoms and side effects?

Should measurement timings be adjusted if a treatment cycle is delayed due to side effects?

It would be burdensome for the patient if we implemented the full 30-item EORTC QLQ-C30 questionnaire (a frequently used general oncology PROM) on a weekly basis when we only need to measure the physical function subscale (five items). However, discarding the other items within an instrument might impact the psychometric properties of the subscale, so it will always be important to verify that this approach is acceptable and valid with the instrument owner.

In the case of the QLQ-C30, the subscales are valid for independent use as discrete “item lists.” Therefore, referring to the schedule of assessments in Figure 1, the five physical function items from the QLQ-C30 could be applied independently on a weekly basis to enable more frequent assessment of this core domain. When doing so, it will be important to include the complete set of physical function subscale items, represent them in the same order as within the full instrument, and apply the same scoring and missing data rules as defined for the subscale within the full instrument.

methods to lessen patient burden should be explored, including use of electronic PRO capture that may allow for assessments outside of the clinic,”the more frequent assessment schedule may be perceived as burdensome when the effects of treatment are particularly compromising. Could this lead to increased missing assessments? In a qualitative interview study of oncology patients, it was identified that certain days within typical treatment cycles are the most difficult, with one patient saying: “Days three to six were my ‘dark’ days, and I did not leave my bed, speak to anyone, or even eat any food.”

Despite this, participants in this qualitative study did not feel that this would deter or inhibit them from completing PROM instruments on electronic devices. For example, one patient stated, “You can be bad, but not so bad that you can’t use nothing at all...I would make the effort.”

Being sensitive to this is important, and we should consider guiding principles for ePRO implementation to limit burden and aid completion rates, including:

Ensure instruments are as short as possible to measure the domains of interest. Computerized adaptive tests may provide an approach to minimize the number of questions.

Implement multi-day completion windows, so that patients can elect to complete assessments on a subsequent day if they do not feel able to complete their PROM immediately.

If implementing more than one PROM, enable patients to come back later to complete the second instrument.

Consider a bring-your-own-device (BYOD) option. On days when the patient is feeling unwell, they may be interacting with their own mobile device but are less likely to have the trial device on hand.

Collect reasons for missing assessments so that intercurrent events, such as feeling too unwell to complete the PROMs at any given time, can be taken into account in the analysis plan, the interpretation of the results, and the optimization of future trials.

Implement PROM completion compliance strategies, such as patient alarms/reminders, site notifications, and reports, to monitor and proactively encourage completion behavior.

In oncology trials, it is common for physicians to implement delays in treatment when a patient has not recovered sufficiently from the previous cycle of treatment. When collecting PROMs at the site, this typically means postponing PROM completion to coincide with the new cycle start date. However, with the change in focus to more frequent measurement, we recommend that the trial objectives will most likely be met when the measurement schedule is not adjusted to allow for changes in the start times of subsequent treatment cycles, especially when the period of frequent measurement is long enough to comfortably cover several cycles. 

Separate and independent assessments of each of the five core domains requires careful appraisal and selection of the measures to be used. To measure each domain using a single scale, or a subscale of a broader instrument, questions should all be related to the domain and comprehensive enough to encompass all the meaningful elements of that domain.

While this sounds straightforward, nuances in the definitions of domains measured by different PROMs mean that researchers should carefully inspect instruments to ensure the FDA-defined domains are accounted for. Let’s consider two examples.

Example 1. NSCLC-SAQ (Critical Path Institute, Tucson, AZ)

The NSCLC-SAQ is a seven-item symptom assessment questionnaire for non-small cell lung cancer (NSCLC)

, developed specifically to assess disease-related symptoms across five concept areas (see Figure 2 below).

The qualitative work used to identify the set of disease-related symptoms that are meaningful to NSCLC patients

 provides strong evidence of the measure’s relevance and content validity. The measure only focuses on disease-related symptoms, so in addition to content validity, it also satisfies the requirement for specificity. For this reason, it is not surprising that this instrument was cited as an example of a suitable instrument to measure disease-related symptoms in the FDA draft guidance.

Example 2. FACT-B, Breast Cancer (FACIT Group, Ponte Vedra Beach, FL)

The Functional Assessment of Cancer Therapy - Breast (FACT-B) is a 37-item questionnaire that is used to measure five domains of health-related quality of life in breast cancer patients: physical well-being, social/family well-being, emotional well-being, functional well-being, and a breast cancer subscale.

 The naming of these domains differs from FDA core domains, and they represent slightly different concepts. For example, physical well-being is not the same as physical function.

If we examine the subdomain of functional well-being, we find that the first three items of the subscale measure one’s ability to work and whether that work is fulfilling, as well as one’s ability to enjoy life. Some of these map to FDA-defined domains—for example, ability to work may be a component of a measure of role function. However, the ways that the current subscales and their scoring are defined do not facilitate specific measures of one or more core domains as identified by FDA.

However, the FACT instruments are well validated and widely used. With that in mind, it may be possible and desirable to reorganize their subscale structure to measure FDA-defined core domains. This takes us to considerations around instrument adaptation.

As described, to meet FDA draft guidance around specificity to measure the five core domain areas, it is important to select PROMs carefully. In some cases, it may not be as simple as implementing an instrument in its off-the-shelf format. Instead, it may require some degree of adaptation. We discuss two examples ahead.

As described, while the subscale construction of the FACT-B does not appear to map directly to the FDA’s core domains, the instrument and its items are well-validated. To satisfy the draft FDA guidance, researchers may consider working with the scale author to validate an alternative subscale based on existing items within the instrument. As a hypothetical example, we might consider developing a specific measure of disease-related symptoms by grouping items from the physical function subdomain (e.g., lack of energy, pain, and nausea) along with items from other subdomains (e.g., shortness of breath, swelling/tenderness of the arms). This would require working with the scale author and evaluating the psychometrics of the new subscale.

Adding to existing instruments using item banks

Item banks are databases of individual validated PROM items. It is possible to use items banks to assemble tailored PROMs more rapidly for specific populations and measurement domains. Most applicable to oncology research are the PROMIS (Northwestern University, IL) and EORTC item banks. The latter, for example, contains around 1,000 items, with many translated in up to 100 languages.

A good illustration of this approach is the use of the EORTC item bank to supplement items in the EORTC’s core instrument (QLQ-C30), to ensure full coverage of all important items in order to measure drug benefit in a set of rare hematological stem cell disorders.

 Instrument developers identified the symptoms and impacts important to patients with these diseases, and used the EORTC item bank to map these to concepts that were not contained within the QLQ-C30. They conducted further testing in patients to confirm the suitability and content validity of these additional selected items. We may see this type of approach more often as we carefully and strategically consider PROM selection for future oncology trials in light of the FDA draft guidance.

Core Patient-Reported Outcomes in Cancer Clinical Trials 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/core-patient-reported-outcomes-cancer-clinical-trials

Gnanasakthy, A.; DeMuro, C.; Clark, M.; et al. Patient-reported Outcomes Labeling for Products Approved by the Office of Hematology and Oncology Products of the US Food and Drug Administration (2010-2014). 

https://pubmed.ncbi.nlm.nih.gov/27069082/

Gnanasakthy, A.; Barrett, A.; Evans, E; et al. A Review of Patient-reported Outcomes Labeling for Oncology Drugs Approved by the FDA and the EMA (2012-2016). 

https://pubmed.ncbi.nlm.nih.gov/30711065/

Mowlem, F.D.; Sanderson, B.; Platko J.V.; et al. Optimizing Electronic Capture of Patient-reported Outcome Measures in Oncology Clinical Trials: Lessons Learned from a Qualitative Study. 

https://pubmed.ncbi.nlm.nih.gov/33274651/

McCarrier, K.P.; Atkinson, T.M.; DeBusk, K.P.; et al. Qualitative Development and Content Validity of the Non-small Cell Lung Cancer Symptom Assessment Questionnaire (NSCLC-SAQ), a Patient-reported Outcome Instrument.

https://pubmed.ncbi.nlm.nih.gov/27041408/

Bushnell, D.M.; Atkinson, T.M.; McCarrier, K.P.; et al. Non-Small Cell Lung Cancer Symptom Assessment Questionnaire: Psychometric Performance and Regulatory Qualification of a Novel Patient-Reported Symptom Measure. 

https://pubmed.ncbi.nlm.nih.gov/34567289/

Functional Assessment of Cancer Therapy - Breast. FACIT. 

EORTC Quality of Life Group Item Library User Guidelines, First Edition. 2022, 

Bell, J.A.; Galaznik, A.; Pompilus, F.; et al. A Pragmatic Patient-reported Outcome Strategy for Rare Disease Clinical Trials: Application of the EORTC Item Library to Myelodysplastic Syndromes, Chronic Myelomonocytic Leukemia, and Acute Myeloid Leukemia. 

https://pubmed.ncbi.nlm.nih.gov/31218454/

Examining the two areas of weakness cited in FDA draft guidance.

Optimal COA Measurement Strategy in Modern Oncology Trials

Rhoda Au remembers well when her interest in using digital biomarkers for disease research was not shared among her peers. The Boston University anatomy and neurology professor recalled when she delivered a keynote speech at a conference, about seven years ago.

“I had [finished] my digital spiel,” says Au, who is also director of neuropsychology, Framingham Heart Study, at Boston University. “Someone stood up and said that everything I said was all nonsense.”

It is unlikely that anyone today would express such a sentiment, at least not as publicly and rudely as that. In the years since, some smart people have hypothesized, and shown, that digital biomarkers can fill in some blanks about disease progression, symptoms, and therapeutic effects of investigational molecules. The interest is strong in using these biomarkers with the latter.

“In general, digital biomarkers will teach early signs and symptoms we don’t know yet about diseases,” Antonella Santuccione Chadha, MD, chief medical officer, Altoida, tells 

. “In early diagnosis, in disease monitoring, in treatment response, and in active populations, they will allow us to learn” and characterize even new disease phases. Altoida has designed an augmented reality platform that measures functional and cognitive brain health.

From a business vantage point, says Chris Benko, CEO, Koneksa, the ability to know steps in disease progression, both physiological and behavioral, and to predict treatment response and dose selection can reduce the number of patients needed for a trial. “Unquestionably, when you get to a point where you can get a signal in half as many patients, the economic value is just staggering.” Koneksa develops digital biomarkers for monitoring, diagnostics, treatment responses, and clinical event prediction.

Traditional blood or cerebrospinal fluid (CSF)-extracted biomarkers show point in time but digital biomarkers can show a patient’s response to a cognition test multiple times a day, every day. It is sussing out the pathway to the response that is important, says Au, who added that digital data will forever be available, whereas blood must be used judiciously.

The data from a digital biomarker can be gathered with various devices, including smartphones and smart watches. There are also platforms for various diseases; Altoida has a platform for Alzheimer’s disease, and Koneksa is developing digital biomarkers for Alzheimer’s and Parkinson’s disease.

Many people in the digital biomarker world are primarily interested in identifying those signs and symptoms of disease before actual onset and the inevitable deterioration they cause. About 80% of all diseases in the US are chronic, notes Au. Such an illness has been “coming for a long time before anyone said there is a problem.”

Today, adds Jason Hassenstab, PhD, associate professor of neurology and of psychological and brain sciences, and director, Cognitive Technology Research Laboratory, Washington University in St. Louis, people understand that decline from neurological disease is not simple or linear. In early disease stages, the brain can reorganize itself a bit, allowing for the patient to cope for some time.

“We have a cognitive reserve, rerouting our skills and wiring in network, to compensate for the cellular level changes,” says Hassenstab, who also leads the Cognition Cores for the Dominantly-Inherited Alzheimer Network-Trials Unit (DIAN-TU) and the DIAN observational study. He and his team designed a smartphone app for use by these patients a few years ago. “We know so much more about cellular level changes,” notes Hassenstab. “It was just a matter of time to find behavioral measures to detect.”

What researchers have long hoped, he adds, is to find changes in the brain as early as possible. “I think digital biomarkers have their place as outcome measures,” he says. “It will be difficult to approve a drug for [Alzheimer’s] without testing behavior.”

, “While diagnosis, disease monitoring, and intervention are traditionally addressed using distinctly separate tools, these different use cases may be overlapping or integrated in single solutions in the digital brain health space.” Sleep trackers, they wrote, can monitor quality of sleep; proffer advice for improving it; and possibly detect changes that are early neurological-disease signs.

The tools of the trade are being used on measures such as voice, temperature, activity, gait, blood oxygen, heart rate, and touch. These are but a handful of examples of what researchers look for in biomarkers.

 But there are literally hundreds more, depending on the disease.

For instance, Alzheimer’s. Published data show that the Altoida platform, which incorporates augmented reality and artificial intelligence, captures 800-plus multimodal features of the patient, through many different sensors.

 These features are then tested against fluid-based biomarkers. Eisai is testing the Altoida platform in a five-year trial with 3,500 patients to diagnose neurological diseases in their earliest stages.

Au’s research is looking at voice as one potential biomarker in particular. As a Framingham researcher, Au started recording participants’ responses to neurological testing beginning in 2005; hand-written notes didn’t provide the objective consistency she wanted, nor would they provide the informational pathways that the participants used to arrive at those answers.

By 2017, the Au team had 5,200 recorded sessions. A subset of 146 participants’ recordings were used from those that showed normal cognition, mild cognitive impairment, or dementia. A panel of neurology experts verified each case via diagnostic process.

 The researchers looked for pauses, hesitations, word-finding difficulties, circumlocution, pitch, timbre, and more. “When you are trying to think about what to say, you will” pause, stutter, or repeat words, says Au. These are all indicators of the person’s cognitive state, she adds.

According to Benko, at Koneksa, the company spent five years developing its Parkinson’s platform with partner Sanofi. In November 2021, the pharma giant extended the collaboration with Koneksa into central nervous system trials, which includes the evaluation of tolebrutinib a novel Bruton’s tyrosine kinase inhibitor for multiple sclerosis.

 He declined to discuss current findings, but demonstrated, via Zoom, how one of the measurements worked. He clicked on “ready.”

“‘With the phone in your right hand,” a voice said, “‘turn your palm up and down as many times as you can for 20 seconds.’” The platform, says Benko, “can measure tremor better than my eyeball.” He adds that Koneksa tested the algorithm in healthy people and then in those with Parkinson’s.

Koneksa is also looking at voice as a biomarker in Parkinson’s, and is recruiting for an asthma trial, comparing at-home mobile spirometry to in-clinic spirometry in participants with moderate asthma who take a long-acting beta agonist.

Not all attempts to find biomarkers are rooted in a smartphone. In order to find early detection of Alzheimer’s and related dementias, the National Institute of Aging and Indiana University are using a machine learning algorithm, embedded in electronic health record (EHR) systems at two sites, to monitor 4,000 Medicare patients using either the patients’ annual wellness visit by itself; the wellness visit with the digital marker; or the digital marker with the Quick Dementia Rating scale.

Not all studies are focused on myriad physiologic and/or cognitive issues. A biomarker of major interest is gait. Mobilise-D is a large 34-partner, 10-country, 2,400-person trial in the planning stages that will look at Parkinson’s; chronic obstructive pulmonary disease; multiple sclerosis; and fall-related issues, diseases, and frailty in the real world.

Other researchers are investigating how digital biomarkers can help with other serious conditions, such as death by suicide and alcoholism. In the latter work, which is testing whether problem drinkers can disassociate with Pavlovian cues, participants will use a smartphone, equipped with a GPS, every day to report on their drinking, activities, and moods.

Evidation is a consumer platform, five-million app-downloads strong, that allows people to get a new view of their health situation. “People can tell us what conditions they have, and whether they are using a smartphone, Fitbit, or CGM (continuous glucose monitor),” says Ernesto Ramirez, Evidation’s chief data scientist. “We use that data to help contextualize their health in everyday life.”

In clinical trials. Evidation has been involved in numerous studies, including the Boston University voice trial study. “We’ve been supporting trials for a long time,” notes Ramirez. Evidation says it has users in 97% of US zip codes and has enrolled over 600,000 participants in sponsored research, and with remarkable speed: The company cited a cardiology study involving US Veterans that attracted 800 veterans in 38 days. 

Ramirez discussed work with a current partner, Johnson & Johnson/Janssen, in a heart study with 43,244 patients.

 (Apple is also a partner.) This three-year observational study will enroll the patients, via the app, into a heart healthy program. The study will keep tabs on patients already prescribed an oral anticoagulant for atrial fibrillation (AF), see how adherent they are, record if and when they are diagnosed with AF, and then look at the percentage of days the medication was taken, minus any time it was not. The controls will have no history of AF, but will be enrolled in the same heart healthy program. There is an ECG sensor in the watch.

One person who sees what’s new in the field is Roozbeh Ghaffari, PhD, editor of 

; biomedical engineer and neuroscientist, and a research associate professor, Biomedical Engineering at Northwestern University; and director of the university’s Translational Research, the Querrey Simpson Institute for Bioelectronics. In the four-plus years that he has been in this role, Ghaffari has seen a few more submissions than when he started.

There are papers whose authors used devices in the course of the study; there are others submitting work on proposed novel biomarkers and biosensors for use remotely. Still others are “more holistic types of studies,” looking to determine the quantity of data needed to achieve an outcome. It’s a highly translational field, says Ghaffari, that needs cooperation from academics, pharma, and commercial interests. Adoption happens, he adds, when many studies validate the biomarkers, which clinicians can then embrace. At which point, it’s “no longer just cool technology,” notes Ghaffari.

Ramirez says he understood why someone would ask about bias, considering the technical sophistication a patient might need to use the Evidation app. A few years ago, he explains, people who used Fitbits and smartwatches were generally confined to two groups: the weekend warriors or the “worried well.” “I don’t think that is true anymore,” says Ramirez.

Use is becoming more widespread, so it crosses into different age groups, social backgrounds, and ethnicities. Data show that the elderly do have difficulty with the technology, but are more than willing to learn.

One person acutely observant of bias is Santuccione Chadha, who is also 

 CEO of the Women’s Brain Project. Brain diseases are more common in women, she notes, and women react differently to treatment; supporting evidence has shown a trend toward better effect in men as seen in the three anti-amyloid drugs and two immunotherapies for Alzheimer’s.

Last June, Santuccione Chadha and a large team of researchers wrote about these differences, based on findings with the Altoida platform. The cohort of 568 people was composed of those with mild cognitive impairment, brought on by Alzheimer’s, and healthy individuals. The platform predicted, with 75% accuracy, the sex of the healthy people and those with mild cognitive impairment.

“These results stress the need to integrate traditional approaches to dementia research with digital biomarker technologies and personalized medicine perspectives to achieve more precise predictive diagnostics, targeted prevention, and customized treatment of cognitive decline,” they wrote.

Santuccione Chadha mused about the immunotherapy drugs approved for Alzheimer’s; the data, she says, all showed a trend that men responded better to the medication than women.

“If we had digital biomarkers embedded in those trials, there would have been a different level of learning than what we know today,” she believes, mentioning speech recognition and microtremors that could have been measured via a digital health technology. The existing scales, she adds, were developed decades ago.

Other challenges in using digital biomarkers? They are of the human kind.

Mixing technology with patients, especially patients who are aging, have trouble processing information, and who are remote from investigators, present whole new challenges to researchers who have to learn to anticipate human glitches in all kinds of ways.

“You have to go through the pain of learning,” says Ghaffari, recalling how elderly patients will tend to throw away their device after day one. “That happens all the time. But the solution, the data, the promise of it is certainly there. The more we do, the more we will learn.”

Adams and colleagues, in work published in April on comparing data gathered from various devices on Parkinson’s, found that patients wore the watch on different wrists and data were lost because permissions restrictions to use the data had been turned off, so data from the watch couldn’t be sent to the database.

Benko notes that researchers have to think through all these questions, like Phase II trials involving therapy-naive patients, but Phase III trials not. Or that similar conditions in a disease spectrum will involve other similarities.

“For a drug developer, being able to distinguish why something is happening and define what is going on in a subgroup is very important,” he says. “You have to make sure that a tool doesn’t discriminate on something you didn’t think to question.”

 is a freelance writer for medical, clinical trials, and pharma information.

Raket, L.L.; Petcu, P.; Wac, K.; Hassenstab, J. Editorial: Digital Brain Health. 

https://www.frontiersin.org/articles/10.3389/fdgth.2023.1142897/full

Harms, R.L.; Ferrari, A.; Meier, I.B.; et al. Digital Biomarkers and Sex Impacts in Alzheimer’s Disease Management—Potential Utility for Innovative 3P Medicine Approach. 

https://pubmed.ncbi.nlm.nih.gov/35719134/

‘No longer just a cool technology,’ efforts to validate these signals are evolving.

Applied Clinical Trials-05-01-2023

Modern Technology is Shaping the Face of COA

Implementing ePROs in Arthritis Clinical Research

Executive Roundtable: The Rise of RWD in Clinical Research

Optimal COA Measurement Strategy in Modern Oncology Trials

Measured Momentum: Digital Biomarkers and Clinical Trials

A Life Less Ordinary: Remembering The Godfather of ePRO

A Complex Picture: What to Know About Clinical Trials Insurance

Striving for Carbon Footprint Benchmarks

Agenda Aims to Recharge UK-Based Clinical Trial Innovation

FDA, Industry Prepare for End to COVID Health Emergency