CRO/Sponsor

Clinical Trial Supply Chain

Decentralized Clinical Trials

Early Phase Development

Investigative Sites

Protocol Design

Risk-Based Monitoring

Study Start-Up

Trial Design

Clinical Operations & Strategy

Diversity Action Plans

Industry Initiatives

Recruitment

Analytics

Electronic Data Capture

Metrics & Benchmarks

Real World Evidence

Data Management

CRAACO

Enrollment

Integrated Research

Patient Participation

Institutional Review Board & Ethics Committee

Regulatory

Artificial Intelligence/Machine Learning

eClinical

mHealth

Technology

News

Academy

Publications

All Publications

E-Books

Media

Editorial Podcasts

Editorial Videos

Peer Exchange

Practice Academy

Conferences

Conference Coverage

Conference Listing

Columns

All Columns

A Closing Thought

Clinical Trial Insights

View from Brussels

View from Washington

Resources

Advertise

Blogs

Brand Insights

Events

Front & Center

White Papers

Webcasts

Subscribe

Latest solutions, case studies, interviews, practical articles for clinical trials operations, data management, technology, investigative sites and services.

About

Contact Us

Editorial Contacts

Do Not Sell My Personal Information

Privacy Policy

Terms and Conditions

Applied Clinical Trials Online

LinkedIN

Instagram

Facebook

Youtube

Soundcloud

Threads

Spotlight

DE&I

Topic

1rem

Authors

Using a specific trial, the importance of assessing the success of therapy masking is examined.

Evaluating the Effects of Therapy Masking

An important criterion for a well-conducted clinical trial is minimization of bias in endpoint assessment.

Reliable data is critical for making informed decisions about the efficacy and safety of new treatments in clinical trials. While self-reporting from the trial participant themselves on signs, symptoms, feelings, and function is always preferable, in certain populations it is common, or indeed necessary, for data reported by caregivers to form the primary or secondary endpoint. For example, trials in children who are unable to self-report, and individuals with cognitive dysfunction or neurodegeneration such as Alzheimer’s disease often rely on data reported by their caregivers (this does not refer to a healthcare professional). Such observer-generated outcome-related data is termed observer-reported outcome (ObsRO) data.

The challenges of missing data vs inter-rater variability

In the ideal scenario, the same caregiver would respond throughout the trial to maintain consistency in the data being reported; however, it is not always the case that a trial participant will have one, dedicated caregiver making and reporting all relevant observations. Although ObsROs should only be based on observable parameters, there is still potential for subjectivity in the way an observer rates a given behavior, symptom, or event, and, therefore, the potential for variability between raters.

 If multiple raters were to report on the same endpoint, then any discordance between raters when pooling data could obscure treatment effects.

Assuming the same caregiver is always with the participant is unrealistic, especially when data is being captured remotely; among many situations, the reality includes children splitting time between separated parents, being looked after by grandparents, and older populations being cared for by multiple caregivers. This issue needs to be addressed regarding the way ObsRO data are managed, analyzed, and reported to authorities.

It is well documented that missing data presents a large challenge to evaluating treatment efficacy, and although there are established techniques to help deal with missing data during analyses, these cannot be seen to be a replacement for the actual data points, and in any case, will only be appropriate in certain circumstances. This, therefore, leads to the question of whether restricting data collection to a single caregiver and potentially increasing the amount of missing data points is preferable to multiple reporters that could lead to more complete but higher-variability data.

It is worth highlighting that this is not a new issue. It is unlikely that the same caregiver consistently completes ObsRO(s) during a trial for a given participant. Even if it is the same reporter, they may not complete the ObsRO(s) within the specified time window. Yet these measures have traditionally been completed on paper, and ensuring attributability and contemporaneousness is far more challenging for that modality.

ObsROs are increasingly completed electronically, and yet there remains a higher level of scrutiny that electronic data capture is subject to despite the numerous benefits it offers over paper. Have the increased safeguards to data collection offered by electronic data capture, such as unique logins and preventing data completion outside of the predefined time windows, brought greater attention to the issue of multiple caregiver report?

Here, we highlight some areas for considerations for data collection in clinical trials that consist of multiple caregivers.

Training is key for all stakeholders in clinical trials, and if there are multiple caregivers reporting in a trial, it is important that they all receive role-specific training. It can be challenging to provide training on specific copyrighted ObsRO measures (beyond any that are already mandated by the copyright holder), as they have often gone through a validation process to ensure that the instructions and items can be understood without assistance from anyone else. 

When the same individual is reporting throughout, any difference in the way items are interpreted and rated between reporters (e.g., mothers of different participants) in a trial would be less concerning, as there would still be consistency within the reporter associated with a given participant. 

Where there are multiple reporters associated with the same participant (e.g., mother and father), any differences in the way an ObsRO is interpreted can introduce noise into the data when a different reporter is responding at different timepoints. Thus, some form of training to reduce this may be beneficial (such as what constitutes a severe symptom). However, as highlighted, this will need to be carefully thought through for copyrighted measures, and according to the specifics of the protocol.

For home-grown measures, this can be more straightforward. For example, it is common for event-driven diaries to be developed for a specific study, and training on what constitutes a discrete event and how to record it will be a core component of training for all raters.

Setting up the electronic data capture system

It is important to ensure the electronic clinical outcome assessment system is set up to facilitate multiple caregiver reports. This includes each reporter having a unique identifier to ensure data attributability. It is also important that data flows into the same location, and given the risk of duplicate data entry, that devices/accounts are synced as close to real-time as possible. Planning for how duplicate data entry (if it does occur) will be addressed in the dataset and analyses will also be important.

Inter-rater variability is a known phenomenon, and it is well-documented that pooling data from multiple respondents can create noise. An ISPOR Task Force Report on this topic states that it “raises the question of how data from different reporters can support a single claim.”

 However, that mainly addresses the scenario for when different age-group versions of the same measure are being used and where the reporter can vary between parent and child, though the concept of multiple reporters is similar. While one approach could be to pool the data into a single analysis, it would be necessary to demonstrate that there is an acceptable level of agreement between the raters. It would also be important to conduct sensitivity analyses and examine if outcomes are similar. Further guidance on this would be beneficial, including how results should be interpreted if the two reporters have different perceptions.

For any clinical trial, a robust statistical analyses plan is critical to outline the approach that will be taken.

As with any topic related to medical product development, regulatory considerations should inform study design. Understanding what is an (un)acceptable level of missing data and what evidence would be required to approve a treatment where efficacy is based on data from multiple caregivers will help ongoing conversations around the reality of multiple caregiver reports.

While the preference is always for participants to self-report, and where ObsROs are used that it is a single observer reporting throughout, this is not the reality. The request for enabling multiple reporters within the electronic data capture system is becoming more common, and best practices should start being developed. While such guidance does not exist yet, we cannot ignore this reality, and we hope this is an area that receives more attention to ensure the optimal approach is taken.

, PhD, Chief Scientific Officer, uMotif; and 

, PhD, PharmD, Chief Medical Officer, Kayentis.

1. Langberg J.M.; Epstein, J.N.; Simon, J.O.; et al. Parental Agreement on ADHD Symptom-Specific and Broadband Externalizing Ratings of Child Behavior. 

https://journals.sagepub.com/doi/abs/10.1177/1087054714561290

2. Davé, S.; Nazareth, I.; Senior, R.; et al. A Comparison of Father and Mother Report of Child Behavior on the Strengths and Difficulties Questionnaire. 

https://pubmed.ncbi.nlm.nih.gov/18266104/

3. Matza, L.S.; Patrick, D.L.; Riley, A.W.; et al. Pediatric Patient-Reported Outcome Instruments for Research to Support Medical Product Labeling: Report of the ISPOR PRO Good Research Practices for the Assessment of Children and Adolescents Task Force. 

https://pubmed.ncbi.nlm.nih.gov/23796280/

The conundrum of missing data vs. inter-rater variability.

Data Collection in Clinical Trials with Multiple Caregivers

Click the title above for a link to open the Applied Clinical Trials February 2025 issue in an interactive PDF format.

Applied Clinical Trials February 2025 Issue (PDF)

FSP Partnership: A Foundation for Innovation and Patient-Guided Development

The Food and Drug Omnibus Reform Act (FDORA) and Food and Drug Administration’s (FDA’s) latest Draft Guidance on Diversity Action Planning underscore a significant shift in the consequences of limited diversity in clinical trials. For leading research sponsors, the operative word has become “action” as the clock is ticking to implement best practices and track measurable progress towards truly representative research. However, the reality of recruiting historically underrepresented community members for clinical trials is that barriers to participation are multifaceted and best practices can be labor-intensive. At times, it may seem impossible for research sponsors to establish sustained engagement with communities at the grassroots level required to achieve affective trust.

Acclinate’s solution is simple: before recruitment, build representation. With the mission of empowering communities to take action for better health, Acclinate has engaged 100,000+ community members on its NOWINCLUDED platform with a comprehensive library of scroll-stopping digital content, a robust network of activation partners, and the dynamic curation of relevant opportunities to participate in clinical research. In this session, you’ll hear from Acclinate CEO, Del Smith, PhD, on:

The important difference between affective and cognitive trust;

The power of mobilizing grassroots community networks and local activation partners at scale;

The critical roles of predictive analytics and AI in driving results for clinical trial diversity.

See how Acclinate accessed, engaged, and mobilized BIPOC women of
childbearing age to increase awareness of and openness to clinical research

8 Best Practices for Clinical Trial Diversity

Discover best practices to make studies more representative and plan
more inclusive studies going forward.

Increase Diversity in Trial Participation Today.

Acclinate’s NOWINCLUDED platform leverages a model of affective trust to boost inclusivity in trials faster and for the long term.

Diversity Plans in Action: How to Boost Inclusivity in Trials Faster and for the Long Term   

Continuing on Day 2 of this year’s SCOPE Summit in Orlando, a noteworthy session examined ways to better recognize and support clinical trial participants. Titled “Setting the Tone for the Patient and Site Journey,” the session was headlined by Steve Rosenberg, CEO of uMotif and a 20-year industry veteran.

Kicking off the session, Rosenberg shared his belief that patients in clinical trials don’t receive the recognition they deserve, emphasizing that they should be treated as heroes.

“I’ve always believed that patients get the short end of the stick,” he explained. “They’re not treated like the heroes that they are. They don’t board planes first. That’s when it hit me—military personnel board planes first, but what about clinical trial participants who are sacrificing both their time and bodies for the greater good?”

Rosenberg then presented several market trends shaping the clinical trial landscape. Current data indicates that the industry is expanding by 6.5% annually, with electronic patient-reported outcomes (ePRO) growing at an estimated 16%. Additionally, involvement from patient advocacy groups has increased, patient burdens have intensified, and regulatory requirements from the FDA have become more stringent in recent years.

“A number of organizations are now measuring patient burden,” he said. “The burden has been rising. It’s gone down slightly in oncology, but it has risen significantly in other therapeutic areas. Some of the key factors driving this burden include extensive travel time, drug side effects, and the increasing number of questionnaires patients must complete.”

He then stressed that these challenges are not temporary; in fact, they are worsening. Clinical trial protocols are becoming more complex, and questionnaires are being used more frequently.

“I do think there’s a tendency to ask more questions than are truly necessary and collect more data than what is actually needed,” said Rosenberg. “But that needs to be balanced with what patient sites can realistically handle. As a result of these burdens, patient dropout rates are rising—not just due to clinical or medical concerns but because patients are making the choice to leave. Excessive time demands, poor communication with trial sites, and cumbersome procedures all contribute to participants withdrawing.”

Shifting the discussion toward solutions, Rosenberg highlighted how technology can play a role in improving patient engagement and retention. He introduced uMotif’s partnership with CLIN One as a key example, explaining how the collaboration aims to enhance study coordination, boost patient engagement, and ultimately prevent dropouts.

“We have the opportunity to team up—and allow sponsors and sites to collaborate—to improve the patient experience,” he explained.

According to Rosenberg, the combined platforms of both organizations will offer centralized tools to track protocol details, schedule visits, and manage patient interactions. These tools include digital reminders, study-related content, and real-time communication between patients and sites. Additionally, analytics-driven alerts will help trial sites identify at-risk patients and intervene before they drop out.

“It’s more about having the right information available so sites can effectively manage patients as they come in,” he stated. “Sponsors like to see what’s happening, and now they can track what they’re putting into the site and how patients are responding.”

In his closing remarks, Rosenberg emphasized that achieving true patient-centricity requires a collaborative effort—sponsors, sites, and technology providers must work together to address individual patient motivations and needs. He also challenged the industry to move beyond standardization and acknowledge the deeply personal reasons why people choose to participate in clinical trials.

“We need to recognize the individuality of each person after they complete a trial,” he concluded.

Conference breakout session explores strategies for trial sites to strengthen patient relationships.

Latest Conference Coverage

SCOPE Summit 2025: Steve Rosenberg on Setting the Tone for the Patient and Site Journey

CluePoints Risk Detection

ACT: What are CluePoints’ key objectives for participating in SCOPE 2025, and how do you see the conference shaping the conversation around risk-based quality management?

I think the very first thing is we're delighted to be here, because there's a need to validate the conversation. 12 years ago, a word that didn’t exist in our vocabulary—nor did AI, to be honest. Now it’s front and center. We’re here really to continue that dialogue, to share and expand, but also transition from the potential to the reality. This is what we're doing today, this is what we’re going to do tomorrow. I think that’s the exciting thing about meeting like this, it’s not a false environment, it's a genuine collaborative environment with so many different conversations, and it's nice to be at the center of it.

ACT: How has CluePoints’ approach to risk detection and mitigation evolved in response to the increasing complexity of global clinical trials?

We’re about to live through the ICH GCP E6(R3) release. In fact, we’re hosting a meeting with the FDA in just about five weeks’ time to go over it. It's going to completely change our approach. I actually think one of the biggest outcomes will be to challenge the use of that word risk. It's a word that confuses, and I wonder whether we might move into more of an informed or a data-driven era where risk isn't a negative word, it's a really positive phrase. But those guidelines are going to change everything from chain of custody to the approach we take, to just embedding a transformational, modern methodology into every decision we make.

At SCOPE this year, the key objective is to advance the conversation around risk-based quality management (RBQM), transitioning it from a theoretical concept to an actionable reality. The conference provides a collaborative platform to discuss the evolution of RBQM, particularly in light of emerging technologies like AI, which were not even part of the industry's vocabulary a decade ago. The goal is to validate and expand discussions, ensuring that RBQM is implemented effectively across clinical trials.

A significant regulatory shift impacting RBQM is the imminent release of the updated ICH GCP E6(R3) guidelines. These changes are expected to transform how sponsors approach RBQM, potentially even redefining the concept of “risk” itself. Rather than being viewed as a negative factor, risk management could shift toward a more data-driven, informed decision-making process. These guidelines will influence aspects such as chain of custody and the integration of modern methodologies into trial management.

The future of RBQM is increasingly centered around data as a comprehensive ecosystem rather than isolated data points. Industry stakeholders are prioritizing different drivers—cost, time, resources, or quality—in their approach to RBQM. While all four factors are important, organizations must determine their primary focus to optimize decision-making. The industry is working toward a model where these priorities can be balanced more effectively, ensuring that clinical trials remain patient-centric while also meeting regulatory and operational goals.

Ultimately, SCOPE serves as a critical venue for shaping the future of RBQM, fostering discussions that drive innovation and efficiency in clinical research.

In an interview with Applied Clinical Trials Associate Editor Don Tracy, Richard Young, chief strategy officer, CluePoints, discusses goals of the SCOPE Summit and CluePoints' evolving approach to risk detection. 

SCOPE Summit 2025: Richard Young on CluePoints' Approach to Risk Detection in Clinical Trials

As the use of artificial intelligence (AI) has greatly accelerated in recent years, AI has been moving to reshape the clinical trial landscape by addressing critical challenges and improving outcomes across the entire process. According to 

AI is streamlining the clinical trial process through recruiting and enrollment, trial design, protocol optimization, data management and analysis, enhanced patient retention and compliance, regulatory compliance, post-trial analysis, and real-world evidence.

The article states that in trial design, machine learning (ML) models simulate outcomes and enable adaptive protocols to enhance efficiency and decision-making. Additionally, AI automates data collection and performs advanced real-time analyses, reducing errors and uncovering actionable insights, while also improving patient retention through personalized engagement and behavioral monitoring.

 US Pharma and Biotech Summit, Tala Fakhouri, associate director, policy analysis, FDA, discussed the FDA’s point of view on ML and AI in drug development.

“We're actually very excited about AI use,” said Fakhouri. “I think we're seeing that it's increasing efficiencies in different parts of the drug development process. If you think about things such as discovery or protein folding, which again, is outside of what we normally look at, it could potentially cut the development time by years. This is all very exciting, because it could translate into faster, safe and effective drugs coming into the market. It can also fill in certain gaps for rare diseases where we can see a lot of potential use for AI to accelerate the development of drugs. In this type of situation, that's what I would say would be the long term.”

Fakhouri added that in the short-term, the primary challenge that stakeholders are working to address is the adoption curve.

“You need to train your staff, you need to bring in the right expertise, and you need to develop the right tools to solve the right problems,” Fakhouri said. “That's going to take some time and that's why I think the short-term uses of AI are going to be mostly low hanging type of fruits where you're increasing operational efficiency, but then that will translate into the development of safe and effective drugs faster.”

On October 31, 2024, the FDA released updated guidelines on the use of AI for drug development. According to the guidance, the regulatory body recognizes the growing role of AI in drug development, spanning the entire drug lifecycle from nonclinical and clinical phases to manufacturing and post marketing.

In order to support industry innovation and maintain safety and efficacy standards, the Center for Drug Evaluation and Research (CDER) released a draft guidance earlier this year addressing AI’s use in regulatory decision-making. According to the updated FDA guidelines, the guidance is influenced by extensive public feedback, CDER’s review of over 500 AI-related submissions since 2016, workshops with stakeholders, and aligns with broader FDA efforts to ensure responsible AI use.

Fakhouri previously stated that the FDA had no intention of placing restrictions on the use of AI and ML as time goes on.

“We get asked if the FDA regulates large language models or if we’re going to ban generative AI use,” she explained. “My response is that we typically don't regulate linear regression. We look at the data and the information that any modeling technique is producing, and we want to make sure that the information is trustworthy. So, I wouldn't say that we would be banning or prohibiting a certain AI or machine learning type of algorithm, what we're actually interested in is how robust how accurate and how credible the information from these models is.”

According to IBM, there are a number of benefits associated with implementing AI in clinical trials and medicine development, including:

IBM states that a trained ML algorithm can help cut down on research time by giving clinicians valuable search results with evidence-based insights about treatments and procedures while the patient is still in the room with them.

 Peer-reviewed studies have suggested that AI-powered decision support tools can help improve error detection and drug management.

 This includes reducing medication errors, customized virtual health assistance, fraud prevention, and supporting more efficient administrative and clinical workflows.

Increasing Physician-Patient Engagement: 

AI can help provide around-the-clock support through chatbots that can answer basic questions and give patients resources when their provider’s office isn’t open.

There are many thought leaders in the industry who believe that AI has the potential improve healthcare in all areas in the future. However, the biopharma industry has moved at a slower rate to implement AI compared to other fields.

At this year’s Young & Partners Pharmaceutical Executive Summit, Najat Khan, chief R&D officer, chief commercial officer, Recursion, offered her thoughts on why utilization has advanced at a slower rate and how companies can properly integrate AI into their everyday workflows.

“A huge part of it is the mindset, but pharma companies, in general, have high margins,” said Khan. “Whenever an industry has high margins, what’s the reason for change? That's the truth of it. Second, there’s a lack of understanding when it comes to using it. I think everybody knows what the pain points are. But then how do you deploy it? Sometimes, it means that your job changes. That's threatening, right? I've seen that firsthand, and this has happened over many generations. There’s two ways to respond. Either you evolve or you get evolved. You don't have to be a coder to actually understand AI. It's just like another science or another technical topic. But it's more important to understand how to use it, what not to do, what to do—given the complexity of the pharma industry, the regulations that are there, the limited understanding of biology today—there's just a fear of both personally being threatened.”

1. How AI is Revolutionizing Clinical Trials. 

September 24, 2024. Accessed November 1, 2024. https://www.linical.com/articles-research/how-ai-is-revolutionizing-clinical-trials

2. Artificial Intelligence for Drug Development. 

October 31, 2024. Accessed November 1, 2024. https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/artificial-intelligence-drug-development

3. US Pharma and Biotech Summit 2024: Artificial Intelligence and Machine Learning Through the Eyes of the FDA Part II. 

May 21, 2024. Accessed November 1, 2024. 

https://www.pharmexec.com/view/us-pharma-and-biotech-summit-2024-artificial-intelligence-and-machine-learning-through-the-eyes-of-the-fda-part-ii

4. What is artificial intelligence in medicine? 

https://www.ibm.com/topics/artificial-intelligence-medicine

5. The Impact of Artificial Intelligence on the Creation of Medicines. 

October 24, 2024. Accessed November 1, 2024. https://www.pharmexec.com/view/young-partners-pharmaceutical-executive-summit-2024-impact-artificial-intelligence-creation-medicines

With the increasing use of artificial intelligence and machine learning in medicine development, what role will they play in the clinical trial space moving forward? 

April Slee

Articles

Latest Updated Articles