Proper training can prevent oversight of inaccuracy, error, and variability rates in data from COAs.
Clinical trial failure rates are estimated to be around 90%, with rates varying by trial phase, therapeutic area and drug class and target.1-3 Trial failures can be attributed to many factors, most commonly the drug optimization process and the pharmaceutical companies’ trial management strategy.2,3 Some often overlooked but critical factors are the rates of inaccuracy, error, and variability in data from clinical outcomes assessments (COAs) used to determine efficacy in a clinical trial. COAs are intended to measure pertinent aspects of a trial participant’s health relating to the trial endpoints. If COAs support the primary outcomes in a study, they effectively determine the success or failure of the trial.4 COA measures can be completed or rated by clinical professionals via a clinician-reported outcome (ClinRO), by trial participants using a patient-reported outcome (PRO), by an observer (e.g., a caregiver) through a observer-reported outcome, or based on performance outcome measures. COAs are subjective and can be influenced by judgment, training and motivation.5,6 For example:
As ClinROs are susceptible to variations in rating and lack of precision,11 it is critical to conduct training to enhance rater reliability and precision by standardizing scale administration and scoring.
Rater training is a well-recognized mechanism for improving the signal-to-noise ratio by minimizing inconsistency, errors, and variability (e.g., inter- and intra-rater reliability) and bias in trials.6,12,13,14
Rater training is employed to varying degrees across therapeutic areas. In central nervous system clinical trials, training assumes a pivotal role, ensuring the reliability of measurements and the sensitivity to detect changes throughout the course of a trial.15 In many CNS trials, ClinROs and PROs are used in the absence of objective outcomes.
Other therapeutic areas such as dermatology and rheumatology use COAs that are prone to significant variability and subjectivity of the scales. Complex scales, such as the British Isles Lupus Assessment group (BILAG) disease activity index used in trials for lupus, may require the rater to consider current and historic symptoms and laboratory results to distinguish manifestations associated with active disease16 and training raters that significantly improve inter- and intra-rater agreement.17 In dermatology, body surface area measurement has been shown to be highly variable, even among trained individuals,18 speaking to the need for within-study training to improve consistency among raters.
With participant or observer/caregiver assessments, individuals may not understand concepts, terminology, or the measurement scale, resulting in spurious or highly variable data.19 Implementing educational materials for participants that outline the terminology used (e.g., what is fatigue) and/or expectations on specific tasks and complex diaries (e.g., Hauser et al.’s Parkinson’s disease diary20 and epilepsy or migraine diaries) will improve data quality.
The success of a clinical trial depends on the ability to determine if a treatment is safe and effective. When data demonstrate high variability due to inconsistent administration within and among raters, subjective scoring criteria, erroneous rating, biases, or any other behavior adversely impacting data quality, the potential for a trial to fail is increased. Training raters, be they clinical professionals, trial participants, or study partners, can improve the ability to detect true treatment effects by improving data quality, contributing to a successful clinical trial.
Authored on behalf of Critical Path Institute’s eCOA Consortium by Rinah Yamamoto, principal scientist, Clinical ink; Jennifer Olt, senior clinical scientist, Signant Health; Sayaka Machizawa, associate director, clinical science, Signant Health; and Martina Micaletto, senior clinical scientist, Signant Health.
References
Unifying Industry to Better Understand GCP Guidance
May 7th 2025In this episode of the Applied Clinical Trials Podcast, David Nickerson, head of clinical quality management at EMD Serono; and Arlene Lee, director of product management, data quality & risk management solutions at Medidata, discuss the newest ICH E6(R3) GCP guidelines as well as how TransCelerate and ACRO have partnered to help stakeholders better acclimate to these guidelines.
Beyond the Molecule: How Human-Centered Design Unlocks AI's Promise in Pharma
June 23rd 2025How human-centered AI that is focused on customer, user, and employee experience can drive real transformation in clinical trials and beyond by aligning intelligent technologies with the people who use them.