Check out the latest features and columns!

Applied Clinical Trials June 2022

ACT's biennial Salary Survey takes a look at the future of industry following increased adaptation of decentralized technology 

Salary Survey: The Age of COVID

Emerging tools in data analytics, automation, EMR aggregation, and even social media are enabling faster and easier patient recruitment

Patient Recruitment Goes High-Tech

TrialNAV, a clinical research solutions company designed to bridge the gaps between industry sponsors, research sites, and cancer trial participants has announced its launch. The company's suite of offerings will offer solutions for the identified challenges in the pharmaceutical and technology industries, clinical research organizations, site management organizations, and clinical research sites within healthcare organizations. The solutions are designed to improve enrollment in clinical research and provide better access to treatment options for people living with cancer.

Articles

TrialNav will connect sponsors, sites, and cancer patients that need access to new treatment options.

Oncology Research Veterans Launch TrialNAV

Worldwide Clinical Trials, Inc. has announced that it expanded its early phase clinical trial capabilities by opening its Bioanalytical Center of Excellence—a larger lab in Austin, TX. This investment builds on Worldwide’s recent addition of three key leadership roles and enables the company to optimally support preclinical and early phase development.

New facility in Austin, TX, enhances early phase research opportunities through new equipment and expertise, positioning CROs for further growth.

Worldwide Clinical Trials Opens Bioanalytical Center of Excellence Supporting Early Phase Development

We have previously described that real-world data has mediocre usage in the clinical trials despite enthusiasm for these analytics and their performance in the COVID-19 pandemic. In this blog, we consider how the data intermediaries software affect usage of real-world data. Data by itself is not useful by itself. There has to be an ecosystem wrapped around the data to curate & protect the data and make it usable. We use the term intermediaries/software to mean the various analytic tools, vendors, and consultants this ecosystem that converts raw data into usable information.

To identify the attributes of the data that drive real-world data adoption, we surveyed 133 pharma & CRO executives. We asked them to rate the importance of various data intermediary/software attributes that are important in selecting a real-world dataset, including the ability to enable analytics and allow evaluation as well as their, cost, reliability, problem solving, flexibility, responsiveness, and understanding of needs.

We averaged across across the attributes to calculate overall importance and satisfaction, as also calculated the difference scores between importance and satisfaction in each attribute. These difference scores are important because they represent the potential growth of real-world data in the current/future usage or opportunities to grow in the gaps between importance/satisfaction.

Importance Ratings for Data Intermediaries/Software Attributes

The overall mean for the importance of data intermediary/softwareattributes was 7.9, which was higher than the overall mean for the actual data (μ= 7.3). Note also that there is a relatively narrow range, suggesting that it is difficult to carve out one attribute on which to focus on data intermediaries/software. Next, we consider satisfaction with the data intermediaries/software.

 Satisfaction with Data Intermediaries/Software

You can see the satisfaction scores were lower than the importance scoressoftware.

 The Difference Between Importance and Satisfaction for Each of the Data Intermediary/Software Attributes

The average difference between importance and satisfaction was 1.5, greater than the average difference between data importance & satisfaction. This, along with the higher importance ratings, suggests that there is a greater opportunity for growth in real-world data usage by focusing on the data intermediary/software aspects over the actual data.

Survey taken by pharma and CRO executives evaluates how effectively various programs convert RWD into usable data.

Data Intermediaries/Software Can Speed the Usage of Real-World Data

While real-world data has shown the potential to transform the clinical trials industry, uptake and usage of real-world data has not met expectations. What drives the uptake and usage of real-world data? In this blog, we look at the attributes or characteristics of the raw data. More-and more companies are acquiring expensive and large datasets that allow for sophisticated analysis. What are the attributes of these datasets that drive real-world data adoption?

To assess this question, we surveyed 133 pharma & CRO executives. We asked them to rate the importance of various data attributes if they were selecting a real-world dataset. The attributes we considered were the ability to tailor the data structure and data dictionary and the availability of analytics support from the data provider, chart reviews, retrospective longitudinal data, prospective longitudinal data, and biomarker data.

 Importance of Data Attributes with Real-World Data

While the ability to handle longitudinal data stands out, all of the data attributes are all ranked fairly high and are all close to each other, suggesting the importance of the data and the difficulties that executives have in one attribute above all the others. Our respondents have expectations for the data. What about their satisfaction with these data attributes?

 Satisfaction with Real-World Data Attributes

Our respondents had low satisfaction relative to the importance of the data attributes. None of the satisfaction ratings across the functions came up to the importance. The average satisfaction level was a mean of 1.2 points below the rated importance. While biomarker data had the lowest satisfaction, we emphasize that is not far below the highest rated data attribute. None of the attributes excelled in their satisfaction scores.

We also examined the gaps between the importance and satisfaction scores to prioritize improvement efforts.

 The Gaps Between Importance and Satisfaction

The best opportunity to improve the performance (i.e. The Return on Data) of data is enhancing the reliability of data, followed by cost and responsiveness. Organizations need data that will work every time and be able to respond to organizational needs at a reasonable cost.

Some of these issues relate not just to the data, but also the intermediaries (analytic software, vendors, and consultants) that convert raw data to information. We consider this in the next blog.

Survey taken by pharma and CRO executives evaluates attributes of large datasets.

The Real-World Data That Matters

The Society for Clinical Research Sites (SCRS) announced that Clinerion has joined the organization as a Global Impact Partner (GIP). SCRS’ GIP Program provides an opportunity for sponsors, CROs, and solution providers to engage and collaborate with SCRS and its site members.

GIP program provides opportunity for sponsors, CROs, and solutions providers to engage and collaborate with SCRS and its site members.

Clinerion Joins SCRS as a Global Impact Partner

Within the domain of clinical trials, the goals of quality by design (QbD) are to assure that the outcome data are fit for their intended purpose and that there is an absence of errors that could impact decision-making. Since all products and services within a clinical trial are subject to regulatory oversight, all sponsors, vendors, and clinical research sites must always be prepared for that knock on the door when your administrator says, as mine said to me a few years ago, “The FDA has arrived and would like to speak to the person responsible for the clinical trial.”

A frequently asked question is what have been the barriers to entry preventing pharmaceutical and medical device companies from widely adopting and implementing paperless and digitally based clinical trials? Ironically, for more than a decade, technologies allowing for paperless and digital clinical trials have been encouraged by regulators. Fortunately, the time has finally come for the transition to a global acceptance by the pharmaceutical industry and regulators of the use of paperless and digital technologies to collect clinical trial data both within the clinic and remotely.

The FDA and others have issued multiple guidance documents addressing the use of digital technology tools when used in clinical trials. Now that the electronic world has finally entered the clinical trials space, as part of routine pre-approval inspections, there must be stakeholder assessments of the impacts of digital systems on 1) informed consent; 2) data collected to determine effectiveness; 3) patient safety by assuring that all safety events are accurately collected and reported 4) data integrity and accountability; 5) protocol compliance; 6) monitoring of the study, including risk-based monitoring; and 7) drug/device accountability etc.

Since digital-based companies have not been routinely subject to FDA laws, rules, and regulations, the following are key elements to be addressed and assessed by these organizations choosing to enter the clinical trials space.

 The basic goal of software validation is to assure that configured applications act as intended when used in clinical trials. Therefore, when regulators inspect software products, it is critical to document that the trial software applications are validated. Fortunately, starting in 2002, the FDA has provided validation guidelines for software supporting regulated products. For new products hitting the market, an assessment should also be made if FDA clearance of the software would be required through the device 510(k) or PMA regulatory approval routes. For global programs, local regulatory requirements must also be addressed.

 These are elements that must be considered during development of digital devices for use in clinical trials. FDA has issued a guidance document to assist the industry on how to follow appropriate human factors and usability engineering processes so that new medical devices will be safe and effective for the intended users, uses, and use environments. It is recommended that these studies focus specifically on the user interface, including displays, controls, packaging, product labels, and instructions for use. As part of their design controls, manufacturers must also conduct a risk analysis that includes the risks associated with device use and the measures designed to mitigate those risks.

 Some of those fully committed to using paper records often ask the question, if a system is fully electronic how do we know that the data are real if there are no paper source records to compare with the electronic record? To assuage some of these concerns, as part of a company’s SOPs, there must be supporting documentation that the vendor has been qualified and that the software is in a validated state. It is also critical that all roles and responsibilities are properly identified, as well as how the original source data are controlled. This includes the presence of date and time stamps for all transactions, and an audit trail of changes cannot be overwritten.

 One of the first things the inspectors may request is to talk to the head of quality as well as who was directly responsible for the trial. Be prepared to share your organizational chart as it relates to the study and copies of relevant contracts/agreements, although inspectors are usually not concerned about financial matters. The inspectors may also ask for the documents supporting vendor qualifications, software validation/testing, and risk mitigation plans.

Tech companies entering space must adapt and adhere to regulations set by FDA.

Regulatory Considerations for Digitally Driven Trials

Marcus A. Banks is a freelance health journalist

Since 2017 many clinical trial vendors have developed products aimed at improving patient recruitment for clinical trials, tracking patient responses to potential therapies, or integrating all data produced in the trial technology ecosystem. The COVID-19 pandemic hastened these changes, especially related to the technologies that support decentralized clinical trials, although some observers say that many technologies now coming into view existed pre-pandemic.

“The pandemic triggered not a digital revolution, but an attitudinal revolution in the pharmaceutical and life sciences industry. Decentralized clinical trials (DCTs) were a fashion statement two years ago, but now everyone has a strategy,” says Nimita Limaye, research vice president for tech-focused market research firm IDC Health Insight.

“There’s been huge growth in things like wearables and sensors, which are really mapping how patients are responding to interventions in a very subtle way,” observes Limaye. Thanks to these sensors, it’s now easier to tell if patients are taking prescribed medications, or trial participants are maintaining experimental treatments, she says. Limaye adds that the practice of e-consenting patients has become much more routine in the last five years, along with using machine learning to detect safety signals that might not be captured in a structured adverse event report but are hidden in the more idiosyncratic narrative records of the trial.

“Extracting intelligence from structured and unstructured data is an important use case” for machine learning, says Limaye. Another is to use machine learning to mine real-world data about how different populations fared on FDA approved drugs post-approval, to build synthetic control arms for future clinical trials in which real-world experience on the already approved drug is compared head-to-head against the new intervention.

Machine learning depends on training algorithms on large sets of data, Limaye notes, but many pharmaceutical companies securely protect their own clinical trial data in a way that’s impenetrable to machine learning algorithms. Limaye adds that federated learning technologies, in which algorithms access data that never leaves its secure location, are emerging solutions to this problem that pharma companies are beginning to embrace.

Overarching these technological developments is a broad sense among clinical trial observers that there’s a once-in-a-lifetime opportunity to infuse equity and inclusion into the design and deployment of clinical trials that use digital tools. Clinical trial participants, today, are not representative, which means that the results of clinical trials are not always generalizable.

“When we pick up these tools—whether that’s the use of real-world data, whether it’s running [artificial intelligence] and machine learning techniques on these data, whether it’s the use of telehealth for virtual visits, whether it’s the use of digital technologies and remote patient monitoring—it’s not enough to just say we’ll do it and we’ll figure out the equity bit afterwards,” says Jennifer Goldsack, CEO of the nonprofit Digital Medicine Society (DiMe), which aims to promote digital-first clinical trials that could not have been done in the analog world.

“We could be encouraged by how the current process works, when we talk about the history of the informed consent process,” adds Jude Ngang, executive director of representation in clinical research at Amgen and a DiMe partner in a new initiative to develop more inclusive clinical trials. According to Ngang, Amgen takes great pains to ensure that everyone, regardless of language or literacy level, understands what they are agreeing to as they begin a clinical trial, and says the same care can be taken with ensuring that all clinical trial participants know how to use a wearable sensor or other digital device.

Part of solving the trial recruitment puzzle is simply to alert more people about relevant trials. Amgen has invested in CancerIQ, a genetic risk assessment software program linked to patient medical records (EMRs). Many people go to community oncologists, not academic medical centers where frequent clinical trials occur, notes Ngang. With CancerIQ, a community oncologist can alert patients about enrolling studies for new cancer treatments they might not have learned about otherwise.

CancerIQ is just one of many clinical trial-related technologies. This article will detail other emerging tools that enhance clinical trial recruitment, improve participant monitoring once a study is underway, or aim to integrate clinical trial data in more impactful ways. Read on to learn about the brand new kids on the block, or the old kids who’ve learned how to pivot.

Linking patients, improving engagement, and novel measurements

“There’s this idea that doctors are afraid of losing patients to clinical trials, and that’s why they don’t refer them, but that’s not true. The main reason physicians don’t tell patients about trials is because it takes extra time,” says Mukul Mehra, a gastroenterologist and co-founder of IllumiCare. The company aims to make clinical trial referral a “prescribable event,” in the same way that a physician might prescribe a statin and then rely on existing infrastructure to get that drug to the patient.

Mehra says that trial notification systems do already exist within electronic health records (EHRs), but as email inbox messages that physicians might not read until hours or days after seeing the patient. At that point the physician needs to call up the patient record and triage whether they are eligible for the study, which is actually a better task for clinical trial coordinators.

IllumiCare’s “Smart Ribbon” technology surfaces matched trials within the EMR workflow, as the doctor is seeing the patient, and works with all major EHR platforms. A doctor can either discuss the trial with the patient on the spot, or refer them to a clinical trial coordinator to help them decide whether they’d like to enroll. Just launched in May for this purpose, after 24 months of development, Mehra expects that Smart Ribbon will make clinical trial recruitment and discovery easier.

Enrolling patients into a trial is just one challenge. Preventing attrition is another.

“How can we improve the participant experience in a meaningful way?” asks Andrea Valente, CEO of the clinical trial technology vendor ClinOne, whose prime client base are biopharma sponsors. Valente says that the average cost per patient of an oncology trial is $100,000, and up to $300,000 when involving a rare disease patient. Replacing patients who drop out mid-trial is expensive, Valente notes, and could lead to delays in availability of new treatments.

Much of ClinOne’s innovation in recent years is due to COVID-19, although the company pre-dates the pandemic. During the pandemic, e-consenting and video consults bloomed, as many ClinOne clients began to offer DCTs. Interactive SMS messages to remind patients to take medications also became more common, and a partnership between ClinOne and Uber Health offered transportation to patients who were unable to drive as a condition of enrolling in a study for a major depressive disorder treatment.

ClinOne found that, at some trial sites, 98% of elderly patients in a cardiovascular drug trial were successfully taking medications 18 months after the trial began, due in part to the medication reminders; and that offering free transportation (up to two round trips a day within a 200 mile radius, for personal and not only trial-related reasons) significantly enhanced enrollment in the depressive disorder study. ClinOne and the trial sponsor expect that retention in this study will be enhanced as well, thanks to the transportation solution.

Medical sensors and wearables, which people wear regularly as they move about their daily lives, open up new possibilities for clinical trials that the five-year-old company ObvioHealth aims to explore. Start with the humble cough.

“It sounds funny, but everyone’s cough is different,” says ObvioHealth’s CEO Ivan Jarry, who is working on a project that uses AI to detect hidden health risks within someone’s coughing patterns. This could ultimately inform new clinical trials, once clinicians and researchers have a better understanding of the health implications of different coughing patterns.

Meanwhile, ObvioHealth has developed a platform that continuously collects data gathered from wearables such as blood pressure levels, oxygenation levels, or heart rates of patients in clinical trials for different treatments—with the data fed to the research site in real time (or as soon as possible if the patient loses connectivity). ObvioHealth doesn’t just transmit the wearable data wholesale to the researcher, but also uses algorithms to detect if there are safety signals, and sends the results of this analysis, too. Its clients include pharma and biotech companies, contract research organizations (CROs), and academic medical centers.

For example, if a patient’s blood pressure always plunges right after receiving an experimental treatment, either that treatment is a total non-starter or can only work if the patient takes precautions. In the analog world, such a relationship between the medication and blood pressure might take weeks or sometimes months to detect.

“Our aim is to detect actionable signals from data,” says Jarry. ObvioHealth also wants to be a leader in women’s health studies, which Jarry notes have historically been understudied or studied with invasive procedures. This year, ObvioHealth was the research vendor for a fully virtual study that concludes that digital therapeutic devices improve the effectiveness of home pelvic floor muscle training (kegels) for reducing urinary incontinence symptoms in women.

Integrating data from every possible source

Several clinical trial technology companies focus on the so-called “back end” of digital clinical trials, or the data management and integration needed to make these studies possible.

KORE has existed since 2002, but just this year the organization acquired the companies Business Mobility Partners and SIMON IoT (internet of things) to enhance its connected health offerings. The acquisition helps KORE to serve high volume, Phase III clinical trials that can involve up to 3,000 participants, according to KORE’s Chief Marketing Officer Landon Garner.

IoT means that a physical object, like an internet-enabled blood pressure monitor, is linked to other connected objects such as a handheld device that records and communicates results (in this case, the blood pressure readings). IoT is why a programmable thermostat, say, can also turn on a HVAC system. And IoT is the reason why scientists and regulators moved from a COVID-19 vaccine concept to placing vaccines in arms within 12 months, Garner says.

“The pandemic became a forcing function for virtual or decentralized trials, using IoT technology,” says Garner, because the connectivity afforded by IoT technologies enabled real-time data capture that showed the efficacy of now-approved COVID-19 vaccines. The same flexibility is available to any company that wishes to run a global drug trial.

“We have access to 600 mobile networks around the world,” says Garner. According to the executive, KORE can provide tools such as digital weight scales or continuous glucose monitors that will work in any trial location. KORE’s technology does not analyze the data generated by a trial directly—that’s the work of algorithms and the clinical trial research team—but instead it federates the various data streams so that they all go to the right place.

The healthcare technology company Seqster created SeqsterOS to capture patient-matched EHRs, genomic DNA, wearables, and pharmacy and lab test data collected during a clinical trial, via its dashboard. Trial participants authorize with whom they share this data, in a HIPAA and FDA-compliant manner that also complies with European privacy regulations, according to Seqster CEO Ardy Arianpour. Once the trial participant authorizes sharing any of their data with vetted Seqster users, it is available.

“We’re at the tip of an exploding patient-centric and digital clinical trials industry,” says Arianpour, who co-founded Seqster in 2016 and launched an early version of its research portal in 2018. Seqster has continuously iterated its operating system since then, with the aim to “break down health data silos at scale,” in Arianpour’s words. Its most recent software integration, announced in April, is with Salesforce Health Cloud.

For health outcomes researchers interested in how economics affects participation and retention in clinical trials, Seqster includes socioeconomic determinants of health information, such as a trial participant’s zip code or annual income.

“Your zip code is one of the most important determinants of health, and most people don’t know that,” adds Arianpour. Seqster’s goal is to integrate every possible source of data relevant to someone’s health while in a clinical trial or as they are receiving care, and to enable algorithmic and human analysis of these various data streams.

Data curation and management will only become more critical to clinical trial deployments and healthcare delivery, contends Limaye, who notes “we’re a data-hungry world,” particularly when it comes to harnessing personalized data. For example, the ability, she says, to capture not only how people’s heart rate fluctuates or glucose levels vary, but also to sequence individual genomes or proteomes or lipid levels.

One challenge today, however, is that much of this data resides in silos, either in the purview of a clinical trial sponsor or EHR or a third-party provider. Limaye argues that developing holistic and longitudinal views of a patient’s health requires developing interoperable protocols that enable data flow between systems while respecting data security, privacy, and sovereignty.

At the same time as data volume is increasing, the proportion of hard-to-interpret medical data is rising as well. IDC predicts that less than 5% of all data generated electronically will be “analyzable” by 2025, meaning that many data-driven health insights will be hard to derive with current tools.

One exciting data analysis tool Limaye foresees is “codified listening,” in which machine learning and speech emotion-recognition algorithms provide new insights into the emotions or sensitivities that drive the behavior of patients or clinical trial participants. A patient might leave a voice memo that such software decodes. This technology is in its early days, Limaye cautions, but could open new doors to understanding the patient or trial participant experience.

Meanwhile, the distance between the clinical trial site and point-of-care delivery is shrinking. “There is a growing confluence between healthcare and life sciences,” says Limaye, noting that pharmacy giants such as CVS and Walgreens are becoming clinical trial sites, not just places for dispensing medications that were approved through an entirely separate clinical trial process. Limaye foresees an eventual collapse of the distance between healthcare delivery and clinical trial sites. Goldsack agrees.

“When you have to lock the data in a filing cabinet and only the [principal investigator] or the office manager has the key, yeah—you can see why that happened,” says Goldsack, in describing the historical separation between the clinical trial and healthcare delivery spaces. “But we are so far from that reality now.”

New Vendors in the Clinical Trials Industry Discuss Their Products

COVID has accelerated to the scene upstart digital efforts to transform patient engagement and data management/integration—but progress remains measured.

Clinical Trial Technology: New Kids on the Block are Changing Industry Dynamic

Veeva Systems announced Veeva Vault Study Training, a new application that brings together research sites, CROs, and sponsors into a single platform for end-to-end clinical study training.

Veeva Vault Study Training unifies learning management and clinical operations for integrated and automated training and real-time inspection readiness.

Veeva Announces Clinical Study Training Application

Kenneth Getz is a senior research fellow at the Tufts Center for the Study of Drug Development.

It is widely recognized that the clinical research enterprise embraced numerous changes and adaptations during the COVID-19 pandemic to maintain patient safety, accelerate the development pathway, and ensure clinical trial continuity. Topping the list of changes is the unprecedented number of protocol deviations that have been implemented since the global lockdown.

Sadly, we do not have empirical data at this time characterizing the expected sharp increase in the average number of deviations per protocol. However, my team at the Tufts Center for the Study of Drug Development (Tufts CSDD) has recently completed a study capturing pre-pandemic protocol deviation metrics from which future comparisons can be made.

Typically, protocol deviations are executed by study staff while the clinical trial is underway in an effort to accommodate study volunteers encountering difficulties complying with the schedule of visits or adhering to study medication administration requirements. Investigative site personnel may also deviate from the protocol when facing difficulties finding eligible patients, and following and performing clinical and administrative procedures dictated by
the protocol.

Most deviations have a minimal impact on protocol integrity and execution. A subset of all deviations, however, are more significant and may harm the quality of the research, may put the rights and safety of study volunteers at risk, and may detract from the completeness, accuracy, and reliability of the study data. There are a number of useful resources to help organizations identify, classify, and manage protocol deviations including those by the Drug Information Association’s (DIA) Good Clinical Practice and Quality Assurance community and by TransCelerate BioPharma.

Finally, its important to note that protocol deviations are the top cause of FDA inspection warning letters. Approximately one-third (30%) of all warning letters are due to the failure to follow the investigational plan. Site failure to adequately maintain case report records is the second most common reason, accounting for 17% of all warning letters.

To our knowledge, little-to-no data exists quantifying and benchmarking protocol deviation experience overall and by clinical phase. For this reason, Tufts CSDD engaged a working group of 20 major and mid-sized pharmaceutical companies and contract research organizations (CROs) to gather hard data on deviations associated with traditional clinical trial protocols (i.e., master protocols and adaptive designs were not included). Our study sample included protocols that had received final approval between January 2013 and December 2018 and had a primary completion date or database lock date before December 31, 2019. The data-collection process deployed in this study followed an approach that Tufts CSDD has been using to evaluate trends in, and the impact of, protocol design practice since 2008.

Data on a total of 187 protocols targeting a range of disease conditions was analyzed. One-quarter of the total were Phase I protocols, 39% Phase II, and 36% were Phase III protocols.

The table on the facing page provides high-level baseline measures of protocol deviation experience by clinical phase. Phase I protocols have the lowest mean number of deviations and involve half the proportion of patients than those observed in Phase II and Phase III protocols. Each Phase III protocol has a mean number of 119 total deviations, involving approximately one-third of all patients participating in that protocol. The coefficients of variation around the mean number are very high—between 1.7 and 2.0—indicating that the average deviations per protocol are widely dispersed and difficult to predict.

Wide variation in protocol deviation experience was also observed by therapeutic area. Phase II and III oncology protocols average almost 20% more total deviations than the average for non-oncology protocols, at 108.8 and 91.9, respectively. Protocol deviations in combined Phase II and III oncology protocols involve 47% of the total study volunteers, nearly double the proportion observed in non-oncology protocols. Rare disease indications average a lower relative number of total deviations (78.1) among a smaller proportion of study volunteers (27.7%) compared to that of non-rare disease indications.

Tufts CSDD also found that the incidence of protocol deviations was significantly, positively correlated with all clinical trial milestone durations, with the exception of study start-up and study close-out. The duration to complete all first-patient visits (first patient first visit to last patient first visit), the overall treatment duration (first patient first visit to last patient last visit), and the total clinical trial duration from protocol approval to database lock showed weak but significant positive correlations with the number of protocol deviations per protocol.

We were not surprised to find that no association existed between the incidence of protocol deviations and study start-up and close-out cycle times. Protocol implementation burden is generally low during the study initiation process. This burden, however, for both study staff and study volunteers, becomes more pronounced as these parties must comply with protocol implementation requirements over time.

A relationship between select protocol design complexity variables and the incidence of protocol deviations was observed. The number of endpoints and the number of procedures per visit were modestly positively associated with, and predictive of, the incidence of deviations per protocol. Adjustments to these select design variables may play a role in preventing protocol deviations and in minimizing their risk to the study and data integrity.

Two operational variables were also positively associated with protocol deviations. The number of countries showed a modest relationship. A positive relationship was also demonstrated between the number of investigative sites and the number of protocol deviations, suggesting that decisions and behaviors that diverge from the investigational plan may be a function of variation in investigative site personnel training, site infrastructure, operating environment, and culture.

Tufts CSDD plans to build on this body of research. We will be looking at the root causes of protocol deviations and how to address them. Future research will also look at deviations with more granularity to understand the incidence and impact of minor and major deviations. Whereas some deviations may improve participation convenience and clinical trial executional feasibility, other deviations may pose substantially greater risk to study volunteer rights and safety and to the quality and integrity of the clinical trial data. We also hope to understand the relationships between types of deviations and patient characteristics—including demographic subgroup and geographic location—and investigative site characteristics.

The results of this initial study provide useful measures that sponsors and CROs can use to benchmark their own protocol deviation experience and identify factors most associated with protocol deviations. 

Study uncovers pre-pandemic deviation levels.

Quantifying Protocol Deviation Experience by Clinical Phase

While the COVID-19 pandemic has undoubtedly catalyzed the adoption of more flexible data- collection approaches within clinical trials, this has been the direction of travel for some time. Rather than bringing participants into sites, allowing them to participate from the safety of their own homes by leveraging remote data-collection methods placed participant preference at the center of the research effort and positively impacted participant engagement. The use of electronic methods to capture clinical outcomes assessment (COA) data has facilitated this pivot, as has the use of telehealth applications. However, some cognitive assessments—such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment(MoCA)—require complex interactions between the participant and clinician to assess dementias and other neurocognitive disorders. These cognitive COAs are more difficult to administer remotely and require creative solutions to accommodate a cognitively-impaired population.

When choosing a cognitive assessment, it is important to consider whether the assessment is both appropriate to the population being investigated and that it will address the hypothesis and fulfill the criteria for the trial estimand. Another consideration is whether the assessment license holder will allow it to be performed remotely. There are several less complex cognitive COAs that more easily translate to a remote environment and indeed some are specifically designed for this purpose. However, these “less-complex” assessments may not satisfy the needs of the trial or be suitable for the population.

Telephone-based visits can accommodate some cognitive COAs, but many validated and standardized examples commonly used in clinical trials include a combination of clinician interview and participant performance tasks (e.g., drawing shapes, completing diagrams, or behavioral tasks). Under traditional administration conditions, the clinician and participant sit facing each other. The clinician can observe the participant’s behaviors and task completion. Drawing, writing, or diagramming tasks are typically done on paper, which becomes a source document and must be stored in the participant’s study file; this is not practical under remote administration conditions. Source documents must be dated and signed off to ensure they are compliant with ALCOA+ principles. In fact, to be truly ALCOA+ compliant, paper source records should always be “available” to the study investigator even before later delivery to the site. Photo images of paper records cannot be considered source. Despite these obstacles, there are creative solutions available for administering cognitive COAs remotely.

An approach that eCOA Consortium members have seen work in persons with dementia is by combining a secure telehealth system with an eCOA application. Clinicians and participants (assisted by study partners) are provisioned with tablet devices (with styli) that allows them to interactively write or draw directly on to the tablet screen when required. The clinician initiates the interview and records responses on their tablet. When it is time for the participant to perform the drawing task the clinician can “hand-off” access to the participant. The participant logs in with their own unique ID and uses the stylus to complete the task on their tablet device. The image created by the participant becomes the source data record and is saved directly to the server along with the associated meta-data that ensures compliance.

The telehealth system also allows the clinician to view and rate other participant behaviors, such as following commands and object naming that are common to these types of COAs. When migrating cognitive COAs to digital format, best practices guidelines should be followed. For example, when using the Repeatable Battery for Assessment of Neuropsychological Status (RBANS), digital drawing would be classified as a Level II change that would require psychometric testing.

Regardless of the pandemic, solutions like these are proving beneficial for all research stakeholders. Participants, study partners, and clinicians are all able to perform critical activities remotely. This approach focuses on the participant and is driven by science and technology, enhancing participant recruitment and retention, and reducing participant burden.

Authored on behalf of Critical Path Institute’s eCOA Consortium by 

, Principal Scientist, Clinical Ink; and 

, Senior Scientific Advisor, eCOA, WCG – MedAvante-ProPhase. The eCOA Consortium wishes to acknowledge the contributions of 

Pandemic aside, remote solutions are advancing care in clinical trials.

Featured Content

Applied Clinical Trials June 2022

Salary Survey: The Age of COVID

Patient Recruitment Goes High-Tech