OR WAIT 15 SECS
Risk-based approaches to monitoring (RBM) have become widely accepted in the clinical trials industry.
Risk-based approaches to monitoring (RBM) have become widely accepted in the clinical trials industry. Key milestones include the amendment to ICH-E6(R2) guidelines on November 9, 20161 and the adoption of the same by regulatory agencies in various regions-Europe (European Medicines Agency) and Canada (Health Canada)2. Other countries, including Japan and Australia, are expected to follow suit. The FDA has also indicated its encouragement of the new model through the guidance document in March 2018.3
In general, large organizations (sponsors and CROs) have begun implementing risk-based approaches to monitoring. It is fathomable that since large organizations command a greater share of drug development and outsourcing of clinical studies, the necessity to initiate and adopt new strategies and innovative models is a strategic imperative for them.
In contrast, due to limited financial and operational resources, smaller organizations face multiple challenges when adapting to rapid industry changes. Process modification, re-engineering SOPs, and operational change management can be daunting. Moreover, the development of new technologies requires upfront capital expenditure, which is risky when the model is evolving. All of these factors contribute to the slower adoption of RBM in smaller organizations.
However, the need to comply with regulatory requirements leaves little room for organizations to delay implementation of ICH E6(R2). Hence, organizational readiness is important when such a time comes into effect. This case study seeks to share the experience of a sponsor organization that decided to utilize RBM in a limited, but effective manner.
A global midsize biopharma sponsor developed a promising anti-infective drug into a Phase II trial. In 2016-2017, the sponsor conducted a global study, spanning 40 sites across seven countries and several continents, including North America, Europe and Africa.The sponsor team believed in risk-based monitoring and the potential benefits of that approach, but was not in a position to fully implement RBM for a variety of reasons, including a reliance on the processes and procedures established by the CRO. Since RBM was in its nascent stages at that time, and the industry had not fully understood the impact, the sponsor utilized RBM in a limited manner, albeit with clear expectations and expected benefits.
Given the importance of the clinical study and its potential impact on business strategy and growth, the sponsor’s primary focus was to ensure that the study conformed to the highest standards. Hence, monitoring the overall study from a risk and quality perspective was the key objective of adopting RBM.
Given the limited scope of the exercise, a simple yet objective risk-planning process was performed by the clinical operations and medical monitor from the sponsor team in consultation with a subject matter expert (SME).
For the global midsize sponsor, risks that could significantly affect the outcome of the study conduct were considered. Key performance indicators (KPIs) and key risk indicators (KRIs) were developed to objectively monitor the data in an efficient manner on a regular basis. The RBM SME facilitated and advised the risk-identification process, and developed the risk-assessment and monitoring framework.
Acuity Analytics - an integrated data analytics and RBM enablement platform-was utilized for developing data visualization and custom analytics for monitoring the KPIs, KRIs and specific critical data. The custom dashboards and analytics were designed in alignment with the sponsor’s specific needs. Data sources included an electronic data capture (EDC) system and the CRO’s clinical trial management system (CTMS). These were integrated with Acuity Analytics platform. Data from the EDC and CTMS were refreshed every 24 hours and once a week, respectively.
During the start-up phase of the study, the team selected three key performance indicators to monitor the progress of study setup at the sites. These included:
Validated Clinical Operations dashboards from the platform’s data visualization library were used to understand the delay points at various process operational touch points with sites during study start-up. Delays (at the site, CRO or sponsor level) were monitored so that proactive and timely intervention could be taken. The sponsor’s operations team and central monitor performed periodic reviews. Outputs were shared with different stakeholders, and timely action was taken on a regular basis.
Figures 1-3 depict a few of the operational dashboards utilized.
From the risk-planning process, six KRIs were developed and utilized during the active phase of the study. Validated dashboards from the Acuity Clinical Insights library, along with analytics related to efficacy endpoints were aligned to the sponsor’s needs. The following are the KRIs and critical data points that were developed:
· Adverse event rate
· Missing clinically acquired bacterial pneumonia (CABP) data points
· Missing eligibility (specific) data
· Potentially ineligible subjects utilizing surrogate data points
· Pharmacokinetic (PK) sampling deviations
· Major protocol deviations
Figures 4-6 depict a few of the dashboards utilized for the central monitor’s review.
A central monitoring plan was created during the initial phase of the study. The central monitor reviewed the KRI and critical data dashboards at frequent intervals (every four weeks) as per the central monitoring plan. The outputs were shared with the sponsor on a regular and timely manner. For certain types of complex data review that are prone to human error, custom algorithms were utilized to automate initial identification of issues. This was specifically applied for detecting potential eligibility violations and non-congruence of CABP symptoms in subjects as per the study protocol. In the event of specific issues, trends or patterns, more frequent reviews were performed. The outcome of the review was also recorded in a data-driven approach as per the criteria below.
Presence or absence of critical data
Extent of deviation from mean
Subsequently, the recorded outcomes from central monitoring were fed into a site-risk assessment model designed for Acuity’s Decision Factory. The model included the following risk thresholds and risk categorization illustrated in Table 1:
Additionally, the number of subjects enrolled at the site was factored into the model. The assumption is that more subjects at a site are directly proportional to increased operational and scientific workload for the site study teams and therefore, higher error rates. If such a scenario occurs, the impact on the safety of subjects at the site and scientific outcomes can jeopardize the overall study outcome. Needless to mention, additional operational and financial resources are required to bolster quality control and monitoring efforts. Hence, it was deemed fit to factor this into the risk-assessment model.
Since this was a short study, the RBM approach to monitoring was conducted over a period of 11 months. The start-up phase of the study was monitored for four months, and the active recruitment and follow-up phases spanned more than six months. The study was successfully executed as per the originally planned timeframe.
The three KPIs as mentioned in the methodology section were reviewed using relevant data visualizations. Figures 1 and 2 illustrate some of the data visualizations that were utilized.
Continuous monitoring of study start-up delays were performed by the sponsor’s operations team that enabled them to review process delays at the site and enhance governance of the CRO that managed the conduct of activities at the sites.
The six KPIs as mentioned in the methodology section were reviewed using relevant data visualizations. Figures 3 to 5 illustrate some of the data visualizations that were used for review by central monitors.
After the review by central monitors against the central monitoring plan, the review outcomes were recorded in the site-risk assessment framework and factored in the subject load at the site as depicted in Figure 6. The time-to-date, site-risk assessment was performed over the duration of central monitoring by comparing from the baseline to understand progress over a period. The team used a normal data distribution test to segregate sites that had significant increase and decrease of risks from baseline. Key outcomes of the analysis were:
o Missing eligibility (21%)
o Missing CABP values (21%)
This study had limited scope to implement RBM. However, this exercise proved to be a precursor for future clinical studies, and the observed significant benefits. The operational benefits included:
Stakeholders from this exercise also learned certain lessons. To a great extent, this could not have been incorporated as the expectation was to keep it light from an execution perspective.
Understandably, operational leaders often enquire about the business or financial benefits, which are as follows:
In summary, this is a good case in point of how small and midsize organizations can implement risk-based approaches to monitoring. The baby steps taken in a pilot study, along with an expert’s help, represent a plausible way to get hands-on understanding of the nuances of execution as well as foresee challenges when RBM requires implementation in a scalable manner on more studies. It helps in formulating the right interventions for adoption and change management between different stakeholders earlier on when organizational processes and SOPs need modification. Utilization of technology will be key and could level the playing field between large and small organizations. Research organizations can overcome apprehensions about investing in comprehensive enablement technology by procuring easy-to-use, smart and customizable innovative technologies on a study-to-study basis. This can be a quick-and-easy starting point before pursuing a strategic roadmap.
Abby Abraham, Vice President of Data Analytics and Risk-Based Monitoring at OmniComm Systems, Inc. and Ranjeet Gutte is the General Manager, Global Clinical Development for Wockhardt Ltd.