Risk-Based Monitoring: An Opportunity to Go Back to Basics

SPOTLIGHT EVENTRisk-Based Monitoring – Part Two In Depth ReviewMarch 13, 2014
Cambridge, MassachusettsDownload BrochureRegister

RELATED
- Increasing Intensity of On-Site Monitoring a Troubling Trend
- Industry Metric Indicates Low ROI with Full Source Document Verification
- Has FDA Guidance on Risk-based Monitoring Impacted SDV Coverage Yet?More in Risk-Based Monitoring

GSK Vaccines uses CluePoints as a complement to its RBM plan

When the EMA and the FDA both released their views on risk-based approaches to clinical trials in August 2011, a light appeared for sponsors looking for relief in the way they manage resources in clinical trials. Specifically, the FDA guidance addresses the need to diversify monitoring, and the EMA delves into data quality.

The EMA paper says: “Since absolute perfection in every aspect of an activity is rarely achievable or could only be achieved by disproportionate allocation of resource, it is necessary to establish clear priorities, to mitigate the significant and serious risks to those priorities, and to establish tolerance limits within which different processes can operate.”

This perfectionist approach toward data in clinical trials has evolved over the years into a de facto practice of its own, 100% source document verification. This practice is now being re-engineered—with regulatory blessing—by sponsors, CROs and service providers to not only address the costs of 100% SDV, but to also challenge the way in which resources are allocated in this very resource-intensive industry. The challenge now is how, with regulatory framework in hand, do companies achieve risk-based monitoring? Between collaborations such as TransCelerate BioPharma, technology solutions, consultants and more, strategies are emerging.

Earlier this year, a new intelligent statistical modeling solution was introduced. The product—and the company—is called CluePoints. The product and the company come to market with a proven pedigree, with research and development from an academic standpoint for the last few years, culminating in a collaboration with GSK Vaccines before its commercial rollout.

As CluePoints’ founder Marc Buyse told Applied Clinical Trials: “ [Risk-based monitoring] is long overdue and, in particular, the reduction of trial costs through a reduction of monitoring visits whenever possible. I think these visits are very helpful to ensure proper trial conduct at all sites, but probably not useful at all for data checking. People are not good at examining data let alone identifying errors, signals and patterns in that information. It’s okay for them to look at the source documents and make sure that patients are real, but I think it doesn’t make sense to ask a human being to go through a very lengthy Case Report Form and make sure that every piece of data is correct. It’s a very expensive process, and not a very effective one.”

And that is where CluePoints fits in. When supporting an ongoing trial, CluePoints finds abnormal patterns in the data of centers. For example, the same values carried forward, repeated measurements that don’t differ enough, too many missing values, or variability in the data that is not consistent with that observed in other centers. Many large pharma companies currently use Key Risk Indicators in an attempt to control the risks by looking closely at the data in order to identify, for instance, centers having too many missing data or where the primary endpoint caused a lot of queries, etc. Key Risk Indicators are important and are used in these companies for a good reason, notes Buyse, but their implementation is far from evident. They need to be pre-defined, programmed, tested, and validated. They only use part of the massive volumes of data collected in clinical trials. In comparison, CluePoints takes a completely neutral view of the data, and looks at all data collected. Says Buyse, “There is no data item that is more or less important than another. So, the primary efficacy endpoint, which everybody worries about and is therefore likely to raise no red flags, is just one variable like any other. And other variables that receive far less attention, such as lab values, vital signs or quality of life measurements are going to be used by CluePoints to find problems just as well as the primary outcome variables. That makes it a very different way to look at the data, compared with key risk indicators.”

Francois Beckers, VP, Head Biostatistics & Data Management for GlaxoSmithKline Vaccines, told Applied Clinical Trials the company has been doing risk-based monitoring for two-and-a-half years. “I think risk-based monitoring is actually the opportunity for people to step back and go back to basics, as opposed to launching the trial, trying to do everything possible, including checking and re-checking everything. Now we can identify where the risks are, apply our efforts in the right place, and force people to make clinical sense of what they are doing.”

Beckers noted that GSK Vaccines’ use of CluePoints has been a key in finding the right complement between what a tool can provide and where resources need to be deployed. “It is really important to see, on the one hand, where the risks are, which is where we need to focus our attention; and, on the other, what is the complementarity between different tools so that we can be efficient in the way we perform on-site monitoring.”

GSK Vaccines has used CluePoints in numerous trials. Beckers notes it is interesting how patterns can be identified, how CluePoints highlights sites that are behaving differently from other sites, and how the signal detection forces the study team to reflect on varying behaviors among sites. Says Beckers, “That’s where the interaction exchange with the monitors who know the sites from visiting them can check different things and can also validate--or not--the signal.”

Change Management
How receptive are monitors to data-discovered site inquiries? Beckers notes that is part of an ongoing change management process. “We really want to emphasize a sense of partnership between the study team, as opposed to a ‘Big Brother is watching you’ attitude. So a key element is the collaboration between the team members.”

He continues, “Our goal is not to identify monitors who are doing well, and those who are not doing so well. The common goal is having the right data quality to meet the study objectives.”

Beckers sees that in some countries, there are very ingrained habits or fears of moving away from 100% source document verification—which represents significant change management. “Their perception is to do 100% SDV in order to have zero errors. This is an impossible goal, even with unlimited resources. There are two types of data errors--errors that are impacting the primary and secondary objectives of a trial, and those that do not affect the trial’s conclusion at all.” Although the study teams will continue using key risk indicators because these are developed specifically to reduce important errors, CluePoints has a place to find patterns in the data that were unexpected and might point to serious problems that would be missed using any other method.”

Beckers recommends the following for other sponsors in the throes of developing their own risk-based monitoring strategy:

• have the risk management mindset and a risk management plan in place. This has to be done upfront, but can evolve with the trial. 

• make sure the different stakeholders understand what you are trying to achieve with risk-based monitoring.

• make sure that the different stakeholders involved in the trial properly understand the different tools and components that are available to support risk-based monitoring.

Having these recommendations in place could be a good starting point for those sponsors currently grasping for more information on risk-based monitoring. According to Patrick Hughes, CluePoints’ Chief Commercial Officer, “It’s very clear that companies are almost desperate to find the solution that’s going to work for them but there’s no real shortcuts in terms of having to establish the right internal teams and look at the metrics of success.  It’s a big process change for any organization whatever the size.  So from a sponsor’s point of view, there isn’t any one-size-fits-all solution and certainly no blueprint for success.”

The short-term goal for CluePoints is to educate the market as to what a centralized statistical monitoring solution can bring to an organization. “It’s not the panacea, it’s not the be all and end all of what can be done, but it certainly is a valuable contributory part of the process as has been witnessed by GSK Vaccines and also now among those other sponsors who are beginning to harness the power of the solution,” says Hughes.

For a history on CluePoints’ development from Founder Marc Buyse, you can listen to this podcast.