Francois Torche

	Poor quality data has the potential to destroy the validity of a clinical trial, wasting precious time and resources while introducing sizable roadblocks into the drug development pathway. But centralized monitoring offers sponsors and contract research organizations (CROs) the program-wide oversight they need to successfully develop the medications of the future. 

	The statistical analysis of real-time data can identify atypical data early, giving study organizers the opportunity to investigate and rectify issues such as fraud, sloppiness, training needs, or faulty equipment before they have a chance to impact on the trial’s outcome.

	At the base level, centralized monitoring matters because it is a key component of the 2017 ICH E6 (R2) addendum. But its potential benefits make complying to this requirement much more than a box ticking exercise. 

	Today’s complex, fragmented studies make data oversight increasingly difficult. Yet reviewing information as it accumulates, says the addendum, can identify missing, inconsistent or outlying data, spot protocol deviations, and examine trends in the range, consistency and variability of data both within and across sites. It can analyze site characteristics and performance metrics, evaluate for systemic errors in data collection and flag up potential data manipulation or integrity problems.

	All this makes it an extremely valuable adjuvant to traditional review activities, such as those performed by clinical data management and data reviewers. By operating at a more contextual level, it can identify site and study issues that are likely to be significantly more impactful than those discovered during transactional, record-by-record inspection processes like data cleaning.

	In short, centralized monitoring that utilises sophisticated data analytics and data visualization offers previously inaccessible oversights that can be continuously used to drive up data quality.

	The primary goal of statistics-based centralized monitoring is identifying operational and clinical study risks. It does that by highlighting atypical patterns of data that could signal issues related to study conduct.

	This approach allows sponsors and CROs to identify issues such as fraud, tampering or sloppiness in data entry, or problems with training or study equipment, and investigate them before they impact on the trial’s integrity.

	By taking a whole-program view, statistical data monitoring can examine the entirety of a trial’s data to spot outlying data trends that could signal a problem (as opposed to Key Risk Indicators that are limited in number).

	It can use statistical tests to identify sites recording an atypical mean for a given measurement or atypical variability in measurements between patients, for example. Discreet, or list, tests can identify sites reporting atypical proportions of a particular value or option, and day/time tests can, among other things, spot sites with an atypical distribution of patient assessment times.

	A sponsor of a phase III study in a chronic disease, for example, was alerted to fraud by looking at date/time tests on metadata. These tests highlighted that one site had entered the ePRO diary entries of all its 15 patients at the same time. This was obviously suspicious, and the subsequent investigation led to all 15 patients being removed from the analysis-with the consequences you can imagine on the statistical power of the trial. 

	In another trial, a diabetes study involving 300 patients, centralized monitoring revealed one site had an unusual lack of variability across patient vital signs measurements. It transpired upon investigation that staff were inventing missed vital signs measurements by interpolating.

	Industry-leading centralized monitoring software will also allow sponsors and CROs to set Quality Tolerance Limit (QTL) predictors similar to KRIs but at the study level.

	ICH-E6 (R2) calls for trials to contain predefined QTLs and stipulates that any deviation from these limits triggers an evaluation to determine any necessary remedial action. QTLs, which are set trial-wide, are designed to identify systemic errors, and any breach must be recorded in the clinical study report (CSR). 

	KRIs and QTLs are particularly useful because they can alert sponsors to potential problems early, meaning they can be investigated and rectified before they impact on safety or become a breach.

	Typical KRI examples are: visit-to-eCRF entry cycle time, missed assessment rate, auto-query rate, early termination rate and (Serious) adverse event (AE) rate. 

	It could be cause for concern, for example, if the rate of AEs was 3 per person and per visit at one site while the overall trial is reporting no more than 1.2. It could be indicative of either sloppy reporting (and therefore a need for re-training) or safety issue. The KRI alert will give sponsors the opportunity to investigate and address the issue before it impacts in the validity of the study.

	Using data analytics and data visualization to oversee the entirety of a clinical protocol or program gives sponsors and CROs an important tool to protect trial integrity. 

	It utilises the realms of data sponsors and CROs have at their fingertips to spot potential issues early on so they can make the informed decisions that will keep their studies on track.

	Data might not give us all the answers, but it can tell us which questions to ask-and therein lies the value of using statistics-based centralized monitoring to improve data quality in clinical trials.

Articles

Centralized monitoring offers sponsors and CROs the program-wide oversight they need to successfully develop the medications of the future, says CluePoints CEO, Francois Torche.

Using Statistics to Improve Data Quality and Maximize Trial Success

Data quality matters most in determining study success and ensuring patient safety. However, patient numbers continue to rise, CRFs are becoming more complex with each new study and, with the use of sites around the globe, these elements significantly impact the ability to guarantee the quality and integrity of clinical trial data while containing costs. Until recently, the standard paradigm for clinical site monitoring was to do routine site visits every four to 10 weeks with 100% source data verification (SDV) in most cases. This process typically accounts for 30% of a sponsor's overall costs but results in less than 3% of any data changes post-monitoring visit. This antiquated and inefficient paradigm costs the industry hundreds of millions if not billions of dollars. As budgets become more difficult to manage and data issues grow by the day, regulatory authorities and most in our industry no longer believe that traditional "on-site monitoring" and "100% SDV" are the best solutions. Humans are less efficient at detecting erroneous data than an automated process and are the most expensive resource employed in any trial. Computer algorithms, by contrast, are relentless in their interrogation of the data and are much more cost-effective.  

The FDA guidance for industry and the EMA reflection paper encourage the use of risk-based monitoring tools as well as increased efficiency in the form of reduced SDV. The documents bring to our attention the fact that sponsors should be focusing or "targeting" their on-site monitoring activities. One of the most common ways to do so is by remote monitoring. This means taking a look at the data off site and determining where the issues may be prevalent. They also refer to data management metrics and trending which includes things such as subjective key risk indicators (KRIs). Then, of course, there is central statistical monitoring. Sponsors need to look at their data in numerous ways in order to target on-site monitoring visits effectively. By doing so, the goal is not only to reduce 100% SDV levels but to de-risk their studies by improving data quality at significantly lower costs.  

In contrast with KRIs, which only indicate the potential for risk, independent central statistical monitoring determines the expected values of each variable by assessing and examining the data from all sites in order to identify statistical outliers. This concept has been perfected by CluePoints over the last 10 years using the patent-pending process within the SMART™ engine. Complex and proven statistical algorithms drill down into individual patient data to detect issues that could put the study at risk and create a road block to successful submission. This approach requires minimal work for the study team in gaining objective information in order to optimize on-site monitoring by targeting centers at risk. In addition, sponsors who strategically outsource to CROs are finding increased efficiencies by utilizing central statistical monitoring as an oversight tool to regularly check the quality of their data.  

Key risk indicators are applied because they focus on important known risk factors, for example, the proportion of AEs or SAEs collected. There is no question that the safety reporting in a clinical trial must be looked at very carefully. However, unlike other providers, CluePoints provides an independent, EDC-agnostic approach to central statistical monitoring that is based on the actual clinical data and not subjective indicators. The idea behind this is that all variables are indicative of quality—whether it is lab data, clinical data, baseline data, or treatment outcomes; everything goes into this analytical system and all data are deemed equally important. In a clinical trial, everything that is collected should be worth collecting, and therefore worth checking—central statistical monitoring ensures the ability to determine the quality and integrity of the data and ensure that monitoring efforts focus on errant sites quickly and efficiently.  

CluePoints has been utilized in several dozen studies with sponsors such as GSK Vaccines and Gilead Sciences. Brian Nugent, Associate Director, Clinical Operations at Gilead has explained his experience with CluePoints and how he and his study teams have practically implemented a central statistical monitoring solution in three quick and easy steps—setup, analysis and report. During the setup stage, the sponsor's data manager and CluePoints discuss the structure of the datasets and their contents and the data is uploaded. During analysis, CluePoints select statistical tests and variables, browse and filter results, and generate outputs. The sponsor is provided with a detailed report that lists any outliers. Data discrepancies are clearly defined and the graphs and summary results are well displayed and informative.  

In Gilead's first study with CluePoints, there were 120 centers and 660 tests conducted, generating 79,200 p-values. The tables of p-values were very detailed and CluePoints provided expert interpretive guidance to make this information easily digestible—both known and unknown risks and issues were detected in this study.  "Rather than to fight the fire once the fire started, we really wanted to uncover the data; which we did, and we addressed a number of things," Nugent said. "We now plan to integrate KRIs and central statistical monitoring into our everyday study quality and risk management plans and of course we expect to have improved data quality and certainly at a lower cost." 

Many companies already have KRIs in place and central statistical monitoring can further improve risk-based monitoring solutions by more efficiently detecting errors, sloppiness, tampering, and even fraud. When utilizing central statistical monitoring during a study, sponsors can take corrective action, such as that taken by Gilead, in a timely manner to prevent issues becoming deep-rooted and re-analyze data regularly as part of an iterative process. The sponsor or CRO can perform focused site monitoring visits, identify CRA and site retraining needs, or develop site tools to provide clarification on protocol and study processes. CluePoints can also be used as a final data quality check at the end of a study. 

During a recent webinar, CluePoints surveyed over 300 industry colleagues from clinical operations, data management, and statistics regarding their experience with risk-based monitoring and the benefits of a centralized statistical monitoring approach (Figures 1 and 2).  

is the CEO of CluePoints, 185 Alewife Brook Parkway, Suite 410, Cambridge, MA, e-mail: 

Central statistical monitoring can improve RBM solutions by efficiently detecting errors, sloppiness, tampering, and fraud.

Francois Torche

Articles

Using Statistics to Improve Data Quality and Maximize Trial Success

The Practical Implementation of Risk-Based Monitoring