In the context of multicenter clinical research, centralized monitoring (CM) is the most efficient way to ensure patient safety, trial integrity, and data quality.

 As it permits the study team to proactively detect anomalous data trends, CM improves the quality of the regulatory submissions with a direct impact on the time to marketing approval.

	Since publication of the regulatory guidance on risk-based monitoring (RBM) five years ago,

 the concept of CM has developed amid the emergence of technological enablers that make clinical research more data-driven than ever. Today, regulators encourage the use of CM in conjunction with on-site monitoring to oversee clinical trials.

 Despite its unique potential for improving the quality of clinical trials, CM can appear so technical that sponsors often elect to renounce its use in favor of costly and less efficient traditional monitoring methods.

	In reality, only a few concepts that are relatively easy to master-and which most life sciences professionals are already familiar with-are required to properly implement CM.

 In fact, to plan a CM strategy, one should be familiar with the concept of risk management, which involves identifying risks, estimating their potential impact, and devising efficacious mitigation strategies. Then, to perform CM, one needs to understand how simple statistics related to the means and the standard deviations can be used to detect outliers. Additional CM skills include the ability to detect scientific misconduct using the chi-squared distribution, which is closely related to the normal distribution.

	The objective of this article is to show that performing CM is relatively easy and accessible to any research professional inspired by the objective of overseeing trials with optimal efficiency while simultaneously saving on-site monitoring resources. The central monitoring techniques presented in this article can be implemented using readily available tools such as Microsoft Excel.

	Because CM is a tool within a risk management process, central monitors must first understand how to identify and mitigate risks. A risk assessment, which is an integral part of a risk management process, allows one to identify a protocol’s inherent scientific and operational risk factors, rate their respective potential impacts, and either eliminate them or develop risk mitigation strategies to control them efficiently. In the context of clinical trials, the risk assessment should focus on risks relevant to a subject’s safety, the trial integrity, and the data quality. A proper risk assessment is especially important as regulators require that sponsors document the rationale for a chosen central monitoring strategy.

	Key risk indicator (KRI) metrics are risk-factor correlates that can be calculated from the data available, and they are identified during the risk assessment process. While KRIs provide quantitative information, they offer a view that may lack context. As such, qualitative information obtained from communication with on-site monitors and study coordinators represent key risk information that should be used in conjunction with KRIs for the proper analysis of risks and the choice of mitigation actions. The purpose of central monitoring is not only to measure and reduce risks but also to provide perspective to the processes under review so that the most effective control strategy can be adopted.

Each KRI metric has associated values corresponding to limits, also known as tolerance thresholds, which are determined during the risk assessment process. When site-specific metrics fall beyond set limits, the root cause should be analyzed by central monitors and mitigation actions, devised during the risk assessment process, implemented as necessary. Figure 1 illustrates how a typical site-specific KRI metric (ex. error rate) may differ from the rest of the population and fall beyond set limits, thereby triggering local specific mitigation actions. Figure 1 also illustrates how limits may be changed according to the observed values as a study progresses.

	Using a risk matrix requires judgment for the probability of occurrence, the potential impact and the detectability of each risk factor, in order to generate a score that permits ranking the KRIs according to their importance. This method of assessing risks is used in the risk assessment categorization tool (RACT) published by TransCelerate BioPharma Inc.

 It should be noted that the relative importance of each KRI does not influence the level of oversight on them but rather serves as a scale for the intensity of the mitigation actions put in place. For example, addressing a KRI of low importance that falls outside its normal limit may require nothing more than emails and phone calls to the site, whereas addressing a KRI of high importance that falls outside its critical limit may require more aggressive and resource-intensive approaches such as the dispatching of on-site monitors or the initiation of corrective and preventative action (CAPA) processes.

	The relative risk importance changes as the study progresses. For example, the enrollment rate at the beginning of a study is an important indicator of trial viability, but after the enrollment is closed, it becomes only an indicator of high enrollers, which does not directly impact trial integrity. In comparison, a high query rate at the beginning of a study might be addressed by retraining research coordinators without significant consequences. But at the end of the study, it may directly impact study quality and the time to database lock. Accordingly, risk assessment should evaluate a study at different phases and the focus of risk management should change with time.

	Table 1 includes the most common clinical trial KRIs and the typical output of a phase-relative risk assessment. Note that additional protocol-specific KRIs identified through a risk assessment process should be considered in different trials.

	Communication between different stakeholders is instrumental to the traceability of the CM process. The periodic central monitoring report should include the site-specific risk factors that are outside tolerance thresholds at the time of review, the specific metrics values, and their variations since the last review. To achieve a traceable central monitoring process, a concise analysis of the reasons why the outlying values are observed and the mitigation actions which are implemented in response should also be included in periodic reports. CM reports specifically serve to indicate how the situations progress in response to mitigation actions. Reporting frequency may vary from weekly to monthly depending on the data acquisition rate, which typically is higher during the study start-up period and lower during the close-out period.

Centralized monitoring: Statistical background

	The FDA defines CM (aka “central statistical monitoring”  or “statistical data surveillance”) as 

“a remote evaluation carried out by sponsor personnel or representatives (e.g., clinical monitors, data management personnel, or statisticians) at a location other than the sites at which the clinical investigation is being conducted.”

 Essentially, CM is used to perform the risk analysis part of a risk management process and involves performing calculations on an ongoing basis to discriminate normal from abnormal data. Today’s technological enablers allow for the calculation of statistics from the accumulating data and it is thus essential for central monitors to be able to interpret the results correctly. The following sections cover the statistical notions that central monitors should be familiar with.

	The normal distribution is the most important concept in statistics and in order to evaluate the normality of calculated metrics, central monitors must understand its parameters-namely the mean and the standard deviation. The mean represents the anchor of normality and the standard deviation represents the stretch from the mean beyond which an observation may be considered relatively abnormal. Accordingly, the standard deviation and its multiples can be used to set tolerance thresholds, also known as outlier limits, beyond which an observation may be considered abnormal. It must be noted that judging what is normal and what is abnormal is a subjective endeavor, but the mean and the standard deviation remain the best parameters on which to base one's judgment. As described in the next sections, the mean and the standard deviation can be used to standardize observations as z-scores and the associated probabilities (p-values) of observing such z-scores.

	Central monitors should take a moment to review Figure 2 and realize how the mean and the standard deviation relate to the z-scores and their p-values. The following section aims to clarify the implications of the term “normal,” statistically speaking, in the context of multi-centered clinical research.

	The z-score corresponds to the number of standard deviations an observation is from the population mean. It is calculated as follows: Z = (χ – μ) / (σ / √n) where χ is the value for which the z-score is calculated, μ is the population’s mean, and σ is the population’s standard deviation, while n represents the sample size that corresponds to the number of observations made to compute the sample mean (e.g., a subject’s mean blood pressure). When calculating a Z-score for a single observation (e.g., a site-specific KRI), n=1 and, therefore, Z = (χ – μ) / σ.

	Once standardized as z-scores, the observations can easily be compared to the mean and to each other. Namely, Z-scores have associated p-values that can be used to judge the normality of the observed values. The z-score p-value is the probability of observing a value equal or more extreme than the value actually observed (χ), by chance alone and assuming that the population from which the value was obtained is normally distributed. The smaller the p-value, the less likely a person would be to observe a value “as extreme” as the one observed.

	The z-score and p-value are very useful in the context of identifying outliers in clinical research datasets. Statistical software, including Excel, can be used to obtain p-values that correspond to the probability that an observation would be smaller (right-tailed p-value), or larger (left-tailed p-value) than an observed value. Figure 3 illustrates common scenarios.

	The cumulative probability corresponds to the left-tailed probability of observing a value as small as, or smaller than, the observed value. As such, the smaller the cumulative probability, the further to the left of the population an observation is. On the other hand, the larger the cumulative probability is, the further to the right of the population an observation is. For example, if a site-specific KRI metric value corresponds exactly to the population KRI mean value, its associated cumulative probability will be 50%. If another site-specific metric has a cumulative probability of 100%, it means that approximately 100% of the other sites’ metrics values fall below the KRI value of that site and as such, it is safe to call it an outlier. In the context of central monitoring, the cumulative probability is particularly useful because outliers are often observations that are greater than the rest of the population.

	CM should not be limited to the monitoring of KRIs. Other methods, such as the ones presented ahead, should be used to detect scientific misconduct. Deliberate data fraud is rare but can have significant impact on trial integrity. One straightforward way to fabricate data is to take existing data and copy them within or across study subjects. Such a data propagation method results in certain values occurring more often than others; a simple way to detect this type of data fabrication is to calculate the frequency of each observation. As such, frequency analysis can effectively detect if vital signs taken from only one subject were copied into two subjects' charts or if a single blood sample was split into two before being sent to the laboratory. In most cases, however, fraudsters are unlikely to be so careless as to copy data without modifying some of the values. Fortunately, there are other ways to detect fraud, which are harder to evade given the predictability of human nature.

	The chi-square distribution is graphically different from the normal distribution, but it can be used in the same manner to assess the normality of values that follow its distribution pattern. Importantly, the sum of squares, which is defined as the squared differences between observed values and expected values, follows a chi-square distribution.

The chi-square statistic (χ²) can thus be used to evaluate the difference between what is observed and what is expected as being normal. It is calculated as follows:

	Like the z-score, the χ² statistic has associated p-values. Figure 4 shows how the χ² statistic (right-tailed p-values) corresponds to the probability of observing a difference between the observed and the expected values that would be as large or larger than the actually observed difference. The larger the χ² p-value, the closer the observed values lie from the expected ones. On the other hand, the smaller the χ² p-value, the farther the observed values will be from the expected ones. The degrees of freedom (DF) of the chi-square distribution correspond to the number of observations taken into account in the calculation of the χ² value. Figure 5 shows how the shape of the χ² distribution changes according to the degrees of freedom. Note that the degrees of freedom also correspond to the mean chi-square value for the different χ² distributions.

	Because expected values are based upon calculated averages and real data never lie too close or too far from the expected values, the χ² p-value may indicate fraud if it is either too small or too large. Indeed, counterfeiters are bad at mimicking the randomness of nature

 and when one observes data that are either too far or too close to expectations, it is reasonable to suspect fabrication. As described ahead, terminal digit analysis and inlier analysis are two types of analyses that use the χ² statistic to evaluate if data lie too far or too close to the expected values, respectively.

Too far from expectation: Terminal digit analysis

	If we consider measurements that have two or more digits, we are expecting the frequencies at which the digits from 0s to 9s appear in the rightmost position to be approximately equal. To perform terminal digit analysis using the chi-square “goodness of fit” test with the χ² formula indicated at left, the 

 values correspond to the number of times each digit, from 0 to 9, appears in the dataset of interest while the 

 values for each digit corresponds to the number of measurements taken into account divided by 10, since each digit is expected to appear with equal frequency. The degrees of freedom associated with the terminal digit analysis is 9, considering the sample size of 10 possible categories (digits 0 to 9) minus 1. The calculated χ² p-value indicates the “goodness of fit” between the observed last digit distribution and a perfectly uniform last digit distribution.

Figure 6 illustrates how the χ² p-value relates to the last digit distribution. Note that if certain digits appear in the terminal position more often than others, it might be caused by measuring instruments or by normal practice that requires rounding.

 In other cases, it might be because of fraud, as humans tend to favor certain digits when fabricating numbers.

 The latter instance constitutes reprehensible behavior that has significant impact on the trustworthiness of all data provided by the site of interest.

Too close to expectations: Inlier analysis

	Real subject-specific data are expected to vary to a certain extent from one physician’s office visit to the next. An inlier analysis can be used to detect whether this is the case or not. A single fabricated variable-let’s say heart rate-may remain plausible on its own, but if considered with other fabricated variables such as respiration rate, systolic BP, diastolic BP, and temperature, the combined data are likely to exhibit an abnormal multivariate pattern that can be detected statistically.

 Inlier analysis specifically evaluates how close to their respective means a set of multivariate observations lies and suggests fabrication if those observations, taken together, lie abnormally close to their respective mean. Specifically, if a subject’s data have been chosen to mimic real data, its measures will consistently lie close to an anchor value,

 such as the population’s mean, and the sum of the differences between its observed measures and the population means for those measures will be smaller than the sum of differences calculated for the rest of the population. The sum of squared z-scores follows a chi-square distribution and it can be used, in place of the χ² statistic, to obtain p-values corresponding to the probability of observing a given sum of squared z-scores.

 The following steps describe how to perform a multivariate inlier analysis:

 Choose the variables to evaluate. In the context of clinical research, variables that can be easily fabricated include physical examination and vital signs data.

 Calculate subject-specific z-scores for each variable using subject-specific mean scores (χ), population`s means (μ) and population standard deviation (σ) using Z = (χ – μ) / (σ / √n) where n represents the number of subject-specific samples used to calculate subject-specific means χ.

 Square those z-scores and add them up to have subject-specific summed Z² values. Subject-specific summed Z² values should follow a χ² distribution with a degree of freedom corresponding to the number of variables considered in the calculation of the subject-specific summed Z².

 An inlier can be identified as a subject with an unusually small summed Z² value and an associated p-value can be obtained using the subject-specific summed Z² value as a χ² value.

	To visualize the inlier analysis, one can graph all subject-specific summed Z² values as cluster points along with anchor points corresponding to the number of variables taken into account for the calculation of the subject-specific summed Z² values (the degrees of freedom). As stated, the degrees of freedom represent the mean summed Z² value and this value corresponds to the normal distance from their respective means. Inliers will be apparent on such graphs as points that lie unusually far to the left from the mean summed Z² value, relative to other subject-specific summed Z² values. For better visualization, one can transform all summed Z² values using natural log function.

	In Figure 7, the subject-specific natural log of summed squared z-scores were calculated using five variables, including heart rate, respiration rate, systolic BP, diastolic BP, and temperature. The normal distance from the multivariate mean is indicated by the red dotted line that corresponds to the natural log of the degree of freedom. The graph indicates that two subjects at site 17 and 18 have measures consistently close to their respective means. With such a scenario, central monitors have good reasons to inquire further with the site staff as to why those subjects’ measures are so close to their respective means.

	It is important to consider that there is no single universally applicable or generic outlier detection approach

 and a direct confirmation of discrepancy or proof of fraud is seldom obtained from statistical evidence alone. Abnormal data analysis only serves as support for further investigation. In addition, because statistical power is dependent on sample size, it is important to consider that a large amount of false positive signals may be observed when the data sampled is small, such as is the case at the beginning of trials or when trials are of small size.

 Thus, site-specific metrics should always be interpreted with the consideration of sample size. One may elect to wait for a sufficient amount of data to be accumulated at a site before initiating analysis for that site. Considering these limitations, CM should not rely only on statistical algorithms. Simple analysis such as checking if examination dates correspond to weekend or holiday dates can serve the purpose of flagging suspicious sites.

 Also, the fact that a given site does not manage to generate enough data to be considered for analysis may constitute a signal in itself.

	The core objective of CM is to support a risk management process that aims to ensure subjects’ safety, trial integrity, and data quality in the most efficient way. CM can undoubtedly play an important role in increasing the quality of clinical trials as it allows sponsors to intelligently decrease the amount of costly on-site monitoring. This is very important since the overall cost of monitoring can represent up to one-fourth of trial costs

 and the efficiency of monitoring efforts, including onsite and central monitoring, has a direct impact on the cost of clinical trials and the price of treatments that ensue. Abnormal data patterns can be readily detected even by simple statistical methods, and the skills required to perform CM do not necessitate extensive training nor the most advanced technology. In fact, most clinical research professionals have already been exposed to the statistical notions covered in this article and can carry on the task of CM with readily available tools such as Excel. Appendix 1 contains a list of functions and formulas which can be used to perform the types of analyses discussed in this article using MS Excel.

is Clinical Data Manager at EndoCeutics Inc. and Consultant at XLSMetrics Inc.; 

 is Senior Director of Data Management at Target Health Inc. and adjunct faculty at New York Medical School; and 

		 Dimitris K. Agrafiotis, Victor S. Lobanov, Michael A. Farnum, Eric Yang, Joseph Ciervo, Michael Walega, Adam Baumgart, Aaron J. Mackey. Risk-Based Monitoring of Clinical Trials: An Integrative Approach. 

		 Baigent, C., Harrell, F.E., Buyse, M., Emberson, J.R. and Altman, D.G. (2008) Ensuring trial validity by data quality assurance and diversification of monitoring methods. 

		 George S.L., Buyse M. Data fraud in clinical trials. 

		 Venet D, Doffagne E, Burzykowski T, Beckers F, Tellier Y, Genevois-Marlin E, et al. A statistical approach to central monitoring  of  data  quality  in  clinical  trials. 

		 FDA. Guidance for industry oversight of clinical investigations – a risk-based approach to monitoring. 2013.	

		 European Medicines Agency. Reflection paper on risk based quality management in clinical trials. 2013.	

		 Transcelerate BioPharma. Position Paper: Risk-Based Monitoring  Methodology. 2013.	

		 Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). 09-Nov-2016.	

		 Ashok Ghone. Centralized Monitoring--A Smart, Reliable Approach. 

		 Linda B. Sullivan. The Current Status of Risk-Based Monitoring. 

		 Buyse M, George SL, Evans S, et al. The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials. 

		 TransCelerate BioPharma. Position Paper: Risk-Based Monitoring Methodology. 2013.	

		 FDA. Guidance for Industry oversight of clinical investigations – a risk-based approach to monitoring. 2013.	

		 5. Buyse M, Evans SJW. Fraud in clinical trials. In: Armitage P, Colton T, editors.

. John Wiley; Chichester: 2005. pp. 2023–2031.	

		 Evans S. Statistical aspects of the detection of fraud. In: Lock S, Wells F, Farthing M, editors. 

. 3. BMJ Publishing Group; London: 2001. pp. 186–204.	

		 Everitt, B.S. (2002) The Cambridge Dictionary of Statistics, CUP, ISBN 0-521-81099-X	

		 Dlugosz, Stephan & Müller-Funk, Ulrich. (2009). The value of the last digit: Statistical fraud detection with digit analysis. 

. 3. 281-290. 10.1007/s11634-009-0048-5.	

		 Bolton RJ, Hand DJ (2002) Statistical fraud detection: a review (with discussion). 

		 E. H. Hsü (1948) An Experimental Study on “Mental Numbers” and a New Application, 

, 38:1, 57-67, DOI: 10.1080/00221309.1948.9711768	

		 Boland, Philip J., and Kevin Hutchinson. 2000. “Student Selection of Random Digits.” 

		 Murray G, Weir C. Fraud in Clinical Trials. Detecting it and preventing it. 

. Volume 8, Issue 4, Version of Record online: Nov. 25, 2011.	

		 Al-Marzouki Sanaa, Evans Stephen, Marshall Tom, Roberts Ian. Are these data real? Statistical methods for the detection of data fabrication in clinical trials. 

		 Wen SW, Kramer MS, Hoey J, Hanley JA, Usher RH. Terminal digit preference, random error, and bias in routine clinical measurement of blood pressure. 

. edited by Carol K. Redmond, Theodore Colton - 204.	

		 Buyse M, Evans SJW. Fraud in clinical trials. In: Armitage P, Colton T, editors. 

. Volume 8, Issue 4, Version of Record online: Nov. 25,  2011.	

		 Sheskin D.J. Handbook of Parametric and Nonparametric Statistical Procedures. Mathematics, probabiliy and statistics. CRC. 4th Edition. 1997: 118.	

		 Randall E. Schumacker, Learning Statistics Using R, SAGE Publications, Jan. 28, 2014. 118-122.	

Articles

Feature Article

Examining the practicality of implementing CM techniques to drive trial oversight efficiency while saving on-site monitoring resources and costs.

The Basics of Clinical Trial Centralized Monitoring

Streamlined clinical trials and continued patient involvement in product research continue to advance the development and market approval of more innovative therapies for rare conditions. FDA has approved more than 60 orphan indications for new and existing products this year, building on 80 approvals for 2017, according to a study from the IQVIA Institute for the National Organization for Rare Disorders (NORD) (see 

https://www.iqvia.com/institute/reports/orphan-drugs-in-the-united-states-growth-trends-in-rare-disease-treatments?utm_campaign=2018_InstituteFallReports_Insititute_HF&utm_medium=email&utm_source=Eloqua&utm_campaign=2018_InstituteFallReports_Insititute_HF&utm_medium=email&utm_source=Eloqua

). FDA approvals of new treatments are “shattering previous records” based on continued innovation in research in this area, observed NORD President Peter Saltonstall in conjunction with NORD’s annual Rare Disease Summit in Washington, D.C. in October.

	Important FDA initiatives aim to increase the use of biomarkers and provide early advice to sponsors to make clinical trials more efficient and less costly for developing targeted therapies, including those for rare conditions. An important draft guidance issued in October encourages sponsors to use “minimal residual disease” (MRD) as a biomarker in testing drugs or biologics to treat certain blood cancers (

https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM623333.pdf

). This general measure of tumor burden in clinical trials, observed FDA Commissioner Scott Gottlieb in announcing the advisory, has the potential to expedite product development by assessing a patient’s response to treatment or the risk of future relapse. The policy supports using MRD to enrich clinical trial populations and define treatment arms, with an eye to further developing the marker as a surrogate endpoint.

	The agency also finalized a guidance on identifying treatments that address underlying factors that may contribute to disease, such as rare molecular changes present in small subsets of patients (

https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM588884.pdf

). This advisory aims to help sponsors enroll in clinical trials those patients with genetic markers that indicate likely response by providing

	information on grouping patients with different molecular alterations and on strategies for evaluating the benefits and risks of targeted therapies to treat diseases with rare molecular alterations.

	Sponsors, researchers, and patients can obtain advice and information through a new FDA web portal on “developing products for rare diseases,” which aims to provide a “central home” on FDA regulatory initiatives and programs in this area (

https://www.fda.gov/ForIndustry/DevelopingProductsforRareDiseasesConditions/default.htm

). A guidance published earlier this year, moreover, clarifies FDA’s orphan drug designation process, which opens the door for sponsors to gain expedited reviews and financial support for clinical studies and other processes (

https://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM589710.pdf

). To help shape early R&D programs for therapies for rare diseases, FDA is encouraging sponsors to sign up for pre-investigational new drug (IND) meetings to discuss studies that utilize smaller patient pools. Another FDA guidance spells out what manufacturing and preclinical data is needed to make such early meetings useful, and what expedited programs, standards, and support is available to researchers seeking to test new treatments (

https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM623293.pdf

	A broader agency reorganization plan announced by Gottlieb in July aims to better coordinate advice and oversight for orphan drugs by establishing an Office of Clinical Policy and Programs (OCPP) that supports cross-cutting clinical programs involving multiple FDA medical product centers. Under the new structure, OCPP will report to FDA Deputy Commissioner Rachel Sherman and include the Office of Orphan Products Development (OOPD), the Office of Pediatric Therapeutics, the Office of Combination Products, and a new Office of Clinical Policy. OCPP also will oversee an expanded agency-wide program for patient affairs and healthcare providers to enhance engagement with these external stakeholders.

	FDA officials plan to expand OOPD’s staff and programs under Janet Maynard, newly named acting director of the office. And FDA’s new orphan product council is meeting regularly and assisting in the development of additional guidance documents, reported Sherman at the NORD summit.

	These initiatives stop short of establishing a separate FDA center of excellence for rare diseases, as sought by patient advocates in this area. But the planned organizational changes will mean that the oversight of new therapies for rare diseases, said Sherman, “will never become orphans” at FDA.

Jill Wechsler is the Washington Correspondent for Applied Clinical Trials

News

FDA initiatives aim to increase biomarkers and early advice to sponsors for more efficient and cost-effective clinical trials for developing targeted therapies, including orphan therapies.


FDA Initiatives Advance Orphan Therapies

Since the turn of the decade, pharmaceutical and diagnostic companies have collaborated more than ever before to advance the commercial and clinical value of precision medicine. Today, the need for these collaborations is greater than ever before, as healthcare tackles one of medicine’s greatest challenges: Alzheimer’s disease (AD). According to the Alzheimer’s Association, more than 5.7 million Americans live with AD, and the number is projected to increase to 14 million by the year 2050.

	Tragically, AD remains virtually untreatable. Currently available medications have limited impact on progression and virtually no impact on the disease’s severity.

	Several barriers have stymied successful research and commercialization of drug targets. In fact, despite a drug pipeline with more than 100 treatment agents, not one drug has been approved to treat the underlying cause or slow the progression of AD since 2003. A 2018 analysis of Alzheimer’s drug development published in 

Translational Research & Clinical Interventions 

noted that eight agents listed in Phase III in 2017 failed in clinical trials. Six drugs listed in Phase II last year are no longer in development, and trials for five particular agents in Phase I in 2017 were either completed or terminated and are not listed in the 2018 pipeline.

	However, recent announcements from regulatory bodies and public health organizations offer great hope that promising new therapies can be accelerated through trials and approvals to the point of clinical care.

	In April, the National Institute on Aging and Alzheimer’s Association (NIA-AA) released a new research framework  that proposes the use of biomarkers to assess the presence of AD, whether symptoms are present or not.

	While not currently intended by clinical use, this new framework is expected to build upon research discoveries that certain biomarkers are predictive of AD by as much as 20 years. Specifically, the NIA-AA “biological construct” proposes to use three general groups of biomarkers-beta-amyloid, tau, and neurodegeneration, or neuronal injury (AT(N))-in a new AD classification system that covers the continuum of the disease.

	The NIA-AA framework comes on the heels of the FDA’s draft guidance in February on using biomarkers in an approval pathway for new AD drugs if drug developers could hit acceptable biomarkers that indicate the drug is working.

	Taken together, these developments could lead to an improved understanding of the disease process and the sequence of events that lead to cognitive impairment and dementia-and an entirely new way of accurately measuring clinically meaningful cognitive and functional outcomes based on biological data. Most of all, they hold the potential to advance AD drug R&D and potentially surmount the many barriers to success in this area.

	But the use of biomarkers in AD research will also require new types of collaborations by pharmaceutical companies and R&D partners-including diagnostic providers. While the new framework does not recommend the use of current biomarkers for clinical applications, some specialists today order them as clinical tests to enhance their assessment of patients. As a result, diagnostic providers specializing in AD, cognitive impairment, and dementia likely already perform these biomarker tests and may have vast data sources on which to glean insights useful in research settings. 

	AD drug research is a long-term game. In order to minimize risk and optimize a smooth and successful transition from lab discovery to clinical trial to regulatory approval, pharma and diagnostic companies should design a long-term strategy to turn science and data into actionable medical innovations. These include, potentially, companion and complementary diagnostics for future AD therapies. Here, a diagnostic provider can help generate the right data to support regulatory approval and payer support post-market launch.

, MD, PhD, Medical Director of Neurology, Quest Diagnostics 

A Closing Thought

Pharma and diagnostics companies need to strengthen alignment in Alzheimer’s disease research-to help turn science and data into actionable medical innovations.

Closer Pharma-Diagnostic Collaboration is Key to Alzheimer’s Drug R&D

Five years ago, the FDA and the European Medicines Agency (EMA) released final guidance to change clinical trial oversight methodology from on-site visits using source data verification (SDV), the gold standard for more than 30 years, to a risk-based monitoring (RBM) approach.

	Implementing this guidance created two daunting challenges to reconcile as follows:

		No standard RBM definition or standard way to implement RBM exists; these myriad of definitions and implementation approaches correspondingly represent different levels of effectiveness to identify “errors that matter.”

		No head-to-head comparisons exist that compare different trial oversight methods.

	This lack of scientific data on trial oversight effectiveness is a critical unmet clinical research need. It affects more than 100,000 research participants per year and their healthcare providers.

	This article represents a prospective analysis comparing the effectiveness of using traditional SDV versus one method of RBM (i.e., the MANA Method). We identified the specific RBM method used herein due to inconsistent RBM definitions and RBM implementation methods, and the varying levels of effectiveness for all the different RBM approaches.

	PaxVax conducted a Phase IV vaccine trial in approximately 500 subjects at nine U.S. sites. 

	The study was conducted using electronic data capture (EDC). The trial was approved by an institution review board (IRB) and each subject signed an IRB-

	approved informed consent prior to participating. Subjects received one dose of the study vaccine. Participants collected any changes in health for nine days in a paper diary aid, and sites entered the results into the EDC. Each research site maintained its own informed consents and site regulatory binders.

	Site monitors visited the research sites monthly and spent approximately 72 days on-site conducting SDV of the trial data. PaxVax’s senior management team (i.e., medical monitor; senior director, biostatistics; director of data management and statistical programming) reviewed the data monthly to identify trends or data errors that would be followed up by the site monitor. 

	MANA RBM modified its risk-based monitoring and remote trial management system (i.e., the MANA Method) to initiate an RBM approach for this study that began after 5.5 months of trial conduct (approximately 500 subjects already enrolled). To fully implement the MANA Method, additional trial oversight and remote document review, including informed consents and site regulatory documents, would also have been implemented and evaluated.

	In this pilot study, MANA RBM independently and remotely reviewed and used the existing trial data available electronically to determine whether errors and trends could be identified faster and more comprehensively than using the traditional SDV method. Analysis of informed consents, regulatory documents, and source documents were not included in this pilot study because the documents were not available electronically.

	MANA RBM first conducted a proprietary risk assessment service based on the protocol. It then designed proprietary study-specific reports and data visualizations to evaluate the high-risk data and processes identified during the risk assessment. The basic categories included: efficacy endpoints, safety assessments, investigational product (IP) management, and human subjects’ protection.

	Trial data was imported from the EDC platform into JReview, hosted by Integrated Clinical Systems, Inc. MANA RBM designed its proprietary Subject Profile Analyzing Risk (SPAR) tool to provide an integrated visualization of the high-risk data for each subject over time and trained the remote monitors in its use. SPAR configuration is unique for each trial based on the critical issues identified during the risk assessment. Additional proprietary, custom reports were also developed to support protocol-specific analysis of high-risk data and processes and trends.

	All review was performed independently of the EDC system and based on MANA RBM data analytics. Results of the review were captured in a separate, proprietary MANA RBM Site Tracker Analyzing Risk database (STAR); MANA RBM developed this tool to conduct study quality oversight. Subject review was documented in JReview.

	MANA RBM conducted review using its remote quality management approach as shown in Figure 1. The MANA Method splits the review process into tiers. Remote site monitors focus on subject review and high-risk data and process oversight at the subject level. Central review focuses remote review on trend analysis by evaluating data across subjects at a single site and across sites.

	The pilot study compared SDV versus the MANA Method in the following areas:

	3) Identifying trends in data affecting trial conduct and/or results

Risk assessment and development of protocol-specific reports: 

MANA RBM conducted the risk assessment and implemented the SPAR within two weeks of uploading the data into JReview. Additional custom reports were developed over eight weeks. These reports included customized, cross-database reports and trend analysis of high-risk data and processes.

Once the SPAR was available, reviewers began reviewing the data immediately. MANA RBM split the subject review. An experienced monitor reviewed half of the subjects during the first month and the data reviewer, new to subject review, reviewed the other half of the subjects during that month. The following month, the subject reviewers switched subjects to review to allow evaluation of oversight by remote monitors with different training and experience. The lead monitor performed quality control (QC) oversight of each of the remote monitors to provide immediate feedback on items missed or documentation correction.

 MANA RBM’s remote site monitors identified critical deviations using the SPAR and accompanying high-risk reports. The MANA Method identified critical deviations not previously identified by the sponsor’s on-site monitors.

 Using remote methods, the monitoring team could have identified deviations faster and earlier than using SDV and on-site visits. Within six weeks, two rounds of review of all critical subject data were completed and all deviations for critical data were identified.

 Differences in classifications of deviations as major or minor were identified between the MANA RBM remote monitoring team and the on-site monitors. This resulted in challenges when comparing the total numbers of deviations. These totals were similar and there were no major deviations discovered by the sponsor’s on-site monitors that the MANA Method did not also identify remotely.

This study was conducted with paper memory aids and transcription by the research sites. Since this was a pilot study, sites were not asked to convert the paper memory aids to certified copies, which would have allowed remote review.

	To evaluate whether there were findings that the MANA Method would not have been able to identify without on-site visits or using eDiaries, MANA RBM reviewed the queries related to subject diaries generated from the study. The MANA RBM team identified 300 queries associated with SDV. Table 1 shows the distribution of the queries and illustrates how remote review would have identified all critical findings with the use of eSource or certified copies of the diary aids. Important data is defined as data that would affect subject safety or analysis of efficacy.

	MANA RBM reviewed the important data remotely from source review query rates and found that two sites had much higher query rates (i.e., 2-10 times the rates of the other sites), as shown in Table 2 (see below). This information, if known to the sponsor, would have allowed it to determine the need for continued on-site SDV and, if needed, focus SDV, additional training, or other strategic considerations on only two sites instead of all sites.

	Central review and trending was conducted in addition to subject review. This review occurred during the second and third month of the pilot study using proprietary reports designed specifically for the high-risk areas identified in the study’s protocol. From this review, MANA RBM central monitors recognized several trends that could have significantly impacted this study as follows:

	1) Deviation evaluation identified at least one trend that could have enabled more evaluable subjects. A higher rate of out of window visits existed for one site. While not usually considered a major deviation, the timing of this critical visit represented the collection point for primary efficacy data. The MANA Method would have identified and corrected this error sooner, leading to more evaluable subjects. On-site monitors did not identify this issue. The PaxVax senior clinical research management team identified this site deviation at its monthly review meeting while the MANA RBM reviewers discovered this issue immediately upon performing central review.

	2) A vital signs evaluation identified one site that had issues with collecting vital signs; specifically, collecting manual temperatures. Analytics identified this issue by using the differences in the mean values and a scattergram of actual values. This indicated a process issue that could have significant impact on future studies where immediate measures of temperature elevation after an IV injection could have been under-reported. Only the MANA Method remote central monitoring approach identified this issue.

	3) Incomplete dosing represented another area where variability existed in performance across sites. Since sites “batch” (i.e., enroll large groups of subjects over a few days) their dosing for vaccine trials, identifying this issue rapidly may have increased the number of subjects that took the complete dose. The senior clinical research management team noted this issue at its monthly meeting. The MANA Method central monitors noted it immediately upon review. On-site SDV did not identify this issue.

	4) Variability in reporting on adverse events of special interest occurred across sites. One site routinely ranked lowest or second lowest among the sites across the reported eight adverse events. While it was not clear if an issue existed, it was a trend that should have been evaluated to understand the processes by which this critical assessment was being conducted. Only the MANA Method central monitoring approach identified this finding. In Table 3, all sites with at least 15 subjects enrolled were evaluated on the ranking, across sites, for severity of the adverse events of special interest using z-scores (i.e., the number of standard deviations from the mean). One site routinely ranked subjects either the lowest or second lowest in severity, while one site routinely ranked subjects at a higher severity.

	1) The early termination rate was higher at one site than at the others. The reasons for early termination were not different across sites. No action was recommended at that time.

	The MANA Method enabled remote, comprehensive subject review of the high-risk data and processes to begin within two weeks of starting the project. No minimum data requirement was required to begin the review after a subject’s visit data was entered.

	Central trend analysis began approximately two weeks after remote subject review and identified additional data errors that could be corrected quickly. This rapid review could have eliminated errors in several aspects of study conduct as follows:

		Large number of out-of-window visits for critical assessments at one site

		Large number of incomplete dosing at two sites

		Confusion about the definition of diarrhea versus loose stools across sites (about 85 queries)

		Errors identified early facilitate site retraining, thus, reducing the future workload for sites and study staff.

	The sponsor assigned eight months of resources to the study as follows:

		1.75 full-time-equivalent (FTE) data manager (DM), a lead DM, and a programmer (study conduct only).

		3 FTE monitors, including a lead monitor (72 days of on-site monitoring).

		Senior management: four senior managers met monthly for four hours. Prep time for the meetings estimated as 40 hours per meeting. A second monthly meeting reviewed deviations. It took approximately 10 hours of senior management and data management resources.

	Design, build, and validate study-specific reports in JReview: 2 FTEs for two months.

	The reviewer (“monitoring”) resources were much smaller than used in a traditional trial as follows:

		1 data reviewer (100 hours)-review time averaged seven minutes/subject

		1 monitor (100 hours)-review time averaged seven minutes/subject

		Quality control of monitor and DM performance (20 hours)

		On-site monitoring: Time to conduct subject safety oversight (onsite SDV versus monitor remote review) resulted in a savings of at least 83% of monitoring time, since only the on-site monitoring time was used for this comparison.

		Data management review-data oversight took 100 hours versus 1.75 FTE (630 hours for two months). When data are cleaner, errors corrected earlier, and central oversight identifies the critical data trends, the time to raise and close queries is significantly decreased. In addition, when central monitoring oversight was used, the time to create the materials for senior management review (40 hours per month) could have been significantly decreased.

		The 20 hours of central monitor review would have saved senior management over 60 hours per month. This results in approximately 1.5 weeks of savings for senior management per month.

Pilot study compares a risk-based monitoring and remote trial management method with traditional on-site source data verification for trial oversight.

Risk-Based Monitoring Versus Source Data Verification

	October offered a striking spectacle of contrasts in Europe’s ponderous attempts to construct a comprehensive policy on health.

	On the one hand, the month opened with calls from a cross-party group of members of European Parliament (MEPs) seeking action from the European Commission in support of orphan drugs and rare-disease patients. “What measures has the Commission taken so far to ensure accurate and timely diagnosis of rare diseases,” ask the MEPs. In addition, they pointed out, patient access to medicines for rare diseases varies widely across Europe-so what is the Commission doing to promote the availability of affordable medicines to treat rare diseases, they demanded.

	Throughout the month, groups as diverse as the European Alliance for Personalized Medicine and the Multistakeholder Pediatric Strategy Forum have been urging vigorous

	action to promote targeted treatments, close collaboration on dialogue between regulators and drug developers, and early scientific advice-with much of the discussion centered on the emerging plan for coordinated European-level health technology assessment (HTA).

	Conversely, the month was also marked by a crescendo of concern that health policymakers may be looking through the wrong end of the telescope with their traditional focus on treatment. Right at the start of October, one of Europe’s biggest annual health policy gatherings, the Gastein forum, named for the mountain resort in Austria where it takes place, heard plenty of calls for a bigger and better European health policy-but with an emphasis on wider access and greater equality, and with a focus on prevention rather than treatment.

	A series of prominent healthcare policymakers, from World Health Organization (WHO) Regional Director for Europe Zsuzsanna Jakab, to Director of the European Center for Disease Prevention and Control Andrea Ammon, underlined what they saw as the need for a multisectoral, societal, and integrated response to health. There were repeated invocations of the United Nations’ Sustainable Development Goals (SDGs), and calls for Europe to lead by example in the program’s global ambitions to tackle poverty, inequality, climate, environmental degradation, and injustice.

	EU’s Commissioner for Health Vytenis Andriukaitis has taken as his theme “the importance of our health-in-all-policies approach.” He told a WHO meeting in Kazakhstan: “We

	need a much stronger focus on health promotion, protection, and disease prevention.” And Andriukaitis has developed the theme throughout the month, with speeches endorsing

	the merits of a broader approach. The mantra is that the health sector cannot succeed alone. “We must address all the risk factors in a more holistic way: obesity and unhealthy nutrition, lack of exercise, tobacco, alcohol abuse, and also wider factors such as working conditions, unhealthy housing, and environment pollution” he said.

	Andriukaitis followed a similar line at the G20 Health Ministerial Meeting in Argentina where childhood nutrition was on the agenda, and at the recent UN high-level meetings on non-communicable diseases and tuberculosis.

	It is more than just a straw in the wind that the EU has set up a new steering group on health promotion and prevention of non-communicable diseases that is tasked with identifying priority areas for action and promoting exchanges of policies and practices between countries. Nor is it entirely a coincidence that it has chosen nutrition and physical activity as one of the areas for priority implementation.

Peter O'Donnell is a freelance journalist who specializes in European health affairs and is based in Brussels, Belgium 

All of Europe, from MEPs to WHO, aims to develop a comprehensive health policy, with a current priority on nutrition and physical activity.

Applied Clinical Trials-11-01-2018

The Basics of Clinical Trial Centralized Monitoring

FDA Initiatives Advance Orphan Therapies

Closer Pharma-Diagnostic Collaboration is Key to Alzheimer’s Drug R&D

Risk-Based Monitoring Versus Source Data Verification

Europe Mulls Rival Approaches to Tackling Disease

Blockchain in Research, from Business to Personal

Applied Clinical Trials, November 2018 Issue (PDF)