Martin Koch

The monitoring of clinical trials is responsible for 22%-30% of overall clinical study costs.

 The practice involves several stakeholders and clinical monitors with specialized skills. However, many researchers have traditionally hesitated over its efficiency and claimed that a more centralized and adaptive risk-based approach to monitoring can be a better alternative.

 And now, due to regulatory changes in clinical trials, such as the introduction of the addendum to the ICH GCP E6(R2), new FDA guidelines for industry on risk-based monitoring (RBM), and the European Medicines Agency’s (EMA) reflection paper on risk-based quality management, the urgency of the RBM process implementation has increased in most biopharma establishments. This has sparked a whole range of internal projects industry-wide, despite the lack of standardization and guidance concerning many RBM procedural aspects. Thus, neither academia nor industry are able to offer a clear description of a holistic, unified, and standardized process for RBM, comprising centralized monitoring procedures. Therefore, such RBM process definition, together with derived roles and function descriptions, as well as the required tools and documents, are more relevant than ever.

	As a result, two pharma-related companies and two academic institutions initiated in 2014 a research project (code name: PUEKS), the aim of which was the development of a holistic system via process integration to effectively control clinical trial risks beyond isolated RBM strategies.

	The PUEKS project started in December 2014 as a winner from the research and development project applications in the framework of the Hessen ModellProjekte (Germany), and was completed in November 2016. The Hessen ModellProjekte program is coordinated by Hessen Agentur, a fully state-owned organization. The objective of this program is the promotion of innovative R&D projects. The project selection criteria are the degree of innovation, exploitation potential, and technology transfer of the project as well as the competencies of the involved partners.

	PUEKS complements other cross-stakeholder collaborations such as the Clinical Trial Transformation Initiative (CTTI),

 and the Metrics Champions Consortium (MCC)

 to innovate and improve clinical trial quality management. The initiatives of these three non-profit organizations depict the need of a common understanding and standardization of risk-based quality management approaches in clinical research.

	This project depicts the importance of the collaborative efforts between industry, research institutes, and academia. Thus, the project initiative of the consortium manager, Cyntegrity, a company specialized in cloud solutions for RBM, has been joined by PPH plus, a contract research organization (CRO) offering clinical development project and medical leadership and consultancy services; the Institute of Biostatistics and Mathematical Modeling at the Goethe University; and the Fraunhofer IME Project Group TMP, sited at the University Hospital, Goethe-University; all located in Frankfurt am Main, Germany.

	The integration of risk-based clinical trial management processes entails a collaborative effort between company departments, academics, and across multiple clinical operations functions. Well-organized teamwork has been streamlined and boosted with fluent risk communication systems supported by online cloud platforms and shared visualizations, to capture, monitor, control, and report not only on clinical trial risks but also on clinical trial overall performance beyond the clinical operations department.

	International regulations, standards (ISO) and guidance concerning clinical trials (ICH,

), previous and contemporary projects on RBM procedures (ADAMON,

), and international risk assessment scoring systems (e.g., TransCelerate Biopharma RBM initiative,

 MRC/DH/MHRA Joint Project for the development of risk-adapted approaches to the management of clinical trials of investigational medicinal products

) have been reviewed and used for ensuring compliance with clinical research regulations and guidelines and adjustment to the variable clinical study requirements. RBM benchmarking sources have been consulted for detecting the pros and cons of the currently available RBM solutions.

	The PUEKS project has been developed in observance of the recent addendum to the ICH GCP guideline, E6(R2),

 the EMA reflection paper on risk-based quality management,

and the FDA guidance for industry concerning RBM.

The outcomes of the project relate to each of the main risk-based quality management system (QMS) processes mentioned in the ICH Good Clinical Practice (GCP) addendum E6(R2):

		Critical process and data identification

	Thus, the PUEKS project has aimed at filling the gaps of the-up-till-now available methods or procedures implemented in each of the above-mentioned processes.

	The PUEKS project has devoted two years to the evaluation and optimization of RBM. For this purpose, PUEKS used data already available from ongoing clinical studies to select substantiated key performance and risk indicators (KPIs, KRIs). Thus, the obtained data-driven KRIs have been tested with real clinical data. The power of the selected KRIs (see Table 1) in terms of risk identification and mitigation has been tested, comprising not only an evaluation of KRIs, but also site risk profiles and performance indicators.

	In the frame of the PUEKS project, data from two investigator-initiated clinical trials (IIT) on autoimmune diseases (studies A and B) have been analyzed with an RBM cloud solution created by Cyntegrity:

		Study A (Phase III, randomized, parallel, multicenter clinical trial, four-year duration) involved the placebo-controlled use of a new combination of two licensed drugs, with the initiation of 44 sites and inclusion of 180 patients.

		Study B (medical device study meeting the German Medical Devices Act §23b requirements, multicenter clinical trial, 18-month duration) was comparing the sensitivity of a diagnostic method utilizing an approved device against the standard of care in a given indication, with participation of 12 sites and inclusion of 31 patients.

		Additionally, Cyntegrity’s database, comprising data from publicly available clinical trial records, has been utilized for the testing and adjustment of RBM procedures and KRIs.

	For study A, PPH plus performed an analysis of the clinical study protocol, the monitoring plan, and other essential documents (e.g., informed consent form) in order to identify and select the most suitable risk indicators, as well as to develop sound standard operating procedures (SOPs) on risk-based quality management and RBM.

	Based on the type of data to be captured and the related data capture system, Cyntegrity, the Institute of Biostatistics and Mathematical Modeling at the Goethe University, and PPH plus identified and developed further KRIs.

	Study B has been used to detect further needs concerning SOP clarification and to enhance risk-reporting procedures.

	The main list of KRIs that have been developed, tested, and implemented is shown in Table 1.

	These KRIs have been complemented by centralized statistical monitoring (CSM) tools such as case report form (CRF) completeness reports, to inform about the ratio of missing critical data, and further CRF data inconsistency analysis (e.g., concerning inconsistent serious adverse event [SAE] reporting, unexpected patient visit schedules, or suspected protocol deviations).

1.  Critical process and data identification

	According to the ICH GCP addendum, EMA and FDA guidance documents, critical data, and critical processes are those which may threaten patient safety and rights, as well as data integrity or reliability.

	Analyzing Cyntegrity’s database, as well as studies A and B, PUEKS has identified critical processes that may be inefficiently supervised by traditional monitoring methodology. Thus, quality management gaps were analyzed by a retrospective analysis of essential documents, including captured data, and clinical study procedures (i.e., data collection and informed consent). Thus, as an example, the requirements of paper and electronic data capture (EDC) systems have been checked to identify what aspects of the design of data capture systems may enhance monitoring.

	Critical processes and data have been evaluated at two levels:

 Adequate data capture systems, document updating systems, and investigational medicinal product (IMP)-handling procedures (i.e., GCP-compliant systems and procedures).

 Study-specific terminology clearly defined in the clinical study protocol, adequate EDC adjustment (configuration) to clinical trial requirements.

	To improve critical processes and data identification, PUEKS methodology advises biopharma to implement the following tools:

		Register of audit findings, corrective action preventive action (CAPA) and lessons learned, comprising the timely communication of known audit findings and quality assurance outcomes to entire study teams, as well as preventive and corrective actions.

		Expert and experienced personnel input: Analysis of previous pitfalls and the impact of each of them on providing evidence of what processes and data are critical.

	Likewise, the outcomes of continuous reviews of clinical monitoring outcomes, during clinical trial conduct, should be shared in a timely manner with all operative clinical research teams and relevant functions, beyond the particular study team. An asset for biopharma will be a list of critical processes and data provided by experts from the involved functions, similar to the Critical-to-Quality Factors Principles document created by the CTTI.

Before identifying any risks, it is essential to understand which clinical trial design factors are enhancing or diminishing trial success.

	Team members involved in risk identification should, above all, focus on identification of risks threatening:

		Critical data of the pertinent clinical study,

		IT systems (e. g., communication tools, electronic files, data capture systems)

 a non-profit organization comprising the world’s leading biopharmaceutical companies, has created an RBM knowledge center freely available online. This knowledge center includes the risk assessment and categorization Tool (RACT) used mainly for risk identification and evaluation based on the failure mode effects analysis (FMEA) method.

	Although the RACT is a sufficient tool for systematically reviewing all study-critical aspects for risk evaluation and identification, it has its drawbacks.

 The major one is that the risk levels are assessed with high subjectivity and in disagreement with other study team members’ evaluations. This generates a bias in the risk prioritization process.

	PUEKS found the need of a consensus for risk level assignments that the RACT has not been covering. Thus, PUEKS developed a risk assessment tool based on an enhanced version of the RACT originally created by TransCelerate, with the incorporation of objective definitions for low, medium, and high risk levels for each risk concerning three risk attributes:

	The enhanced risk assessment tool is a list of questions that facilitate the identification as well as the qualitative assessment of clinical trial risks, organized in 13 risk categories (as per TransCelerate’s RACT): safety, study phase, study complexity, subject population, technology, data collection procedures, study endpoints, organizational experience, IMP, IMP supply chain, blinding, operational complexity, and geography. The enhanced RACT enables the study team to identify risks and to evaluate the severity of risk impact (I), probability of occurrence (P), and detectability (D) by assigning a score (from 1 to 3, higher scores meaning higher risk) to each of these three attributes (same as the RACT). I, P, and D scores are multiplied to obtain a single score for each risk, the so-called risk priority number (RPN). The assigned risk scores permit single and grouped risk prioritization and the calculation of a total risk score for each clinical study.

	Importantly, the enhanced RACT enables the objective assignment of risk scores regardless of the individual (team member) who is deploying it. As an example, an excerpt from the enhanced RACT is provided in Table 2 and Table 3.

	In the frame of the PUEKS project, Cyntegrity has developed a publicly available cloud 

 of the RACT. This electronic tool offers different risk identification and assessment catalogs to adjust to the need of different clinical trials and diverse requirements of biopharma, CROs, and clinical research institutions. The cloud tool comprises as well a catalog with standardized risk level definitions developed by PPH plus (as per the enhanced RACT) to streamline risk score assignments via a more objective evaluation of risks.

	Full risk identification and analysis have been performed in PUEKS for study B, which has served both the development and optimization of the enhanced RACT. The objectivity of the definitions has been verified by getting different project leaders to complete the risk assessment separately. In those cases for which the selected risk scores were in disagreement, the definitions had been revised and updated until all project managers opted for exactly the same scores. Thus, the standardization of the responses enables a more objective risk prioritization. During study conduct, the risk priorities may be automatically adjusted according to risk occurrence or the frequency of alerts triggered by the risk monitoring system or risk monitor.

Articles

A look at the results of the recently completed project, which aimed to develop an holistic monitoring system via process integration to more effectively control clinical trial risks beyond isolated RBM strategies.
  

Martin Koch

Articles

The PUEKS Project: Process Innovation in Clinical Trial Monitoring