Johanna Schenk, MD

The monitoring of clinical trials is responsible for 22%-30% of overall clinical study costs.

 The practice involves several stakeholders and clinical monitors with specialized skills. However, many researchers have traditionally hesitated over its efficiency and claimed that a more centralized and adaptive risk-based approach to monitoring can be a better alternative.

 And now, due to regulatory changes in clinical trials, such as the introduction of the addendum to the ICH GCP E6(R2), new FDA guidelines for industry on risk-based monitoring (RBM), and the European Medicines Agency’s (EMA) reflection paper on risk-based quality management, the urgency of the RBM process implementation has increased in most biopharma establishments. This has sparked a whole range of internal projects industry-wide, despite the lack of standardization and guidance concerning many RBM procedural aspects. Thus, neither academia nor industry are able to offer a clear description of a holistic, unified, and standardized process for RBM, comprising centralized monitoring procedures. Therefore, such RBM process definition, together with derived roles and function descriptions, as well as the required tools and documents, are more relevant than ever.

	As a result, two pharma-related companies and two academic institutions initiated in 2014 a research project (code name: PUEKS), the aim of which was the development of a holistic system via process integration to effectively control clinical trial risks beyond isolated RBM strategies.

	The PUEKS project started in December 2014 as a winner from the research and development project applications in the framework of the Hessen ModellProjekte (Germany), and was completed in November 2016. The Hessen ModellProjekte program is coordinated by Hessen Agentur, a fully state-owned organization. The objective of this program is the promotion of innovative R&D projects. The project selection criteria are the degree of innovation, exploitation potential, and technology transfer of the project as well as the competencies of the involved partners.

	PUEKS complements other cross-stakeholder collaborations such as the Clinical Trial Transformation Initiative (CTTI),

 and the Metrics Champions Consortium (MCC)

 to innovate and improve clinical trial quality management. The initiatives of these three non-profit organizations depict the need of a common understanding and standardization of risk-based quality management approaches in clinical research.

	This project depicts the importance of the collaborative efforts between industry, research institutes, and academia. Thus, the project initiative of the consortium manager, Cyntegrity, a company specialized in cloud solutions for RBM, has been joined by PPH plus, a contract research organization (CRO) offering clinical development project and medical leadership and consultancy services; the Institute of Biostatistics and Mathematical Modeling at the Goethe University; and the Fraunhofer IME Project Group TMP, sited at the University Hospital, Goethe-University; all located in Frankfurt am Main, Germany.

	The integration of risk-based clinical trial management processes entails a collaborative effort between company departments, academics, and across multiple clinical operations functions. Well-organized teamwork has been streamlined and boosted with fluent risk communication systems supported by online cloud platforms and shared visualizations, to capture, monitor, control, and report not only on clinical trial risks but also on clinical trial overall performance beyond the clinical operations department.

	International regulations, standards (ISO) and guidance concerning clinical trials (ICH,

), previous and contemporary projects on RBM procedures (ADAMON,

), and international risk assessment scoring systems (e.g., TransCelerate Biopharma RBM initiative,

 MRC/DH/MHRA Joint Project for the development of risk-adapted approaches to the management of clinical trials of investigational medicinal products

) have been reviewed and used for ensuring compliance with clinical research regulations and guidelines and adjustment to the variable clinical study requirements. RBM benchmarking sources have been consulted for detecting the pros and cons of the currently available RBM solutions.

	The PUEKS project has been developed in observance of the recent addendum to the ICH GCP guideline, E6(R2),

 the EMA reflection paper on risk-based quality management,

and the FDA guidance for industry concerning RBM.

The outcomes of the project relate to each of the main risk-based quality management system (QMS) processes mentioned in the ICH Good Clinical Practice (GCP) addendum E6(R2):

		Critical process and data identification

	Thus, the PUEKS project has aimed at filling the gaps of the-up-till-now available methods or procedures implemented in each of the above-mentioned processes.

	The PUEKS project has devoted two years to the evaluation and optimization of RBM. For this purpose, PUEKS used data already available from ongoing clinical studies to select substantiated key performance and risk indicators (KPIs, KRIs). Thus, the obtained data-driven KRIs have been tested with real clinical data. The power of the selected KRIs (see Table 1) in terms of risk identification and mitigation has been tested, comprising not only an evaluation of KRIs, but also site risk profiles and performance indicators.

	In the frame of the PUEKS project, data from two investigator-initiated clinical trials (IIT) on autoimmune diseases (studies A and B) have been analyzed with an RBM cloud solution created by Cyntegrity:

		Study A (Phase III, randomized, parallel, multicenter clinical trial, four-year duration) involved the placebo-controlled use of a new combination of two licensed drugs, with the initiation of 44 sites and inclusion of 180 patients.

		Study B (medical device study meeting the German Medical Devices Act §23b requirements, multicenter clinical trial, 18-month duration) was comparing the sensitivity of a diagnostic method utilizing an approved device against the standard of care in a given indication, with participation of 12 sites and inclusion of 31 patients.

		Additionally, Cyntegrity’s database, comprising data from publicly available clinical trial records, has been utilized for the testing and adjustment of RBM procedures and KRIs.

	For study A, PPH plus performed an analysis of the clinical study protocol, the monitoring plan, and other essential documents (e.g., informed consent form) in order to identify and select the most suitable risk indicators, as well as to develop sound standard operating procedures (SOPs) on risk-based quality management and RBM.

	Based on the type of data to be captured and the related data capture system, Cyntegrity, the Institute of Biostatistics and Mathematical Modeling at the Goethe University, and PPH plus identified and developed further KRIs.

	Study B has been used to detect further needs concerning SOP clarification and to enhance risk-reporting procedures.

	The main list of KRIs that have been developed, tested, and implemented is shown in Table 1.

	These KRIs have been complemented by centralized statistical monitoring (CSM) tools such as case report form (CRF) completeness reports, to inform about the ratio of missing critical data, and further CRF data inconsistency analysis (e.g., concerning inconsistent serious adverse event [SAE] reporting, unexpected patient visit schedules, or suspected protocol deviations).

1.  Critical process and data identification

	According to the ICH GCP addendum, EMA and FDA guidance documents, critical data, and critical processes are those which may threaten patient safety and rights, as well as data integrity or reliability.

	Analyzing Cyntegrity’s database, as well as studies A and B, PUEKS has identified critical processes that may be inefficiently supervised by traditional monitoring methodology. Thus, quality management gaps were analyzed by a retrospective analysis of essential documents, including captured data, and clinical study procedures (i.e., data collection and informed consent). Thus, as an example, the requirements of paper and electronic data capture (EDC) systems have been checked to identify what aspects of the design of data capture systems may enhance monitoring.

	Critical processes and data have been evaluated at two levels:

 Adequate data capture systems, document updating systems, and investigational medicinal product (IMP)-handling procedures (i.e., GCP-compliant systems and procedures).

 Study-specific terminology clearly defined in the clinical study protocol, adequate EDC adjustment (configuration) to clinical trial requirements.

	To improve critical processes and data identification, PUEKS methodology advises biopharma to implement the following tools:

		Register of audit findings, corrective action preventive action (CAPA) and lessons learned, comprising the timely communication of known audit findings and quality assurance outcomes to entire study teams, as well as preventive and corrective actions.

		Expert and experienced personnel input: Analysis of previous pitfalls and the impact of each of them on providing evidence of what processes and data are critical.

	Likewise, the outcomes of continuous reviews of clinical monitoring outcomes, during clinical trial conduct, should be shared in a timely manner with all operative clinical research teams and relevant functions, beyond the particular study team. An asset for biopharma will be a list of critical processes and data provided by experts from the involved functions, similar to the Critical-to-Quality Factors Principles document created by the CTTI.

Before identifying any risks, it is essential to understand which clinical trial design factors are enhancing or diminishing trial success.

	Team members involved in risk identification should, above all, focus on identification of risks threatening:

		Critical data of the pertinent clinical study,

		IT systems (e. g., communication tools, electronic files, data capture systems)

 a non-profit organization comprising the world’s leading biopharmaceutical companies, has created an RBM knowledge center freely available online. This knowledge center includes the risk assessment and categorization Tool (RACT) used mainly for risk identification and evaluation based on the failure mode effects analysis (FMEA) method.

	Although the RACT is a sufficient tool for systematically reviewing all study-critical aspects for risk evaluation and identification, it has its drawbacks.

 The major one is that the risk levels are assessed with high subjectivity and in disagreement with other study team members’ evaluations. This generates a bias in the risk prioritization process.

	PUEKS found the need of a consensus for risk level assignments that the RACT has not been covering. Thus, PUEKS developed a risk assessment tool based on an enhanced version of the RACT originally created by TransCelerate, with the incorporation of objective definitions for low, medium, and high risk levels for each risk concerning three risk attributes:

	The enhanced risk assessment tool is a list of questions that facilitate the identification as well as the qualitative assessment of clinical trial risks, organized in 13 risk categories (as per TransCelerate’s RACT): safety, study phase, study complexity, subject population, technology, data collection procedures, study endpoints, organizational experience, IMP, IMP supply chain, blinding, operational complexity, and geography. The enhanced RACT enables the study team to identify risks and to evaluate the severity of risk impact (I), probability of occurrence (P), and detectability (D) by assigning a score (from 1 to 3, higher scores meaning higher risk) to each of these three attributes (same as the RACT). I, P, and D scores are multiplied to obtain a single score for each risk, the so-called risk priority number (RPN). The assigned risk scores permit single and grouped risk prioritization and the calculation of a total risk score for each clinical study.

	Importantly, the enhanced RACT enables the objective assignment of risk scores regardless of the individual (team member) who is deploying it. As an example, an excerpt from the enhanced RACT is provided in Table 2 and Table 3.

	In the frame of the PUEKS project, Cyntegrity has developed a publicly available cloud 

 of the RACT. This electronic tool offers different risk identification and assessment catalogs to adjust to the need of different clinical trials and diverse requirements of biopharma, CROs, and clinical research institutions. The cloud tool comprises as well a catalog with standardized risk level definitions developed by PPH plus (as per the enhanced RACT) to streamline risk score assignments via a more objective evaluation of risks.

	Full risk identification and analysis have been performed in PUEKS for study B, which has served both the development and optimization of the enhanced RACT. The objectivity of the definitions has been verified by getting different project leaders to complete the risk assessment separately. In those cases for which the selected risk scores were in disagreement, the definitions had been revised and updated until all project managers opted for exactly the same scores. Thus, the standardization of the responses enables a more objective risk prioritization. During study conduct, the risk priorities may be automatically adjusted according to risk occurrence or the frequency of alerts triggered by the risk monitoring system or risk monitor.

Articles

A look at the results of the recently completed project, which aimed to develop an holistic monitoring system via process integration to more effectively control clinical trial risks beyond isolated RBM strategies.
  

The PUEKS Project: Process Innovation in Clinical Trial Monitoring

	The RbM Consortium is a transnational alliance of quality risk management industry experts, risk-based technology firms, data analysts, and biopharmaceutical business strategists. Members of this group include 

. The objective of this collaboration is to improve awareness of Risk-based Monitoring (RbM) in clinical trials, to advise biopharmaceutical enterprises, medical device companies, CROs, study sponsors, and study sites on best practices, and communicating the best implementation strategies on sound quality risk management systems, technologies, and business methodologies in RbM.  This guidance document will offer information about RbM strategies.

, data-driven monitoring or centralized monitoring have become interrelated terms of a hot topic, and there is hardly an organization that does not claim to already apply or considers to implement such an approach for clinical development and pharmacovigilance. This begs the question: is it just hype or a shift in paradigm? We are convinced that this is more than just hype. However, the shift in paradigm has not yet happened either. We-the 

RbM Consortium, which includes recognized experts from this area-

observe many approaches to a risk-based study management but came to the conclusion that it is too early to celebrate an advent of a new era in clinical trials and development. We notice that the majority of approaches proposed lack fundamental elements of Quality by Design and Quality Risk Management strategies.

Lack of a uniform and broad understanding of underlying principles, methodology and approach:


		A shared understanding of terminology, scope of tasks, deliverables, as well as roles and responsibilities across the entire stakeholder community involved in the planning, conduct, analysis, reporting, and assessment of clinical trials-e.g., sponsors, contract research organizations (CROs), investigators (and, as a result, patients)-is important to the realization of a risk-based study management approach.
		 

Decisions about risks are not based on soundly defined objective decision criteria:


		Risk assessments and their resulting decisions today are mostly based on the evaluation and opinion of teams or individuals. What happens if those teams or individuals are replaced by others? Will their decisions be the same? This is highly doubtful, since individuals base their decisions on their experience and the situation they are currently in. As important experience is for establishing a risk-based system, independent decision criteria or decision frameworks are needed to keep the human factor as small as possible once the system goes live.
		 

Emphasis is not put on the foundation of risk-based monitoring:


		The foundation of a sound risk-based approach starts much earlier than the trial itself with aspects such as protocol design, study start-up assessments, site qualification, patient enrollment optimization, to name a few.
		
		The common approach focuses only on a centralized review of incoming data or a strategy of reduced source document verification (SDV) and site monitoring but protocol design aspects, which are the blueprint for the trial, are neglected. As a result, protocols of inadequate design (structure and content) are implemented with a study management approach that is likely to be unfit for the complexity of the protocol.
		 

No integrated quality strategy is developed and implemented:


		Such strategy must include study design, site selection (investigators who have the required therapeutic and Good Clinical Practice (GCP) experience as well as access to the patients to be enrolled), study management (centralized/data driven monitoring) and general oversight aspects (quality management plan, third party oversight, audits, etc.).
		 

Risk assessments are conducted in silos and not shared within and between sponsors:


		In order to save time and money, sharing of assessments of “common” entities, such as CROs, central laboratories and others can contribute to a leaner and more effective approach to managing clinical trials. In addition, networks of service providers should be created to serve as one-stop shop for a sponsor of a clinical study covering all elements from protocol design to reporting of trial results including disclosure, submissions, training, qualifications, risk assessments, safety duties and many more.

	Based on the above-mentioned deficiencies we have put together a catalogue of ten key aspects on how to navigate QbD, QRM and RbM and to implement a sound 

1. Is it true that only larger pharma or device companies can apply a risk-based approach?

	Designing and implementing a risk-based approach does not depend on the size of a company. Smaller biotech, pharmaceutical and device companies as well as academia can do it, too. All will benefit as much as Big Pharma from a smart approach to monitoring. We all face the same challenges: lack of resources, time pressure and growing regulatory requirements.

	It is important that the designed solutions to RbM are meeting the criteria defined above in our statement. There are many solutions to RbM, whether they are built by using a spreadsheet or supported by a highly sophisticated IT system with automated data analysis. The more clinical studies and the more complex they are, i.e., global, multiple arms, comparators, etc., the better would be a solution that is technically sound and to a high degree automated. However, this is not needed in all scenarios. We will explain it further in the following.
	 

2. Do I need sophisticated IT tools to implement a risk-based approach?

	A risk-based approach starts with the identification of risks followed by the analysis as to their impact, likelihood and detectability, and the possible actions to mitigate them. For such an assessment, no sophisticated IT tools are required. The tracking of risk profiles – at the sponsor, CRO or clinical trial center level – and of actions triggered by the risk assessments can be performed with off-the-shelf office tools (e.g., spreadsheets). If larger volumes of data need to be assessed, more sophisticated web-based tools may be user-friendlier.

	However, when clinical trial data are to be regularly reviewed for outliers, errors or other non-compliances, and large volumes of data need to be processed; this would require the use of a validated and well-designed computerized system to minimize workload. In addition, data need to be presented in reports that are easy to understand, classified in risk categories that are based on objective criteria, and result in actionable mitigating actions.

3. Is it true that we need to change the way we write protocols and set up trials to implement successfully a risk-based approach?

	There is consensus that the current approach to clinical development is not sustainable due to the following reasons:

		About 50% of sites fail to reach the planned recruitment targets.

		The majority if not all milestones missed in the clinical trials-more than 95% of clinical trials do not end on time and on budget as planned in the first place.

		90% of the studies meet their recruitment goals, but at the expense of mostly twice as much time as originally planned.

 The costs attributable to these delays are gigantic.

		Too many avoidable protocol amendments, with a first amendment implemented even before the very first patient has been enrolled-most of them due to the rush towards “First-Patient-In”.

		Compliance flaws causing delays and additional costs.

		Bottom line: trials take much longer and cost much more than originally planned.

	A senior inspector summarized recently her reservations about the current approach as follows: “If sponsors and CROs do not learn to write better protocols, moving from the current trial monitoring approach to a risk-based or centralized monitoring approach, it is likely to end in a compliance disaster.”

	We are also absolutely convinced that the quality of protocols needs substantial improvement, for example, by implementing the following plan changes:

		Focusing on one study rationale and one or two primary objectives.

		Verifying with real-life data that inclusion criteria are fit for purpose and support the value proposition of the new product.

		Verifying that the study set-up is aligned with patients’ and investigators’ priorities and capabilities through patient-centric models and simulations.

		Verifying with real-life data that enrollment plans (recruitment goal and rate) are realistic and the right sites are being involved

	We should never forget that the rationale and objectives of a protocol should target a clear value proposition and that the key stakeholders of a protocol are not the editors of a scientific journal but the professionals performing reviews at health authorities and Ethics Committees/Institutional Review Boards (EC/IRB) prior to clinical trial approval or in the context of Health Technology Assessments (HTA) for the decision on pricing and reimbursement.
	 

4. What are KRIs and KPIs, and how can they support a risk-based approach?

	KRIs are used to measure in an objective manner the risk imposed on a process or system. We distinguish leading and lagging KRIs. A leading KRI measures parameters that indicate a problem building up before it materializes so there is still time to correct. Lagging KRIs are indicating deviations that already happened, but as a single event would not have a great impact, however, when piling up, impose a risk on the process/deliverables looked at. Generally, leading KRIs are more effective but are also more difficult to measure than lagging ones. A proper suite of KRIs combining leading and lagging KRIs allows a study management team to plan and execute a risk-based strategy, and KRIs can be benchmarked via empirical data analyses.

) is an example of a leading KRI. A delay in on-site monitoring does not per se threat patients’ safety, integrity and rights, and data integrity but increases the risk of tardy detection of a GCP issue, and as such it is sensible from a study oversight perspective to address such delay in a proactive manner. 

is an example of a lagging KRI: the GCP deviation has already occurred and although the deviation cannot be corrected its recurrence can be prevented.

	KRIs can be designed as “digital” triggers, i.e., the KRI is either 

: when the threshold is reached the KRI “fires” and generates a signal. This approach, rather than having “multi-level” KRI thresholds, avoids complexity in the calculation logic and also facilitates the “validation” of the threshold. Indeed, any threshold needs to be justified by means of objective criteria. Validation of a KRI threshold can in essence be achieved in three ways:

	a)   Thresholds taken from regulatory, SOP, industry, etc. standards, e.g., the KRI for processing a serious adverse event would be set at day 13 to avoid non-compliance with the 15 days reporting timelines.

	b)   Analyzing compliance data from past studies, i.e., analyzing the behavior of a KRI threshold in studies known for good compliance vs. studies with recognized GCP deficiencies.

	c)   Analyzing “real world” data, e.g., analyzing Electronic Health Records (EHR) to get clues about realistic expectations vs. protocol requirements. For instance, EHRs can be analyzed to determine how frequently and to what extent a patient’s visit deviates from the scheduled visit date.

	d)   If none of the above approaches can be applied, an arbitrary threshold can be set and the adequacy of the threshold can be verified or fine-tuned later through on-site monitoring and auditing approaches.

	When defining KRIs study teams must practice caution to measure what has an impact on compliance and process effectiveness and not what can easily be measured. For instance, measuring compliance against timelines may induce team members to take short cuts, which can be the root cause of new deficiencies.

	Reports aggregating KRI analyses data can be presented via tables, graphic mapping, odometers, and other analytical visualizations.

5. Will a risk-based approach make site selection/qualification and patient recruitment into my trials any easier, and will it help to get better data faster and cheaper?

	Risk-based approaches apply not only to monitoring, but also to many facets of a clinical trial, such as site selection/qualification, protocol design and subject enrollment/retention. Sponsors can mitigate clinical trial risks in the following areas:

Predict Study Site Enrollment and Quality Potential Prior to Initiation:

 Initiating poorly performing study sites is common and is costly. Up to one third of study sites never enroll a single patient 

, and it can cost up to 50,000 USD per site to seek, initiate, maintain and close poorly performing sites 

. With the availability of numerous data sources (i.e., patient populations, number of ongoing clinical trials at an investigative site, competitive trials, site resources, years of investigator experience, etc.), we advise sponsors/CROs to conduct empirical analyses and run predictive models against historical site performance to unveil factors that affect quality and enrollment outcomes. Quantifying site enrollment and quality potential prior to site initiation enables sponsors to minimize exposure to poorly performing sites, which saves costs and improves quality outcomes.

 According to FDA and EU inspectors, a well-designed protocol is the blueprint for quality clinical trials, and many studies experience timeline slippage and compliance issues because of poorly designed protocols, which result in numerous amendments, and costing more than 400,000 USD per amendment 

. In order to minimize timeline slippage, we advise sponsors/CROs to test protocols against real-time aggregated Electronic Medical Records data. This test enables sponsors/CROs to optimize protocol inclusion/exclusion criteria to better match real-world patient medical criteria, which can minimize enrollment slippage and the number of protocol amendments.

 Missing clinical trial data due to subject dropouts has a significant impact on clinical trial timelines, budgets and quality outcomes. Reasons for subject dropout can vary from subject geo-demographics, protocol complexity, subject commitment, education, distance from study site, employment status, physical activity and much more 

. Albeit there are technologies that focus on engaging patients, many are unproven in the realm of healthcare and clinical trials. We advise sponsors/CROs to not only test out feasible technologies, but also engage patients through validated technologies, such as SMS (Short Messaging Service, also known as Text Messages), which is a proven and cost-effective way to improve medication adherence 

, and motivate patients to stay in a clinical trial.

6. Will a risk-based approach allow me to stop SDV (source document verification) and reduce the burden of on-site monitoring?

	With RbM, source document verification (SDV) is not becoming obsolete but the way it is applied needs to change as outlined below. The FDA clearly states, “No single approach to monitoring is appropriate or necessary for every clinical trial. FDA recommends that each sponsor defines a monitoring plan that is tailored to the specific human subject protection and data integrity risks of the trial. Ordinarily, such a risk-based plan would include a mix of centralized and on-site monitoring practices.”

 Therefore, through the systematic use of a centralized review of the study data we can consider a reduction in the burden of on-site monitoring, improvements in clinical data quality and an increase in the clinical monitors’ effectiveness. All sources of data should be mined and analyzed such as the data from the CRF, its metadata (e.g., audit trail data), the CTMS, the TMF, safety database, etc. When such an approach to study management is applied, SDV will serve as a root cause analysis tool when needed rather than a data comparison tool. We should always keep in mind that the goal of any monitoring activity – regardless of the tools used – is to protect patients’ safety, integrity and rights as well as ensuring data integrity. A proactive approach to study oversight increases efficiency by reducing the need for corrective actions.

7.  How much money do I save by implementing a risk-based approach?

	
	Depending on the complexity of a clinical study, the costs of monitoring (personnel, travel, expenses, etc.) can reach magnitudes of 25-30 % of the clinical trial budget.

 Since quite some time this has been a target for cost-saving exercises at many pharmaceutical and biotech companies. However, none has actually achieved the goal of cost reduction in this budget line item. With risk-based monitoring, however, many believe that now has come the time, and finally the tools are available, to reduce the monitoring budget.

	Although certain tasks can now be done centrally, there is still a need for visiting the centers to ensure that patients’ safety and the integrity of the data are observed. Therefore, the current monitoring effort that is rather distributed equally will be shifted towards sites in greater need, such as centers with many patients, not experienced in conducting GCP trials, potential problems indicated through KRIs and central data analysis/risk assessment, etc. This means that we will rather see a shift in effort than a reduction of it. However, the positive result from this shift is that the sites in need now get the attention and support necessary to help them to deliver good results.

	Savings in the long run will be realized when through well planned and implemented clinical studies the timelines to data base lock and ultimately to the launch of the product will be shortened, which will ensure longer patent exclusivity. When we take a medicinal product with annual sales of 500 million USD and divide those annual sales by 365 days, we can see that the real savings of 1.65 million USD per day are waiting for those who set up their studies in the QbD spirit.

8.  How will Health Authorities react if they discover a major or critical finding that was not detected or not addressed through your risk management approach?

	We have discussed this aspect multiple times with Health Authorities’ inspectors and their feedback was clear and consistent with regulators’ messages about RbM: “errors in clinical trials are acceptable to regulators as long as had we perfect data we would still make the same decision and come to the same conclusion.” In other words, inspectors will continue to detect and report non-compliances, however if inspectors and reviewers come to the conclusion that through those non-compliances patients’ safety, integrity and rights as well as data integrity were not compromised, a trial/system/process will still be considered compliant in the sense that the goal of GCP (patients’ safety, rights, integrity, and data integrity) was reached. Obviously, even if such findings were not considered as critical, the sponsor must, however, implement adequate CAPAs.

	Additionally, the risk management approach has the advantage of knowledge formalization and incorporation of the critical findings into the future risk-monitoring process, which leads to continuous improvement.

9. How does a risk-based approach impact on my organization, i.e., study site, Quality Assurance, Data Management, Clinical Operations, Drug Safety, Biometrics, etc.?

	Our experience shows that change management activities around the development and rollout of an RbM/QRM strategy in study management and oversight are key to success. A “must do” is early communication, honestly and proactive alignment of the entire organization with the plans, goals and – most importantly – the organizational and people implications.

	Unfortunately, still today, a silo approach to study oversight activities is the norm. Within a sponsor company, Clinical Operations develops monitoring plans, QA audit plans and Clinical Science develops plans about the benefit–risk assessments. Resources are allocated by functions and not in an integrated manner. For instance, the data management budget may allow developing an EDC and data management solution but does not include resources for programming and maintaining KRIs. Similarly, the biostatisticians may be tasked to develop a Statistical Analysis Plan (SAP) but do not have resources to develop programs for the early detection of outliers or abnormalities in the data structure. When CROs are involved, the wealth of insight about process efficiencies and deficiencies are rarely shared across their clients to generate learnings and trigger process improvements, too.

	Therefore, when a company decides to adopt an RbM approach, a holistic, integrated risk-based study management plan needs to be developed. From this follows that all stakeholders need to work together to identify the required tools, activities and minimum tolerable error levels. The latter is nothing else than the definition of the “design space” for a given study and the justification with objective evidence of that “space.” RbM requires a change in mindset and, therefore, change management that is introduced at the very beginning of an RbM project is key to its success.

10. What is the best implementation strategy for a risk-based approach?

	Our experience has shown that a systematic introduction of an RbM approach rather than a big bang launch is the best implementation strategy for such a project. Indeed, a big bang strategy is likely to overwhelm an organization, underestimates the technical and organizational challenges and thus triggers resistance amongst concerned staff members. A four-step approach has proven multiple times to work best:

Step 1. Predictive Modeling and Empirical Analysis:


	In the predictive modeling and empirical analysis phase, study teams need to (a) determine clinical operations/business research objectives, and (b) conduct empirical research on data from numerous completed clinical trials to uncover factors that impact clinical trial risk and performance outcomes, for example, factors that predict clinical trial failure. Once teams identify risk factors, we advise teams to conduct statistical analyses to determine analytical parameters, which can be used as benchmarks, and to predict future outcomes. After developing a portfolio of statistically linked factors and analytical expected outcomes, study teams can use these parameters to proceed with the proof of concept phase (step 2), and examine the application of these predictive models in real time.


	In the PoC phase, the tools, methodologies and processes needed to implement a Quality by Design (QbD) and RbM approach are developed and applied in one to a maximum of three studies with teams that are eager to move to a modern RbM methodology. Involving people enthusiastic about RbM is essential as it allows overcoming hurdles and backslashes without being forced to fight skepticism. This PoC generates important learnings on how to implement RbM in the sponsor organization. Specifically, on how to address gaps, conditions for success and limitations to RbM imposed by external constraints. For instance, it can give an overview of the capabilities of current IT systems that cannot be easily fixed. The PoC also delivers “success stories” that can be shared within the sponsor organization as a preparation for step 2. Typically, in the PoC there is little IT involvement and the tools developed and employed use simple technology such as structured questionnaires built in spreadsheets.

This guidance document will offer information about RbM strategies.

RbM Guidance Document: Ten Burning Questions about Risk-Based Study Management

Clinical development programs in the biopharmaceutical industry have become increasingly complex and continue to do so. In view of strong competition for productive sites; cost and time constraints; challenging internal and external environments; and large multinational stakeholder teams, the skill set of a clinical project leader is nowadays a key success factor. Executive clinical project leaders are involved in development programs encompassing a range of highly complex clinical trials. The key operational challenge of these programs are Phase III clinical trials involving hundreds of investigational sites and thousands of patients across different regions, cultures, healthcare systems, and regulatory environments. How can it be ensured that all contributors to the clinical trial production chain work properly on time, within expected scope, and on budget? One of the major obstacles still appears to be missing effective leadership at the project level.

What are the three most important interpersonal characteristics (soft skills) of a successful/ideal project leader? The authors addressed this question to the clinical research professionals attending the regional chapter meetings, organized by the German Society of Pharmaceutical Medicine (DGPharMed) in Mannheim, Hamburg, and Berlin between February and September 2011. The survey results are presented and discussed in this article.

	The survey was conducted among the clinical research professionals who attended the regional DGPharMed workshops entitled Project Management in Clinical Research held in Germany in 2011. The attendees heard about the principles of project management as applied to the conduct of clinical trials. The vast majority of participants then took part in the survey and selected the three most important characteristics of a successful/ideal project leader and rated these characteristics as most, second, and third most important. The meaning of the term project leader was explained to the respondents as a clinical project manager, study manager, or similarly named function with the responsibility to manage and primarily lead the clinical trial, project, or program depending on the size of the organization and the applicable settings. A skill set differentiation between clinical project leadership and clinical project management was not conveyed.

	The multiple-choice questionnaire offered the following pre-defined characteristics without further definition of terms: long-term experience in clinical research, risk awareness, medical expertise, knowledge of drug development, innovative, confidence-building, dependability, effective communication, clear goals, and proactive working style. The authors pre-selected these characteristics based on results of a Drug Information Association (DIA) tutorial attended by Schenk-prior to the DIA Conference entitled The Future of Pharmaceutical Project Management, Weston, Florida, November 13-14, 2007-where the participants (N=15) identified the three most important personal characteristics of an ideal project leader without any pre-defined answers.

	In the 2011 survey, a free text option allowed for the addition of further characteristics. To rule out any influence by the order of pre-defined answers, the last workshop (23% of participants) utilized the questionnaires with a changed sequence of the characteristics displayed in the questionnaire forms. The order of options on that finally used form was selected on a random basis.

	A total of 226 out of 235 clinical research professionals attending the workshops participated in this survey. Fifty-nine (26%) respondents represented pharmaceutical companies (mid-size to global pharma), 124 (55%) were from CROs, and 43 respondents (19%) worked in other organizations including biotech and medical institutions, or were freelance professionals.

	As illustrated in Figure 1 the majority of participants worked in project or study management (122; 54%), followed by trial monitoring (28; 12%). Clinical research and development functions (both non-medical and medical) were underrepresented in this survey (5% and 7% respectively). Twelve percent of respondents worked in trial monitoring or trial assistance functions. Only 2% of all participants had a data management role. Forty-four (20%) participants worked in other functions not listed above including freelance professionals, investigational sites, or SMO personnel and other functions. The distribution of participants by function and organizational affiliation was similar across all three regional workshops.

	The majority of project and study managers worked in CROs (20% and 13% of all respondents, respectively). Pharmaceutical companies were represented mostly by the study management functions (10% of all participants).

	The overall results of the survey are displayed in Figure 2. They demonstrate that 44% of participants rated effective communication as the most important characteristic of a successful/ideal project leader and 38% rated it second most important, followed by having "clear goals" (34% rated as most important). A proactive working style arrived on rank 3 (rated by 23% as third most important). Thus, 82% of participants identified effective communication as either the most or second most important factor. Overall, besides effective communication, clear goals and a proactive working style were the most frequently selected categories.

	The subgroup analyses by organization (Figures 3 and 4) and function did not reveal obvious differences in the results. Effective communication was the key characteristic rated by all analyzed subgroups as either the most or second most important characteristic.

	Representatives of pharmaceutical companies consistently rated effective communication (rated by 42% as most important, and by 36% as second most important), clear goals (rated by 31% as most important and by 29% as second) and proactive working style (rated by 32% as third most important) as the most important characteristics. Additionally, they rated dependability (rated by 12% as third most important) and long-term experience in clinical research (rated by 10% as third most important) of considerable importance (Figure 3).

	Contrary to pharmaceutical companies, CROs (Figure 4) valued risk awareness (rated by 16% as third most important) and dependability (rated by 15% as third) as one of the most important characteristics in addition to effective communication (rated by 48% as most important and by 38% as second), clear goals (rated by 34% as most important and by 23% as second) and proactive working style (rated by 22% as third most important). The value of a proactive working style was slightly lower rated by representatives of CROs as compared to pharmaceutical companies (22% versus 32% rated as third most important by CROs versus pharmaceutical companies).

	Fifty-three percent (i.e., 8 of 15) of representatives of the research and development medical functions from both CROs and pharmaceutical companies rated clear goals as the most important characteristic. Of notice is that the majority of respondents in this group selected effective communication as second most important characteristic only (rated by 20% as most important and 53% rated as second). This group valued the knowledge of drug development (20% rated third) and proactivity (20% rated third) that appeared to be higher as compared to other function subgroups.

	The overall results of the research and development function subgroup (both medical and non-medical functions) showed similar results defining clear goals (rated by 48% as most important) and effective communication (rated by 52% as second) to be at the top. A proactive working style (rated by 33% as third) and knowledge of drug development (rated by 19% as third) were on rank 3 and 4 of this subgroup.

	For the project and study management functions, effective communication was either the most or second most important characteristic (rated by 49% as most important, and by 36% as second), followed by clear goals (34% rated as most important and 27% as second). Risk awareness and dependability as the third most important characteristic were rated in 16% of cases reflecting the focus on risk management and trust in this subgroup of functions.

	As shown by different subgroup analyses, effective communication, clear goals, and proactive working style were considered to be the most important characteristics of a successful project leader. Knowledge of drug development, risk awareness, and reliability were rated as second most important by some of the subgroups in addition to communication, clear goals, and proactive working style. None of the subgroups and a very limited number of all respondents assigned a high value to long-term experience in clinical research, confidence-building personal skills, innovative behavior, as well as medical expertise.

	A number of educational textbooks on project management describe effective leadership as an essential prerequisite for project success. These textbooks provide comprehensive lists of project leader skills and competencies. While many sources provide representative data on technical skills, only limited data is available to characterize and evaluate the importance and role of personal/soft skills of successful project leaders. Most often this data is based on opinions of a limited number of project leaders and not based on large empirical research and representative surveys.

	Overall, only few investigations have been conducted to date to characterize the most important soft skills of successful project leaders, and a limited number of basic research studies has been performed so far, particularly in the pharmaceutical development sector.

	Aitken and Crawford16 evaluated the core set of behaviors of superior project managers (n=41). The essential behavioral competencies included delivering results and meeting customer expectations, planning and organizing, deciding and initiating action, leading and supervising, as well as persuading and influencing. Personality characteristics were consistent in rating as compared to behavioral competencies and included conscientious, vigorous, controlling, socially confident, evaluative, and persuasive. Of notice is that none of those characteristics specifically identified highly-developed communication skills as such as an essential competency.

	Skulmoski and Hartman described different competencies across all phases of the project based on 33 qualitative interviews conducted with 22 information systems project managers in Calgary, Canada. Communication competencies were identified as most critical during initiation, planning and closeout phases of a project.

	According to different sources communication or part of the skills related to communication are described as important or highly important. Thus, one of the critical skills that typically differentiates a project leader from a technologist is the ability to communicate well, both orally and in writing.

	The US DIA tutorial that induced the European survey presented in this article identified a proactive working style (47%, n=7), clear goals (33%, n=5) and effective communication (33%, n=5) as the personal characteristics of a successful project leader across the 15 respondents representing primarily the pharmaceutical industry

 who each was entitled to a maximum of three responses.

	In this survey, a large European panel (n=226) with experience in or exposure to project leadership in clinical trials confirmed the results obtained in the small US sample with effective communication assuming rank 1 ahead of clear goals and a proactive working style.

	The authors acknowledge the survey limitations due to restriction to soft skills of a project leader. Hard skills including education and expertise in the specific therapeutic area of project leaders have intentionally been disregarded. However, these types of skills are by all means considered to be a prerequisite and a minimum requirement for any project leader.

	It might also be argued that there was limited personal project management experience of a considerable proportion of survey participants. However, it should be noted that in view of their job functions all respondents must have been exposed to the performance of more or less successful project leaders. The results of this survey thus represent an external opinion of stakeholders of clinical development projects as well as own experience of executive project leaders.

	A next logical step would be to investigate how frequently current clinical project leaders are being rated as successful/ideal along the characteristics that came out as being of highest importance for project success, including their ability of handling different types of communication.

	Effective communication appears to be the key soft skill of a successful/ideal project leader far ahead of clear goals and a proactive working style. This conclusion mirrors the personal perception of 226 clinical research professionals representing predominantly CRO and pharmaceutical companies who work as project managers/clinical trial managers or are interested in this role. Long-term experience in clinical research and overall knowledge of drug development appear, however, to be less important for a project leader as compared to other knowledge, behavior, and skills. This outcome is in line with the rather scarce literature on the soft skill set of successful clinical project leaders. Required technical skills that are prerequisites for this job function were not subject to this survey.

	It appears to be mandatory that training programs offered to clinical project leaders attach high value to excelling in soft skills with a focus on effective communication and appropriate working style. The obvious aim would be to strengthen clinical development teams and boost the effectiveness of clinical development programs.

* MD, PMP, has worked at PPH plus as Project Leader. He is currently Clinical Research Physician at Boehringer Ingelheim, Biberach an der Riss, Germany, e-mail: 

 MD, FFPM, is Managing Director & COO at PPH plus, Frankfurt am Main, Germany, e-mail: 

	*To whom all correspondence should be addressed.

	J. Koedam and J. Schenk, "Project Leadership: Getting It Right," 

	C. Burgess, "The Changing Role of Project Managers," 

	M. Moyer and M. Dunphy, "Management vs. Leadership – How to Strike the Perfect Balance," 

A Guide to the Project Management Body Of Knowledge (PMBOK® Guide)

-Fourth Edition (PMI, Newtown Square, PA, 2008).

	G. J. Skulmoski and F. Hartman, "Information Systems Project Manager Soft Competencies: A Project-Phase Investigation," 

	PM Tutorial. "Project Management 101: The Project Manager-Administrative Skills," 

, Project Management Institute (PMI) (1994).

	PM Tutorial. "Project Management 101: The Project Manager-Technical Skills," 

	J. Kelso, "No Matter Your Title, Effective Communication is Vital," 

, Project Management Institute (PMI) (1997).

Four New Skills for the Human Side of Project Management

, Proceedings of PMI Global Congress 2003-North America.

Evaluating Communication Effectiveness in a Project Environment

, Proceedings of PMI Global Congress 2003-Europe.

Project Communication: Effective Writing Styles and Cross-Cultural Considerations

Proceedings of PMI Global Congress 2004-North America.

Effective Project Leadership: A Combination of Project Manager Skills and Competencies in Context,

 Proceedings of PMI Research Conference 2006.

	E. Muzio et al., "Soft Skills Quantification (SSQ) for Project Manager Competencies," 

Intercultural Communication in Global Business,

 Proceedings of PMI Global Congress 2008-Asia Pacific.

Senior Management Perceptions of Effective Project Manager Behavior: An Exploration of a Core Set of Behaviors for Superior Project Managers

, Proceedings of PMI Research Conference 2008.

 Proceedings of PMI Research Conference 2000, Paris, France.

	DIA Conference "The Future of Pharmaceutical Project Management," Tutorial on "Leadership: How We Do It and How We Can Do It BETTER" at Weston, Florida (2007).

A survey of clinical research professionals asked: What makes you a successful project leader?

Johanna Schenk, MD

Articles

The PUEKS Project: Process Innovation in Clinical Trial Monitoring

RbM Guidance Document: Ten Burning Questions about Risk-Based Study Management

Leading Clinical Projects

Project Leadership: Getting It Right

Measures of Success