Mary Banach

At the 2018 DIA Annual Meeting, our DIA Clinical Data Management (CDM)/Risk-Based Monitoring (RBM) Community met with many people who were implementing or preparing to implement RBM and risk-based quality management (RBQM) approaches. Our attendees were looking for support and a forum to address their concerns. They knew that there were new guidelines and that the industry was anxious to get started with RBM activities, but they also wished to address the pitfalls one might expect. To address this need for a discussion of the changes taking place in our industry and to support each other in this endeavor, our group presented a segment of our roadmap beginning with a look at pre-study risk planning, vendor selection, and change management in RBM and RBQM.

 has focused upon the implementation of RBQM activities in clinical trials. Multiple stakeholders were looking for real experiences that can be incorporated into their risk management strategies. The FDA’s comment period on the draft guidance “A Risk-Based Approach to Monitoring of Clinical Investigations”

reminded us of the need to focus on how we were implementing the RBM guidance. The comment period also offered us an opportunity to communicate our suggestions for how to interface with and incorporate the needs of our communities and stakeholders. Another starting point for our roadmap was the case study from Cyntegrity.

 To assist in this overall endeavor, we have been working on an RBM Roadmap produced by the CDM and RBM DIA Communities. We envision a set of best practices.

	ICH GCP E6 (R2) requires clinical trial sponsor companies to incorporate RBM in the risk management plan.

	Risk planning should start with the development of the target product profile (TPP)

 and then continue into the development of an individual protocol for a study. Some aspects of the TPP, such as once-daily dosing or a positive safety profile, must be taken into account when developing the risk management plan for a study.

	The most impactful operational risks to a prospective clinical trial will most often be associated with the most important processes. Yet, it is not only the critical data and processes, as highlighted by ICH E6 R2, but also the systems, (devices, sensors, or software apps) that are used in clinical trials that contribute to risk.

	Thus, to begin with, the study team identifies critical data, tools, and processes. One example of critical data is the survival data of patients that withdrew their informed consent or those that are lost-to-follow-up in an outcome trial. These inputs may come from data management (DM), biostatistics, or clinical operations (CO). Examples of critical processes include the timely reporting of serious adverse events to the sponsor and the adequate management of those by the sponsor. This process depends on multiple stakeholders, including DM, pharmacovigilance/drug safety, regulatory affairs (RA), medical affairs (MA), and CO. In addition, it may be worthwhile to check inspection findings from the FDA or European Medicines Agency (EMA) to see which have been areas of concern based on historic data. Keeping a closer eye on these findings may be very helpful in the future. As we will discuss later, each study team member plays a unique and crucial role in the risk-identification process. Thus, while RBM focuses on risk-based monitoring, the integration of a systemic risk management strategy for identifying and managing risks-from study design to the submission of the clinical study report-has resulted in a more evolved descriptor called risk-based quality management (RBQM).

	Once the critical data, processes, and systems used in a trial have been identified for the study, a core study team representing cross-functional expertise should conduct an exercise of risk identification. This includes identifying all of the potential risks that may impact patient safety, data quality, and/or the operational success of the study. Identified risks should be ranked and prioritized based on three dimensions: impact, likelihood, and detectability (ILD). The product of ILD helps derive the risk score or the risk probability number (RPN). This will help the team to focus their risk mitigation efforts on the greatest risks to the study.

	Once the operational risks have been identified and characterized, the team develops a risk mitigation action plan to address the highest-priority risks. As previously mentioned, a high rate of lost-to-follow-up or discontinued patients might be an identified risk for the study. A number of mitigation actions might be considered for this risk:

		Removing one or more ancillary procedures from the protocol requirements that are not critical to support study endpoints may reduce the relative burden on patients participating in the study.

		Incorporate additional awareness training for sites related to this risk, including the need to check proactively with patients on their burden/comfort level with continued study participation, and what can be done to improve their experience, etc.

		Implement one or more key risk indicator (KRI) metric to identify sites that may be experiencing a high rate of lost-to-follow-ups, missed or missing visits, or withdrawn consents.

	These mitigation planning items need to be documented carefully, either within the risk management plan (RMP), risk log, or integrated quality risk management plan (IQRMP), where accountabilities can be assigned. The individuals accountable for managing the various risks must be identified via the operational plans, such as the central monitoring plan, the CDM plan, the site monitoring plan, or the pharmacovigilance plan. Thus, the RMP serves as a master plan, feeding into the individual operational plans and is, hence, described as an “integrated” risk management plan.

	Once the key risks have been specified and mitigation plans established, the complete set of KRIs to be used in support of risk detection should be confirmed. The list should ideally be comprised of therapeutic area-specific and study-specific KRIs identified during mitigation planning, along with a standard set of generic KRIs that the organization has identified to be used across studies.

The study team should identify about 10 to 25 KRIs for a study. When the team finds lost-to-follow-up patients and withdrawn informed consent, it should monitor the KRI for an increasing number of missing or missed visits at sites throughout the study. Applying relative weights to each of the KRIs may be helpful in supporting decisions on which KRIs to follow up on as they emerge during the study and the urgency with which outliers need to be addressed. Not every risk in a study can or should be mitigated. This exercise, though, must be run very carefully in order to not overlook critical data or processes. Changes in local regulations is illustrative of a risk that needs to be accepted. It is also important that teams are trained not only on defining the KRIs, but on interpreting them as well.

	Vendor selection for the implementation of RBM

	Vendor selection when implementing RBM activities is not an easy task. It involves internal and external challenges.

	At the start, one needs to have clarity within the organization as to what are the driving factors for implementing RBM. Is this a strategic decision with clear objectives (i.e., decreased number of site visits and increased quality) and clear milestones (i.e., decreased number of site visits) that improve quality? Many companies have not bought into this model, but are merely testing the waters. Some are recognizing the significant weightage that the FDA, EMA, TransCelerate BioPharma, and the latest ICH E6R2 guidelines have given to RBM and are making tactical decisions to perform a pilot study. However, it is important to remember that the implementation of RBM involves a significant investment in technology-and the benefits of this approach may not be evident by performing one small RBM pilot study.

	Another internal challenge is whether to follow a one-stop-shop approach or to identify smaller, more cost-effective and amenable partners. There is often a lack of clarity as to whose SOPs should be followed and whose KRI libraries should be used-the vendors or the sponsors? Even more critical is the lack of pre-defined key performance indicators (KPIs).

 These are essential to measure the success of the RBM implementation strategy and of the partnership as well.

	A survey conducted at the 2018 DIA meeting in Boston demonstrated that around two-thirds of the participants have never been involved in RBM vendor selection, did not have internal RBM expertise, and had not defined KPIs for RBM implementation within their organizations. This shows us the challenges that the industry is facing today.

	Implementing RBM is not just about analytics, technology, clinical, or RBM domain expertise-it is really about all of them. Finding 

a vendor who really understands the science of RBM is important. It is crucial to work with a vendor who not only has a lot of experience, but is also willing to share its learnings with the sponsor company. A vendor (CRO) may have significant clinical trial experience but does not have a rich RBM implementation experience. This needs to be validated carefully. Alignment in terms of scale and geographical footprint is also important when establishing a preferred partnership. Does the vendor actually have RBM capabilities at the required locations?

	Perhaps even more important than all of the above is the cultural fit, the alignment in vision, and the flexibility and transparency offered by the vendor. This would go a long way in terms of transforming this from a sponsor-vendor relationship to a true partnership.

	Principles of change management applied to RBQM

	Why do we need to address change management as part of RBQM preparation? What do we mean by change management? John Kotter, the guru of change management, refers to the eight stages that are required for the transformation and change to take place.

		Planning for and creating short-term wins

		Consolidating improvements and producing still more changes

Change management, in the context of instituting RBM and RBQM practices, refers to more than just the relationship between the sponsors and the sites. In following the stages listed above and implementing a vision, one needs to integrate a change management strategy from the risk identification stage itself and through the entire risk management planning and vendor selection stages, as well as developing a cohesive RBQM plan, addressing the needs of each stakeholder.

	Consider changes within the organizations and how they are affected by RBQM. Successful change management for the adoption of RBQM is much bigger than our relationship with the sites. It speaks to changes in our relationship and understanding of all of the clinical trial processes and study data. It speaks to transformations in how we conduct clinical trials and how we, as a community, work together and support each other. The new RBM approach is not a competition between stakeholders, but rather a way to assist each other in minimizing risk and enhancing the quality of the output of all stakeholders.

	Bottom line-change management and the adoption of RBQM means disruption. RBQM offers a disruption that will improve the organization.

	How do we differentiate resistance versus misunderstanding? How do we address those who see the benefits but are lacking in a commitment to the rigors required of change? Do we have time to make all the changes that are necessary? How do we prepare everyone for the bumpy first pass?

	As we have seen in implementing RBQM best practices, we are involved in actualizing all eight stages at the same time. We are often not in a position to build a powerful guiding coalition before implementing the changes. Yet, the implementation of all eight stages is required for success. There is no option to not implement RBQM. How can we use Kotter’s stages to help us in our preparation and planning?

	Pharma is experiencing a sense of urgency for adopting the principles of RBQM. For DM preparing internally for the adoption of RBQM, questions about systems, case report form (CRF) designs, and the timeliness of implementation lend all data activities a sense of urgency. Clinical operations teams express a sense of urgency regarding training and education on the new terms and processes. For regulatory, quality, and program managers, the sense of urgency is two-fold: first, assessing preparedness for RBQM, and, second, demonstrating preparedness. This is equally important for quality assurance (QA), as QA needs to ensure that all SOPs are in place, teams are trained on the same, and training records are in place as well.

	It is not only the leader who is in charge of developing the KRIs, but rather a coalition or a group that can work together. Many leaders look forward to building a guiding coalition in implementing RBQM. Who has the right skills for taking the lead? Who is writing the SOP?

	Monitors often fear that they have become dispensable. They fear that communication with the site will be centralized and personal interactions will disappear. They also fear that data managers may not understand how much the sites might skew the data. For regulatory, quality, and program managers, a guiding coalition of leaders is essential. Their underlying fear is that they will not be seen as an important voice in the leadership program. The main fear is disruption and loss of quality due to the new process.

	The vendor’s experience in change management is worth assessing. Both the sponsor and the vendor need to join hands in the implementation of a strategic change management approach within the sponsor’s organization, and attempt to address the concerns of naysayers.

	Creating a vision for the implementation of RBM and RBQM depends on a succinct determination of what are considered to be best practices within the organization. With many participants offering their visions, the essential elements can be obscured or may, at times, appear too discordant to be put into practice. As discussed earlier, it is a shared vision that brings the team together.

	For the data manager, a clear vision entails updating and executing new systems for RBQM. Who is doing the testing? When does one have time for testing? How are deviations from critical data points identified, interpreted, and actioned? For monitors, there may be conflicting visions. Some individuals may not see the changes as beneficial. Changes in visit schedules and data points to be reviewed can be disruptive. RBQM may feel too open-ended-how does one check for deviations and ensure quality at the site remotely? How does one retrain the site when difficulties are noted?

	RBM results in a disruptive change, not only in monitoring methodology but also in organizational structures and processes, which is not an easy thing to accept. It is important that the sponsor recognizes the commitment that these changes will require. How does its vision for RBQM answer the requirements of the regulatory authorities as well as address the institutional procedures?

	In communicating the vision, we need to show value. What is the value that RBM and RBQM will bring to our organization? It is important to let everyone know what is being planned and to address all concerns. Concerns vary in nature for different stakeholders. Data managers may have concerns regarding who is going to define the KRIs and how they will be collected in the study. Monitors may be concerned about what they should be communicating to the site and whether the site will buy into that message. RA may have its own perspective regarding whether RBQM will really add value, or will it just pose one more regulatory hurdle? Sites are often not even informed about the change in approach, despite being the most important stakeholders after the subjects. Technology teams may have concerns around whether their platform solution will pass muster with the end users (the central monitors) and with auditors.

	An organization must have internal RBM experts (or utilize external consultants) to be able to make meaningful, well-informed decisions. A lack of internal champions may often set this up for failure.

	All stakeholders within the company and externally should be empowered to follow and contribute to the vision. As the key in the structure of implementing RBM and RBQM in the institution, central monitors are connected to all parties within and outside the company. A clear message on the implementation strategy and how structures are changing will empower everyone that is impacted by RBQM.

	The DIA survey revealed that 40% of the participants found the lack of internal champions to be the biggest challenge within their organization, followed by a lack of vision.

6-Planning for and creating short-term wins

	It is good to demonstrate success early on to show where the organization is headed and what can be accomplished. Again, showing value is essential in creating short-term wins.

	The RBQM approach is integrated right from the start of the study.

 Central monitors can show how well its worked by identifying outliers in critical data points, and alerting monitors to take corrective action. The monitors can demonstrate short-term wins by addressing these issues remotely and actually enhance the quality of the data being received from the site. Sites that are doing well, despite fewer visits, should be recognized for their achievements. For sites that need more frequent visits, the win can be to demonstrate early identification of issues and ease of follow-up.

	Once one has a short-term win, it needs to keep the momentum in place for the next set of changes. This requires preparation and planning. The first short-term wins are great, but consolidating these improvements requires vigilance. One may not have all the answers to the problems and challenges along the way. For example, a KRI that is cumulative may not be sensitive enough to show us how the site is improving or worsening, but it gives us an opportunity to consolidate our improvements and produce more changes.

	The final reminder in any change management plan is that change takes time. If one rushes through the process without validating that everything is done, it is likely that the program, in turn, will suffer. We have often seen that starting and then stopping the implementation and going back to the old ways can be more harmful than waiting until everyone is prepared to get started.

	Although companies that transition an ongoing study to the RBM approach may find it more challenging than starting with an RBM approach from the beginning, it is not impossible. But it’s crucial to get started with RBM the sooner the better.

	Institutionalizing these new approaches always takes time. The secret to success for implementing change management includes transparency and the consistent ongoing communication of a shared vision.

		ICH H. (2016). Integrated addendum to ICH E6 (R1): guideline for good clinical practice E6 (R2). Retrieved from 

https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Step_4_2016_1109.pdf

		FDA. (2019). A Risk-Based Approach to Monitoring of Clinical Investigations. Retrieved from 

www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM633316.pdf

		Alsumidaie M, Andrianov A. Case Study: Using RBM to Evaluate Site Engagement. 

http://www.appliedclinicaltrialsonline.com/case-study-using-rbm-evaluate-and-predict-site-engagement

		FDA. (2007). Guidance for Industry and Review Staff Target Product Profile-A Strategic Development Process Tool. Retrieved from 

www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm080593.pdf

		Telfer G. Risk-Based Monitoring Insights from the Industry: Steve Young. 2016, 

https://cluepoints.com/risk-based-monitoring-insights-from-the-industry-steve-young-2/

		Limaye N, Jaguste V. Risk-Based Monitoring (RBM) Implementation: Challenges and Potential Solutions. 

Therapeutic Innovation & Regulatory Science

https://journals.sagepub.com/doi/abs/10.1177/2168479018769284

		Limaye N. The Three Biggest Challenges in RBM Related to Key Risk Indicators. 2018, 

https://cyntegrity.com/the-three-biggest-challenges-in-rbm-related-to-key-risk-indicators/

		Limaye N. Twenty Five Key Questions associated with Vendor Selection for Risk Based Monitoring. 2017, 

http://nymro.com/twenty-five-key-questions-associated-vendor-selection-risk-based-monitoring/

		Kotter J. Leading Change: Why Transformation Efforts Fail. 

http://www.lighthouseconsultants.co.uk/wp-content/uploads/2010/08/Kotter-Leading-Change-Why-transformation-efforts-fail.pdf

		Alsumidaie M, Proupín-Pérez M, Andrianov A, Widler B, Schiemann P. Ten Burning Questions about Risk-Based Study Management. A

http://cyntegrity.com/rbm-consortium-10-burning-questions-risk-based-study-management/

, Vanderbilt University, School of Medicine, Dept. of Biostatistics; 

, Practice Lead, Life Sciences, Applied Technology Solutions, Inc.; 

, Chief Operations Officer, CluePoints; 

, Sr. Manager, Boston Scientific Corporation; 

, Consultant-Retired FDA Director, Biostatistics; 

, Chief Scientific Officer, Cyntegrity GmbH

Articles

The importance of applying change management techniques throughout the preparation, planning, and execution of RBM and RBQM approaches.

A Roadmap for Implementing RBM and Quality Management

	At the 2018 DIA Annual Meeting, our DIA Clinical Data Management (CDM)/Risk-Based Monitoring (RBM) Community met with many people who were implementing or preparing to implement RBM and risk-based quality management (RBQM) approaches. Our attendees were looking for support and a forum to address their concerns. They knew that there were new guidelines and that the industry was anxious to get started with RBM activities, but they also wished to address the pitfalls one might expect. To address this need for a discussion of the changes taking place in our industry and to support each other in this endeavor, our group presented a segment of our roadmap beginning with a look at pre-study risk planning, vendor selection, and change management in RBM and RBQM.

has focused upon the implementation of RBQM activities in clinical trials. Multiple stakeholders were looking for real experiences that can be incorporated into their risk management strategies. The FDA’s comment period on the draft guidance “A Risk-Based Approach to Monitoring of Clinical Investigations”

To assist in this overall endeavor, we have been working on an RBM Roadmap produced by the CDM and RBM DIA Communities. We envision a set of best practices.  

Risk-based monitoring – pre-study risk planning

	ICH GCP E6 (R2) requires clinical trial sponsor companies to incorporate RBM in the risk management plan. 

and then continue into the development of an individual protocol for a study. Some aspects of the TPP, such as once-daily dosing or a positive safety profile, must be taken into account when developing the risk management plan for a study.

	The most impactful operational risks to a prospective clinical trial will most often be associated with the most important processes. Yet, it is not only the critical data and processes, as highlighted by ICH E6 R2, but also the systems, (devices, sensors or software apps) that are used in clinical trials that contribute to risk.

	Thus, to begin with, the study team identifies critical data, tools, and processes. One example of critical data is the survival data of patients that withdrew their informed consent or those that are lost-to-follow up in an outcome trial. These inputs may come from data management (DM), biostatistics, or clinical operations (CO). Examples of critical processes include the timely reporting of serious adverse events to the sponsor and the adequate management of those by the sponsor. This process depends on multiple stakeholders, including DM, pharmacovigilance/drug safety, regulatory affairs (RA), medical affairs (MA), and CO. In addition, it may be worthwhile to check inspection findings from the FDA or European Medicines Agency (EMA) to see which have been areas of concern based on historic data. Keeping a closer eye on these findings may be very helpful in the future. As we will discuss later, each study team member plays a unique and crucial role in the risk-identification process. Thus, while RBM focuses on risk-based monitoring, the integration of a systemic risk management strategy for identifying and managing risks-from study design to the submission of the clinical study report-has resulted in a more evolved descriptor called risk-based quality management (RBQM).

	Once the operational risks have been identified and characterized, the team develops a risk mitigation action plan to address the highest-priority risks. As previously mentioned, a high rate of lost-to-follow-up or discontinued patients might be an identified risk for the study. A number of mitigation actions might be considered for this risk:

	1.    Removing one or more ancillary procedures from the protocol requirements that are not critical to support study endpoints may reduce the relative burden on patients participating in the study.

	2.    Incorporate additional awareness training for sites related to this risk, including the need to check proactively with patients on their burden/comfort level with continued study participation, and what can be done to improve their experience, etc.

	3.    Implement one or more key risk indicator (KRI) metrics to identify sites that may be experiencing a high rate of lost-to-follow-ups, missed or missing visits, or withdrawn consents.

Vendor selection for the implementation of RBM

	At the start, one needs to have clarity within the organization as to what are the driving factors for implementing RBM. Is this a strategic decision with clear objectives (i.e., decreased number of site visits and increased quality) and clear milestones (i.e., decreased number of site visits) that improve quality? Many companies have not bought into this model, but are merely testing the waters. Some are recognizing the significant weightage that the FDA, EMA, TransCelerate BioPharma, and the latest ICH E6R2 guidelines have given to RBM and are making tactical decisions to perform a pilot study. However, it is important to remember that the implementation of RBM involves a significant investment in technology-and the benefits of this approach may not be evident by performing one small RBM pilot study. 

These are essential to measure the success of the RBM implementation strategy and of the partnership as well. 

	Implementing RBM is not just about analytics, technology, clinical, or RBM domain expertise. It is really about all of them. Finding a vendor who really understands the science of RBM is important. It is crucial to work with a vendor who not only has a lot of experience, but is also willing to share its learnings with the sponsor company. A vendor (CRO) may have significant clinical trial experience but does not have a rich RBM implementation experience. This needs to be validated carefully. Alignment in terms of scale and geographical footprint is also important when establishing a preferred partnership. Does the vendor actually have RBM capabilities at the required locations?

	Perhaps, even more important than all of the above is the cultural fit, the alignment in vision and the flexibility and transparency offered by the vendor.  This would go a long way in terms of transforming this from a sponsor-vendor relationship to a true partnership.

Principles of change management applied to RBQM

	Why do we need to address change management as part of RBQM preparation?  What do we mean by change management? John Kotter, the guru of change management, refers to the eight stages that are required for the transformation and change to take place.

	Change management, in the context of instituting RBM and RBQM practices, refers to more than just the relationship between the sponsors and the sites. In following the stages listed above and implementing a vision, one needs to integrate a change management strategy from the risk identification stage itself and through the entire risk management planning and vendor selection stages, as well as developing a cohesive RBQM plan, addressing the needs of each stakeholder.

	Consider changes within the organizations and how they are affected by RBQM. Successful change management for the adoption of RBQM is much bigger than our relationship with the sites.  It speaks to changes in our relationship and understanding of all of the clinical trial processes and study data. It speaks to transformations in how we conduct clinical trials and how we, as a community, work together and support each other. The new RBM approach is not a competition between stakeholders, but rather a way to assist each other in minimizing risk and enhancing the quality of the output of all stakeholders.

	As we have seen in implementing RBQM best practices, we are involved in actualizing all eight stages at the same time. We are often not in a position to build a powerful guiding coalition before implementing the changes. Yet, the implementation of all eight stages is required for success.  There is no option to not implement RBQM. How can we use Kotter’s stages to help us in our preparation and planning?

	It is not only the leader who is in charge of developing the KRIs, but rather a coalition or a group that can work together. Many leaders look forward to building a guiding coalition in implementing RBQM. Who has the right skills for taking the lead? Who is writing the SOP?  

	Creating a vision for the implementation of RBM and RBQM depends on a succinct determination of what are considered to be best practices within the organization. With many participants offering their visions, the essential elements can be obscured or may, at times, appear too discordant to be put into practice. As discussed earlier, it is a shared vision that brings the team together. 

	For the data manager, a clear vision entails updating and executing new systems for RBQM. Who is doing the testing? When does one have time for testing? How are deviations from critical data points identified, interpreted, and actioned? For monitors, there may be conflicting visions. Some individuals may not see the changes as beneficial. Changes in visit schedules and data points to be reviewed can be disruptive. RBQM may feel too open-ended-how does one check for deviations and ensure quality at the site remotely? How does one retrain the site when difficulties are noted? 

	RBM results in a disruptive change, not only in monitoring methodology but also in organizational structures and processes, which is not an easy thing to accept. It is important that the sponsor recognizes the commitment that these changes will require. How does its vision for RBQM answer the requirements of the regulatory authorities as well as address the institutional procedures? 

	In communicating the vision, we need to show value. What is the value that RBM and RBQM will bring to our organization? It is important to let everyone know what is being planned and to address all concerns. Concerns vary in nature for different stakeholders. Data managers may have concerns regarding who is going to define the KRIs and how they will be collected in the study. Monitors may be concerned about what they should be communicating to the site and whether the site will buy into that message. RA may have its own perspective regarding whether RBQM will really add value, or will it just pose one more regulatory hurdle? Sites are often not even informed about the change in approach, despite being the most important stakeholders after the subjects. Technology teams may have concerns whether their platform solution will pass muster with the end users (the central monitors) and with auditors.

	All stakeholders within the company and externally should be empowered to follow and contribute to the vision. As the key in the structure of implementing RBM and RBQM in the institution, central monitors are connected to all parties within and outside the company. A clear message on the implementation strategy and how structures are changing will empower everyone that is impacted by RBQM.  

Central monitors can show how well its worked by identifying outliers in critical data points, and alerting monitors to take corrective action. The monitors can demonstrate short-term wins by addressing these issues remotely and actually enhance the quality of the data being received from the site. Sites that are doing well, despite fewer visits, should be recognized for their achievements. For sites that need more frequent visits, the win can be to demonstrate early identification of issues and ease of follow-up. 

7-Consolidating improvements and producing still more changes

	RBM results in a disruptive change, not only in monitoring methodology, but also in organizational structures and processes-and change is not an easy thing to accept.  It is important that the sponsor company is aware of the commitment that these changes will require.

	The final reminder in any change management plan is that change takes time. If one rushes through the process without validating that everything is done, it is likely that the program, in turn, will suffer. We have often seen that starting and then stopping the implementation and going back to the old ways can be more harmful than waiting until everyone is prepared to get started.  

	Although companies that transition an ongoing study to the RBM approach may find it more challenging than starting with an RBM approach from the beginning, it is not impossible. But it crucial to get started with RBM the sooner the better.

, Project Manager, CTSpedia.org, Vanderbilt University, School of Medicine, Dept. of Biostatistics; 

, Practice Lead, Life Sciences, Applied Technology Solutions, Inc.

, Sr. Manager, Boston Scientific Corporation;

, Consultant-Retired FDA Director, Biostatistics

Integrated addendum to ICH E6 (R1): guideline for good clinical practice E6 (R2)

A Risk-Based Approach to Monitoring of Clinical Investigations

	3. Alsumidaie M, Andrianov A. Case Study: Using RBM to Evaluate Site Engagement. 

Guidance for Industry and Review Staff Target Product Profile-A Strategic Development Process Tool

	5. Telfer G. Risk-Based Monitoring Insights from the Industry: Steve Young. 2016, 

	6. Limaye N, Jaguste V. Risk-Based Monitoring (RBM) Implementation: Challenges and Potential Solutions. 

	7. Limaye N. The Three Biggest Challenges in RBM Related to Key Risk Indicators. 2018, https://cyntegrity.com/the-three-biggest-challenges-in-rbm-related-to-key-risk-indicators/

	8. Limaye N. Twenty Five Key Questions associated with Vendor Selection for Risk Based Monitoring. 2017, 

	9. Kotter J. Leading Change: Why Transformation Efforts Fail. 

	10. Alsumidaie M, Proupín-Pérez M, Andrianov A, Widler B, Johanna Schenk, MD, Schiemann P. Ten Burning Questions about Risk-Based Study Management.

ttp://cyntegrity.com/rbm-consortium-10-burning-questions-risk-based-study-management/

Integrating a systemic risk management strategy for identifying clinical trial risks-and the importance of applying change management techniques throughout the preparation, planning, and execution stages.  

A Roadmap for Implementing Risk-Based Monitoring and Quality Management

Due to ever increasing expenditures and difficulties associated with successful development and launch of innovative new medical products, the pharmaceutical industry has devoted substantial effort to root cause analyses.

 Taken together, these analyses show increasing complexity of clinical development with concomitant significant risks to the return on R&D Development investment.

 Furthermore, efforts to streamline clinical development processes with automation and digitization of data capture and reporting have failed to improve overall product approval rates or lower the cost of clinical research.

 The seeming paradox of process improvement without concomitant positive impacts on clinical development outcome led to the formation of a collaborative team in 2013 within the European Federation of Pharmaceutical Industries and Associations (EFPIA) called “Smart Program Design.” The Smart Program Design team published an article in 2015, “Smart Program Design through a Common Information Model,” which proposed improvements to the standard clinical program design approach involving precompetitive collaboration and information sharing.

 The group identified four key challenges to consistent and repeated smart clinical program designs:

		There is an inability to learn from past programs, both within organizations and externally

		Current industry information standards do not cover program level or the rationale behind these designs

		There is limited opportunity for progress in future opportunities in improving clinical program

	Thus, a gap exists for a common information model to describe the key building blocks essential in representing a clinical program and the corresponding design rationale, which allows for organization of information for reuse, facilitation of communication, and enablement of innovation. The common model should bridge between data and decision, and between decision and target product profile (TPP).

	A presentation of these concepts at the 2014 Clinical Data Interchange Standards Consortium (CDISC) IntraChange conference resulted in a recommendation by Wayne Kubick, Chief Technical Officer (CTO) of Health Level Seven International (HL7) and former CTO of CDISC, to initiate work on a broader set of information standards to address clinical development program design, including the capture of design decisions, rationale, and references. 

	Current data standards focus on describing and capturing the clinical trial protocol and data rather than the design process. No data standards that apply to research operations or their related critical decision-making process exist. Hence, we took advantage of an opportunity in the Pharmaceutical Users Software Exchange (PhUSE) Semantic Technology group to develop the Clinical Development Design (CDD) Framework.

	The purpose of the CDD Framework is to apply the principles and process of a design-based approach to the development program for a medical product, with emphasis on the identification, collection, and use of relevant information in a structured manner. This will enable decision-making and construct a learning system framework that facilitates ongoing improvement and efficiency. The model is based on the principle that decision-making is dependent on and enhanced by what is termed “enabling information.” In this model, the capture and use of enabling information is based on design principles. 

	Our first question was how are design decisions made in other industries? Other domains with ongoing dynamic decision-making can inform clinical research program development. In 

, Daniel Kahneman explores decision-making.

 He stresses that each industry needs ways to ensure the product’s quality from initial design through production and final inspection. In each stage, we are framing the problems that need to be solved and the corresponding decisions that need to be made. He suggests that at each of these stages-design, production, and final inspection-we look for ways to improve our design decisions in order to improve our product. Kahneman also stresses the concept of “noise” or lack of reproducibility, the variability in design decisions.

 He recommends ways to decrease the noise in design decisions by building algorithms, rather than depending solely on individual human judgment.

	Overcoming human inconsistency as a solution to better decision making is supported by the work of Theresa Winhusen.

 She focuses on the pre-implementation phase of multi-site pharmacological clinical trials and details the causes of delays and how they affect the study. The three primary causes for delays in the pre-implementation period are: (1) unforeseeable events, (2) underestimation of how long a project will take, and (3) difficulties in coordination of the many parties involved in the clinical trial. She proposes that with the proper tools, the impact of these types of delays can be overcome.

	Roger Buehler, consistent with Winhusen and Kahneman, shows examples of planning fallacies found in a wide range of industries, and presents tools that can assist in more accurately planning a project.

	While incorporating algorithms and tools into the decision-making process is advantageous to improve consistency and precision, the tools must be applied to sufficient and accurate information to be effective. K.M. Sutcliffe notes that too little as well as too much information can be difficult for a manager to interpret and apply to successfully manage a project.

 Dan Lovallo and Kahneman show how managers need the subjectivity removed from the design decisions to put out a more accurate estimation of the success of the product.

 Kahneman suggests managers and decision-makers can reduce noise and improve consistency by employing filters.

 Sheehan, Hirschfeld, Foster et al. point out that another way to reduce the noise is to use a set of common data elements (CDE) to obtain a more accurate signal.

	High quality design is most commonly manifested by success in the creation, development, commercialization, and optimization of a tangible product with a specified use, such as a car, a house, or a software application. While product users make multiple decisions during its use, the product is typically not changed on a daily basis. Design in the realm of clinical research is different, however, in that the research can be reevaluated and adjusted continuously. Clinical research on a medical product is dependent on the process of creation, development, and optimization of a scientifically robust, operationally feasible, and economically relevant research program to generate compelling evidence for internal decision-making, regulatory evaluation, and public confidence in its efficacy, safety, and cost-effectiveness. In contrast to the mass manufacture of consumer products, each clinical development program for every biomedical product is unique.

	Design decisions in clinical development programs and trials rely on consolidating, analyzing, weighing, and prioritizing a broad range of sources of information from guidance, past experience, expert advice, and real-world evidence. This information is typically captured in an unstructured format, in the form of clinical development plans, protocols, documents, and presentations.

	Such a process is not optimal. Designing a successful clinical research program can benefit from:

		Generation, capture, and effective sharing of data and historical information

	The CDD Framework we propose encompasses the support for design-based decisions and how to apply them to a clinical development program. It should accurately tell the story of how plans, data, and information evolve during the full life cycle of a therapeutic product from before clinical testing, then entry into first-in-human dosing, followed by the clinical trials, marketing authorization, as well as the experience from postmarketing surveillance and subsequent studies. This proposed CDD Framework is different from a Clinical Development Plan (CDP), which we define as the tactical execution of a research plan.

	The CDD Framework provides a methodology for the organization and preservation of information critical to medical product development decisions, information we refer to as “enabling information” (EI). EI includes process data, timelines, costs, and resource burden, among other quantitative and qualitative items. EI exists at every phase of the process, from concept to non-clinical and clinical testing, through regulatory approval, marketing, and post-marketing surveillance. The complete EI is ideally a comprehensive representation of an entire medical product development program. 

	Thus, the CDD Framework collates the EI and captures the perspectives of all stakeholders, scientists, administrators, care providers, patients, regulatory agencies, financial stakeholders, collaborators, and sponsors in the product development process. One approach to collating EI and populating the framework is to focus on three well-recognized, general objectives of medical product development (see Figure 1):

		Set goals that describe the product and patient population who will receive the product 

		Characterize the product administration (intervention) and define outcome measures (both efficacy and safety)

		Assess product viability against a benefit and risk profile

	These three well-recognized objectives of product development allow for a tentative mapping of our proposed CDD Framework. The following is a suggested approach to this mapping.

	When setting the goals, information on the condition (indication), the target population, and the expected benefit to that population (see Figure 2) is needed. This step also requires a description of the product under development, as well as measurable endpoints that are connected to the objectives and measures. The target population is typically defined by phenotype and demographics, but one can also consider genotype, geographic location, age, or developmental stage and lifestyle. Information generation and collection in this step may originate from many sources, including scientific and medical subject matter expertise, epidemiology, outcomes research, biostatistics, real-world data, and clinical operations.

	Characterizing the intervention requires investigation of its expected clinical and biological effects, and establishing a correlation between exposure and effect. These tasks will also require input from scientific and medical subject matter experts and statisticians. If relevant, a central role should be taken by experts in biomarker selection and analysis. This step requires an understanding of the nature of the intervention, dosing, and administration. Route, frequency, duration, and other parameters related to exposure need to be understood and documented. There needs to be a focus on defining informative and relevant outcome measures that vary with deterioration or amelioration of the clinical condition in a predictable manner. The clinical trial assessments must be feasible, acceptable, linked by time, and connected to the outcome measures.

	There needs to be an emphasis on risk management (see below), with assessments and outcome measures demonstrating an acceptable risk/benefit profile. This profile must be based on clinical trials with credible study design maximizing participant safety while minimizing bias and uncertainty as well as analysis of data assured of integrity. 

	To determine that all aspects are covered in the design, the perspectives of patients, investigators conducting the trial, oversight bodies, regulators, and the sponsor should be reviewed. Table 1 gives examples of some of the aspects to be validated as the output of the design of a clinical trial protocol within the CDD Framework.

	To summarize, the vast number of questions and ensuing answers required for each objective are interdependent and temporally linked. With that in mind, the design of the CDD Framework requires a comprehensive understanding of the design decisions and why these decisions were made. Each objective is supported by enabling data, consisting of regulatory information, early research and development data, clinical study data, epidemiological data, payer-focused data, and justifications.

	The clinical design effort will not necessarily be a process in which goals are set, interventions characterized, and risk-benefit demonstrated in a linear fashion. As new information is generated that more clearly demonstrates aspects of the benefit-risk profile, the intervention, the condition, or the target population may become better characterized, leading to the setting of new goals. Also, at any given stage of medical product development or the life cycle management of a medical product, there will be key questions that remain unanswered, or the uncertainty about the outcomes based on the collected evidence may become greater than desired. The CDD Framework captures these gaps and uncertainties, and helps moving forward by prompting the establishment of a logical plan for setting goals that will fill those gaps and decrease uncertainties. 

	The CDD Framework is intended to evolve into a knowledge base for the design of products supporting key design objectives, and to facilitate the validation of those objectives. It should properly represent the interdependencies between design constraints, decisions, and information-and facilitate the communication of those relationships between stakeholders to ensure that the work designated in the CDD actually takes place.

	There are many types of risks to be addressed throughout the CDD Framework, each of which requires appropriate management. Besides risks relating to regulatory issues, risk considerations within CDD include those for business and scientific validity, as well as concerns such as loss of time, opportunity, effort, and reputation.

	Certain risks will not be addressed in the CDD Framework, because those risks could not reasonably be attributable to aspects of design, such as reporting delays, data issues, misconduct identified, incorrectly enrolled patients at a site, etc. Moreover, the European Medical Association (EMA) notes that “risk management is a systematic process for the assessment, control, communication, and review of risks associated with the planning and conduct of clinical trials and clinical development programs.”

 The CDD Framework should help to identify risks to determine what can happen, when, where, how, and why-and to enable analysis of the likelihood of the occurrence and detection. Finally, there should be decisions on acceptable tolerance levels.

	Implementation of the CDD Framework can leverage many of the tools, methods, and technologies that have been applied to other areas of knowledge acquisition, storage, and analysis. Utilization of various combinations of these tools can enhance the power and interoperability of the CDD Framework for the capture of enabling information, decision options and rationale, and relationships of choices to outcomes.

	The conventional approach to store and access information is by categorizing and indexing information into relational databases based on static structures. XML-based data standards, such as those currently used in CDISC data capture and transfer standards, are designed for such structured data. A primary challenge here is that information associated with design thinking can initially be apparently unstructured, incomplete, or subject to further changes and definitions. However, similar to data standards, updates are inevitable.

	Semantic technologies represent a family of technologies related to the capture, storage, indexing, querying, and classification of unstructured data governed by the World Wide Web Consortium (W3C).

 In particular, semantic technology allows the development of flexible information models, where concepts can have different meanings, links can be created between disparate information sources, and information is incomplete. 

	For semantic technology, well-defined rules are established that convey meaning to specific words.

 In doing so, an assortment of words that have the same meaning (as defined by semantic rules) allows for an overarching search with far greater returns than relying on a search for each individual word. Moreover, with linked web data, the user can search for information on the Internet and circumvent the need to rely entirely on conventional relational databases. However, some of the linked data sources can be:

	The flexibility offered by semantic technology makes it well suited to be leveraged for the CDD information model. We need to obtain relevant information for input into the program design, as well as the intended protocols, appropriate data analyses, clinical study reports, and regulatory submission. These components, attainable through semantic technology, are incorporated in the CDD Framework in a comprehensive yet temporal arrangement.

	For any information model, a common vocabulary or ontology is a prerequisite. Ontologies help us to organize terms and define relationships, to enable reuse of domain knowledge and separate domain knowledge from the operational knowledge.

 Ontologies organize domain knowledge in a common structure of entities and their relationships.

	An example of applying ontologies for semantic data supporting clinical research is the ontology for FDA regulations in the Resource Description Framework (RDF), being created by the PhUSE Reg2RDF group.

 Currently, the work focuses on indexing 21 CFR terms and evaluating the key phrases and presenting a web interface. The best way to organize the review process is part of their development process. The CDD Framework aims to apply a similar approach for establishing a corresponding ontology.

	In order to develop the CDD ontology and to begin mapping the design process, we can start with the checklists from the FDA (including the TPP and prescribing information labeling templates and checklists) and quality by design reference tools from the Clinical Trials Transformation Initiative (CTTI) as well as other data sources.

	Before we can apply the terms, we must follow the examples found in developing biomedically-based common data elements (CDE).

 We need to bring together all of the above terms in a single resource with links, redundancies, and hierarchical relationships. Our starting place in building our ontology is Vanderbilt University’s Research Electronic Data Capture (REDCap) application.

 This application allows us to capture redundancies and provide links between the terms that we are referencing.

	A semantic information model can serve as the backbone for software applications. However, a visual representation of the CDD Framework can also serve as a guide for design teams on its own. A two-dimensional open source concept mapping tool is available from the Florida Institute for Human & Machine Cognition (IHMC).

 Using this software, we will be able “construct, navigate, share, and criticize” our models of the CDD Framework. Cmap tools allow us to not only share our concepts and understanding of the CDD Framework, but to link our maps to related concept maps and other types of media.26,27

	Figure 3 maps the FDA’s 21 CFR Part 201 regulations with the Cmap tool. We can then hyperlink and download each of these sections of the regulations. For the CDD Framework, we can develop Cmaps for each of our areas: setting goals, characterizing the intervention and delivering the medical product, and link the information that medical/scientific, operations, and regulatory groups need for their work.

	The Cmap concept maps give us a two-dimensional mapping. The next step beyond the two-dimensional concept maps is the work done by Kerstin Forsberg and Maria Benjegard with Neo4j mapping.

 This tool allows for multi-dimensional modeling and linking.

	Figure 4 depicts the concept of a visual, interactive information model representing the interdependencies between activities, information, and decisions.

		When it needs to happen, influencing work to come

How the Clinical Development Design (CDD) Framework can offer repeatable, reusable clinical designs based on "enabling information."


Mary Banach

Articles

A Roadmap for Implementing RBM and Quality Management

A Roadmap for Implementing Risk-Based Monitoring and Quality Management

Barriers and Solutions to Smart Clinical Program Designs

Barriers and Solutions to Smart Clinical Program Designs