Hon-Sum Ko

Due to ever increasing expenditures and difficulties associated with successful development and launch of innovative new medical products, the pharmaceutical industry has devoted substantial effort to root cause analyses.

 Taken together, these analyses show increasing complexity of clinical development with concomitant significant risks to the return on R&D Development investment.

 Furthermore, efforts to streamline clinical development processes with automation and digitization of data capture and reporting have failed to improve overall product approval rates or lower the cost of clinical research.

 The seeming paradox of process improvement without concomitant positive impacts on clinical development outcome led to the formation of a collaborative team in 2013 within the European Federation of Pharmaceutical Industries and Associations (EFPIA) called “Smart Program Design.” The Smart Program Design team published an article in 2015, “Smart Program Design through a Common Information Model,” which proposed improvements to the standard clinical program design approach involving precompetitive collaboration and information sharing.

 The group identified four key challenges to consistent and repeated smart clinical program designs:

		There is an inability to learn from past programs, both within organizations and externally

		Current industry information standards do not cover program level or the rationale behind these designs

		There is limited opportunity for progress in future opportunities in improving clinical program

	Thus, a gap exists for a common information model to describe the key building blocks essential in representing a clinical program and the corresponding design rationale, which allows for organization of information for reuse, facilitation of communication, and enablement of innovation. The common model should bridge between data and decision, and between decision and target product profile (TPP).

	A presentation of these concepts at the 2014 Clinical Data Interchange Standards Consortium (CDISC) IntraChange conference resulted in a recommendation by Wayne Kubick, Chief Technical Officer (CTO) of Health Level Seven International (HL7) and former CTO of CDISC, to initiate work on a broader set of information standards to address clinical development program design, including the capture of design decisions, rationale, and references. 

	Current data standards focus on describing and capturing the clinical trial protocol and data rather than the design process. No data standards that apply to research operations or their related critical decision-making process exist. Hence, we took advantage of an opportunity in the Pharmaceutical Users Software Exchange (PhUSE) Semantic Technology group to develop the Clinical Development Design (CDD) Framework.

	The purpose of the CDD Framework is to apply the principles and process of a design-based approach to the development program for a medical product, with emphasis on the identification, collection, and use of relevant information in a structured manner. This will enable decision-making and construct a learning system framework that facilitates ongoing improvement and efficiency. The model is based on the principle that decision-making is dependent on and enhanced by what is termed “enabling information.” In this model, the capture and use of enabling information is based on design principles. 

	Our first question was how are design decisions made in other industries? Other domains with ongoing dynamic decision-making can inform clinical research program development. In 

, Daniel Kahneman explores decision-making.

 He stresses that each industry needs ways to ensure the product’s quality from initial design through production and final inspection. In each stage, we are framing the problems that need to be solved and the corresponding decisions that need to be made. He suggests that at each of these stages-design, production, and final inspection-we look for ways to improve our design decisions in order to improve our product. Kahneman also stresses the concept of “noise” or lack of reproducibility, the variability in design decisions.

 He recommends ways to decrease the noise in design decisions by building algorithms, rather than depending solely on individual human judgment.

	Overcoming human inconsistency as a solution to better decision making is supported by the work of Theresa Winhusen.

 She focuses on the pre-implementation phase of multi-site pharmacological clinical trials and details the causes of delays and how they affect the study. The three primary causes for delays in the pre-implementation period are: (1) unforeseeable events, (2) underestimation of how long a project will take, and (3) difficulties in coordination of the many parties involved in the clinical trial. She proposes that with the proper tools, the impact of these types of delays can be overcome.

	Roger Buehler, consistent with Winhusen and Kahneman, shows examples of planning fallacies found in a wide range of industries, and presents tools that can assist in more accurately planning a project.

	While incorporating algorithms and tools into the decision-making process is advantageous to improve consistency and precision, the tools must be applied to sufficient and accurate information to be effective. K.M. Sutcliffe notes that too little as well as too much information can be difficult for a manager to interpret and apply to successfully manage a project.

 Dan Lovallo and Kahneman show how managers need the subjectivity removed from the design decisions to put out a more accurate estimation of the success of the product.

 Kahneman suggests managers and decision-makers can reduce noise and improve consistency by employing filters.

 Sheehan, Hirschfeld, Foster et al. point out that another way to reduce the noise is to use a set of common data elements (CDE) to obtain a more accurate signal.

	High quality design is most commonly manifested by success in the creation, development, commercialization, and optimization of a tangible product with a specified use, such as a car, a house, or a software application. While product users make multiple decisions during its use, the product is typically not changed on a daily basis. Design in the realm of clinical research is different, however, in that the research can be reevaluated and adjusted continuously. Clinical research on a medical product is dependent on the process of creation, development, and optimization of a scientifically robust, operationally feasible, and economically relevant research program to generate compelling evidence for internal decision-making, regulatory evaluation, and public confidence in its efficacy, safety, and cost-effectiveness. In contrast to the mass manufacture of consumer products, each clinical development program for every biomedical product is unique.

	Design decisions in clinical development programs and trials rely on consolidating, analyzing, weighing, and prioritizing a broad range of sources of information from guidance, past experience, expert advice, and real-world evidence. This information is typically captured in an unstructured format, in the form of clinical development plans, protocols, documents, and presentations.

	Such a process is not optimal. Designing a successful clinical research program can benefit from:

		Generation, capture, and effective sharing of data and historical information

	The CDD Framework we propose encompasses the support for design-based decisions and how to apply them to a clinical development program. It should accurately tell the story of how plans, data, and information evolve during the full life cycle of a therapeutic product from before clinical testing, then entry into first-in-human dosing, followed by the clinical trials, marketing authorization, as well as the experience from postmarketing surveillance and subsequent studies. This proposed CDD Framework is different from a Clinical Development Plan (CDP), which we define as the tactical execution of a research plan.

	The CDD Framework provides a methodology for the organization and preservation of information critical to medical product development decisions, information we refer to as “enabling information” (EI). EI includes process data, timelines, costs, and resource burden, among other quantitative and qualitative items. EI exists at every phase of the process, from concept to non-clinical and clinical testing, through regulatory approval, marketing, and post-marketing surveillance. The complete EI is ideally a comprehensive representation of an entire medical product development program. 

	Thus, the CDD Framework collates the EI and captures the perspectives of all stakeholders, scientists, administrators, care providers, patients, regulatory agencies, financial stakeholders, collaborators, and sponsors in the product development process. One approach to collating EI and populating the framework is to focus on three well-recognized, general objectives of medical product development (see Figure 1):

		Set goals that describe the product and patient population who will receive the product 

		Characterize the product administration (intervention) and define outcome measures (both efficacy and safety)

		Assess product viability against a benefit and risk profile

	These three well-recognized objectives of product development allow for a tentative mapping of our proposed CDD Framework. The following is a suggested approach to this mapping.

	When setting the goals, information on the condition (indication), the target population, and the expected benefit to that population (see Figure 2) is needed. This step also requires a description of the product under development, as well as measurable endpoints that are connected to the objectives and measures. The target population is typically defined by phenotype and demographics, but one can also consider genotype, geographic location, age, or developmental stage and lifestyle. Information generation and collection in this step may originate from many sources, including scientific and medical subject matter expertise, epidemiology, outcomes research, biostatistics, real-world data, and clinical operations.

	Characterizing the intervention requires investigation of its expected clinical and biological effects, and establishing a correlation between exposure and effect. These tasks will also require input from scientific and medical subject matter experts and statisticians. If relevant, a central role should be taken by experts in biomarker selection and analysis. This step requires an understanding of the nature of the intervention, dosing, and administration. Route, frequency, duration, and other parameters related to exposure need to be understood and documented. There needs to be a focus on defining informative and relevant outcome measures that vary with deterioration or amelioration of the clinical condition in a predictable manner. The clinical trial assessments must be feasible, acceptable, linked by time, and connected to the outcome measures.

	There needs to be an emphasis on risk management (see below), with assessments and outcome measures demonstrating an acceptable risk/benefit profile. This profile must be based on clinical trials with credible study design maximizing participant safety while minimizing bias and uncertainty as well as analysis of data assured of integrity. 

	To determine that all aspects are covered in the design, the perspectives of patients, investigators conducting the trial, oversight bodies, regulators, and the sponsor should be reviewed. Table 1 gives examples of some of the aspects to be validated as the output of the design of a clinical trial protocol within the CDD Framework.

	To summarize, the vast number of questions and ensuing answers required for each objective are interdependent and temporally linked. With that in mind, the design of the CDD Framework requires a comprehensive understanding of the design decisions and why these decisions were made. Each objective is supported by enabling data, consisting of regulatory information, early research and development data, clinical study data, epidemiological data, payer-focused data, and justifications.

	The clinical design effort will not necessarily be a process in which goals are set, interventions characterized, and risk-benefit demonstrated in a linear fashion. As new information is generated that more clearly demonstrates aspects of the benefit-risk profile, the intervention, the condition, or the target population may become better characterized, leading to the setting of new goals. Also, at any given stage of medical product development or the life cycle management of a medical product, there will be key questions that remain unanswered, or the uncertainty about the outcomes based on the collected evidence may become greater than desired. The CDD Framework captures these gaps and uncertainties, and helps moving forward by prompting the establishment of a logical plan for setting goals that will fill those gaps and decrease uncertainties. 

	The CDD Framework is intended to evolve into a knowledge base for the design of products supporting key design objectives, and to facilitate the validation of those objectives. It should properly represent the interdependencies between design constraints, decisions, and information-and facilitate the communication of those relationships between stakeholders to ensure that the work designated in the CDD actually takes place.

	There are many types of risks to be addressed throughout the CDD Framework, each of which requires appropriate management. Besides risks relating to regulatory issues, risk considerations within CDD include those for business and scientific validity, as well as concerns such as loss of time, opportunity, effort, and reputation.

	Certain risks will not be addressed in the CDD Framework, because those risks could not reasonably be attributable to aspects of design, such as reporting delays, data issues, misconduct identified, incorrectly enrolled patients at a site, etc. Moreover, the European Medical Association (EMA) notes that “risk management is a systematic process for the assessment, control, communication, and review of risks associated with the planning and conduct of clinical trials and clinical development programs.”

 The CDD Framework should help to identify risks to determine what can happen, when, where, how, and why-and to enable analysis of the likelihood of the occurrence and detection. Finally, there should be decisions on acceptable tolerance levels.

	Implementation of the CDD Framework can leverage many of the tools, methods, and technologies that have been applied to other areas of knowledge acquisition, storage, and analysis. Utilization of various combinations of these tools can enhance the power and interoperability of the CDD Framework for the capture of enabling information, decision options and rationale, and relationships of choices to outcomes.

	The conventional approach to store and access information is by categorizing and indexing information into relational databases based on static structures. XML-based data standards, such as those currently used in CDISC data capture and transfer standards, are designed for such structured data. A primary challenge here is that information associated with design thinking can initially be apparently unstructured, incomplete, or subject to further changes and definitions. However, similar to data standards, updates are inevitable.

	Semantic technologies represent a family of technologies related to the capture, storage, indexing, querying, and classification of unstructured data governed by the World Wide Web Consortium (W3C).

 In particular, semantic technology allows the development of flexible information models, where concepts can have different meanings, links can be created between disparate information sources, and information is incomplete. 

	For semantic technology, well-defined rules are established that convey meaning to specific words.

 In doing so, an assortment of words that have the same meaning (as defined by semantic rules) allows for an overarching search with far greater returns than relying on a search for each individual word. Moreover, with linked web data, the user can search for information on the Internet and circumvent the need to rely entirely on conventional relational databases. However, some of the linked data sources can be:

	The flexibility offered by semantic technology makes it well suited to be leveraged for the CDD information model. We need to obtain relevant information for input into the program design, as well as the intended protocols, appropriate data analyses, clinical study reports, and regulatory submission. These components, attainable through semantic technology, are incorporated in the CDD Framework in a comprehensive yet temporal arrangement.

	For any information model, a common vocabulary or ontology is a prerequisite. Ontologies help us to organize terms and define relationships, to enable reuse of domain knowledge and separate domain knowledge from the operational knowledge.

 Ontologies organize domain knowledge in a common structure of entities and their relationships.

	An example of applying ontologies for semantic data supporting clinical research is the ontology for FDA regulations in the Resource Description Framework (RDF), being created by the PhUSE Reg2RDF group.

 Currently, the work focuses on indexing 21 CFR terms and evaluating the key phrases and presenting a web interface. The best way to organize the review process is part of their development process. The CDD Framework aims to apply a similar approach for establishing a corresponding ontology.

	In order to develop the CDD ontology and to begin mapping the design process, we can start with the checklists from the FDA (including the TPP and prescribing information labeling templates and checklists) and quality by design reference tools from the Clinical Trials Transformation Initiative (CTTI) as well as other data sources.

	Before we can apply the terms, we must follow the examples found in developing biomedically-based common data elements (CDE).

 We need to bring together all of the above terms in a single resource with links, redundancies, and hierarchical relationships. Our starting place in building our ontology is Vanderbilt University’s Research Electronic Data Capture (REDCap) application.

 This application allows us to capture redundancies and provide links between the terms that we are referencing.

	A semantic information model can serve as the backbone for software applications. However, a visual representation of the CDD Framework can also serve as a guide for design teams on its own. A two-dimensional open source concept mapping tool is available from the Florida Institute for Human & Machine Cognition (IHMC).

 Using this software, we will be able “construct, navigate, share, and criticize” our models of the CDD Framework. Cmap tools allow us to not only share our concepts and understanding of the CDD Framework, but to link our maps to related concept maps and other types of media.26,27

	Figure 3 maps the FDA’s 21 CFR Part 201 regulations with the Cmap tool. We can then hyperlink and download each of these sections of the regulations. For the CDD Framework, we can develop Cmaps for each of our areas: setting goals, characterizing the intervention and delivering the medical product, and link the information that medical/scientific, operations, and regulatory groups need for their work.

	The Cmap concept maps give us a two-dimensional mapping. The next step beyond the two-dimensional concept maps is the work done by Kerstin Forsberg and Maria Benjegard with Neo4j mapping.

 This tool allows for multi-dimensional modeling and linking.

	Figure 4 depicts the concept of a visual, interactive information model representing the interdependencies between activities, information, and decisions.

		When it needs to happen, influencing work to come

		Where to learn more (e.g., guides to formal procedure and guidance)

Articles

How the Clinical Development Design (CDD) Framework can offer repeatable, reusable clinical designs based on "enabling information."


Barriers and Solutions to Smart Clinical Program Designs

	Figure 3 below (click to enlarge) maps the FDA’s 21 CFR Part 201 regulations with the Cmap tool. We can then hyperlink and download each of these sections of the regulations. For the CDD Framework, we can develop Cmaps for each of our areas: setting goals, characterizing the intervention and delivering the medical product, and link the information that medical/scientific, operations, and regulatory groups need for their work.

	Figure 5 (click to enlarge) is an example what is considered for “Evaluation of Standards” in Benjegard and Forsberg’s 