	The instructions of a legendary 19th recipe for jugged hare began: “First catch your hare.” By the same token, the first task of drug developers in this new year should be to make their thoughts known on how drug development should be assessed-and they have the perfect opportunity to do just that with the construction of a new strategy from one of the world’s leading authorities.

	The European Medicines Agency (EMA) has drafted a 

 for closer engagement with regulatory science over the next decade, and it is actively seeking comments on its approach. To make it even easier for input, the agency has put a 

	The proposed “Regulatory Science to 2025” strategy “aims to build a more adaptive regulatory system that will encourage innovation in human and veterinary medicine,” according to Guido Rasi, EMA’s executive director. It is an explicit response by European regulators to the challenges that have emerged in recent years from new technologies. Against the background of the long-established classic medicines testing systems dependent on “gold-standard” randomized clinical trials, it holds out the prospect of a genuine assessment of the benefits and risks of innovative therapies and diagnostics.

	Stakeholders are being asked to give views on what the priorities should be in this recast. The proposed top themes in the draft strategy range from “catalyzing the integration of science and technology in medicine development” and “driving collaborative evidence generation-improving the scientific quality of evaluations” to “enabling and leveraging research and innovation in regulatory science,” with “advancing patient-centered access to medicines” and “addressing emerging health threats” thrown into the mix, too. Should the clever money go into supporting developments in precision medicine, biomarkers, and ‘omics, or into translating cell, genes, and tissue-based products into patient treatments through the EMA’s (so far underused) advanced therapy medicinal products scheme? Do novel manufacturing technologies get enough attention at present? Should regulators create an integrated evaluation pathway for the assessment of medical devices, 

	Should the accent be on leveraging novel non-clinical models and foster innovation in clinical trials, or catering for more digital clinical data generation? What emphasis should be given to initiatives for special populations, to making more use of modeling and simulation? And how far should artificial intelligence play a role in decision-making?

	The draft ranges across all these questions. But there is also scope for offering additional priorities or approaches, too. “Are there any significant elements missing in this strategy? Please elaborate,” asks the questionnaire. “We want to hear from our stakeholders whether they consider this strategy is ambitious enough,” confirmed Rasi, launching the consultation at the end of 2018. So, stakeholders, you have until June 30 to influence the future. Over to you.

	Right at the start of the year, the calls for radical change in the way that drug development is organized are emerging thick and fast in Europe. The first week back at work saw a conference in the European Parliament where speaker after speaker denounced the failings of the current model and the rigidity of the authorization and delivery systems in the face of new science and new technology. The central message was the same: opportunities for better care are being missed, and shouldn’t be. Read more about the discussions 

 is a freelance journalist who specializes in European health affairs and is based in Brussels, Belgium

Articles

News

EMA drafted a plan for closer engagement with regulatory science in the next ten years, and it is looking for comments on its approach.

EMA Seeks Input on its Regulatory Science Strategy

	Guillaume Marquis-Gravel, MD, a research fellow with the Duke Clinical Research Institute (DCRI), and Kevin Monroe, director of innovation at DCRI, discuss the advancements and related challenges of incorporating new technological tools into drug development, including eClinical approaches currently undertaken in the foundation’s own studies.

Q: Can you provide an overview of the software and technologies the DCRI uses in clinical trials and why they’re being used?

	Technology plays an increasingly integral role in the research process. As a research organization, the DCRI utilizes a bevy of software products to facilitate the execution of research projects and the data analysis of findings. These include tools to:

		Streamline site selection and activation

		Enrich patient recruitment and retention efforts

		Facilitate contracting and payment processing

		Collect data from patients, providers, and health systems (including range of EDC, EHR, ePRO, eCOA, clinical registries)

		Monitor data collection and safety profile

		Evaluate and adjudicate clinical endpoints

		Provide ongoing project management, study oversight and reporting

	Historically, many of the tools needed for research have been provided through in-house software development efforts of niche vendors largely architecting their solutions through on-premise infrastructure. With the explosive growth of cloud technologies, the vendor landscape has grown dramatically.

	The trend toward cloud-based solutions is clearly underway. With this rapid rise of viable solutions, the challenge has changed from an era of too few solutions to navigating a complex landscape of point solutions. The ongoing efforts to provide a platform and ecosystem for clinical research is long overdue and an advance which we welcome enthusiastically.

Q: When so many processes are still conducted on paper, can the right technology make or break a clinical trial?

	Inasmuch as technologies can provide economies of scale at every stage of the trial process, from design to analysis, poor implementation can lead to wasted resources. Capitalizing on the efficiencies enabled by technologies to facilitate the conduct of clinical trials requires the expansion of the study team to include information technologies specialists, health systems specialists, and data scientists.

	Also, experts in advanced analytics are the linchpin required to enable the integration of massive volumes of data accrued through emerging technologies. EHR-based, electronic patient-reported outcomes (ePROs) from mobile health applications and digital biomarkers derived from biosensors allow researchers to streamline safety monitoring and endpoint ascertainment processes by eliminating requirements for in-clinic follow-up with traditional paper-based case-report forms.

	However, no compromise on the quality of data can be made, as evidentiary standards for regulatory and scientific purposes needs to be maintained. For example, before replacing traditional central blinded endpoint adjudication by EHR-derived data, completeness, accuracy, and precision of the data requires validation efforts to be conducted, and regular curation of the databases is necessary. Digitization of the protocols has the potential to generate efficiencies and support complex operationalization of protocols at the site level, at the cost of extensive training for the personnel in charge of digitization.

	Inaccurate data attribution, mobile device failure, programmed obsolescence, and silos of representativeness excluding participants with poor technological literacy all constitute challenges specific to the conduct of technology-enabled trials that have to be taken into consideration early in the planning stage.

Q: If you had one technology to use on a clinical trial, what would it be and why?

	Shifting the paradigm from site-centric to patient-centric clinical trials is best exemplified by the conduct of the so-called virtual trials (also called decentralized trials) made possible by a set of mobile technologies to continuously capture ePROs and digital biomarkers such as step count, heart rate, and blood pressure. Upon proper validation, these web-based platforms-where patients can be consented, interact with investigators through telemedicine devices, record ePROs, and automatically upload vital signs through applications on the participants’ own smartphones-not only have the potential to reduce the time burden imposed on the participants and study teams by obviating the requirement for traditional in-clinic study visits, but also permit continuous real-time data collection more representative of the daily living environment of participants, and empower them to be more engaged in research.

Q: Are there any processes currently conducted on software programs that you have found problematic, wanting, or all-over challenging?

	The movement toward technology-enabled clinical research in which software products are integral to the research process does introduce new challenges. As new products are introduced, they require a comprehensive approach to change management both within the coordinating center as well as the participating research sites. Common product-level challenges include poorly designed user experiences and reliance on built-in data visualizations and reporting with limited ability to access raw data. Many of the solutions fail to adequately address the totality of the problem space in which their offerings focus.

	For example, while numerous products focus on the study startup process, the results have not been as successful as anticipated largely because they are designed from a study-specific focus, whereas both the coordinating center and research sites are managing their own unique portfolio of research studies. Platforms that empower both perspectives with an enterprise approach are needed to realize the economies of scale envisioned.

A discussion of the advancements and related challenges of incorporating new technological tools into drug development, including eClinical approaches currently undertaken in the foundation’s own studies.




Q&A: Capitalizing on Technology-Enabled Clinical Research

	The 2018/2019 CRO Industry Compensation, Turnover, and Plan Design Trends Report from BDO USA, LLP (view 

of compensation and benefits trends in the CRO industry over the last 10 years and includes a special focus on clinical research associates (CRAs). From average merit increases from individual countries, to incentives per position, to retention bonuses and much more, the BDO report offers a robust look at an industry that has quickly eclipsed its “small company mentality” roots.

	Judy Canavan, managing director and leader of BDO’s Compensation Surveys practice and founder of the company HR+Survey Solutions, told 

 that the CRO industry is not only growing fast, but is also becoming more sophisticated. However, she noted, “they need to start looking at different strategies and not just cash” to retain employees.

	Turnover rates among CRO professionals is pretty well-known in the industry. If there is one aspect of sponsor consternation, it is the loss of a well-liked and responsible project manager, a situation that can happen many times during any one seven-to-10-year clinical trial study. (

http://www.pharmexec.com/cro-oversight-roundtable-0

). In BDO’s report, from 2008 to 2018, turnover for the roles of data manager and project manager have stayed fairly level in the 14% to 22% range, whereas the clinical monitoring role scores consistently above that in the 22% to 28% range.

	Those CRA rates, says Canavan, continue unchanged for a variety of reasons that, too, remain unchanged-including extensive travel, loose connections based on home-based reporting, and demanding schedules. According to Canavan, CRAs ramp up their skills pretty quickly and by years two or three, find themselves very valuable; but their merit raises, hovering at 3% to 5%, don’t match what their market value is. “The merit raises just don’t reflect how quickly they are progressing,” says Canavan. “Suddenly, an HR department may be in a fire drill to get someone a pay increase.”

 is echoed in the report: additional strategies should be well-examined and considered for implementation. These include annual incentive plans, long-term incentive plans, and attraction and retention bonuses. The report found that sign-on bonuses for new employees are used more frequently than retention bonuses. Meanwhile, retention bonuses lag far behind. Canavan said, “Companies may want to rethink their use of bonuses as a way to help improve retention rates. A well-designed plan that focuses on both team and individual performance can provide a nice reward to good performers.”

	Canavan also suggested that companies consider administering bonuses farther down in the organization to help retention. In addition, with increased external training and accreditation opportunities, HR could use those as a benefit to differentiate the organization from other companies. Canavan concludes, “The industry is robust, and it is evolving and maturing. Career-wise, CROs remain a good opportunity.”

From the Editor

From average merit increases from individual countries, to incentives per position, to retention bonuses and much more, a recent BDO report offers a robust look at an industry that has quickly eclipsed its “small company mentality” roots.

The Case for Thinking Outside the HR Box

Toward the end of last year, the life sciences industry was immersed in chatter about the new and the “what’s next,” picking out key trends that will define 2019. It’s a great time to get ahead of the game, but also to separate the hype from reality. Here are the five transformation trends with regard to clinical trials that I’m watching right now.

1. Data- and technology-enabled patient discovery and recruiting.

 Patient recruitment has been conducted in the same way for years now. First, we ask the research sites to identify appropriate patients, then we fill in the gaps with media. While we have made some headway in moving toward more effective social and digital advertising, recruitment is still largely “push” marketing. That’s been changing over the last three years as companies like Circuit Clinical and Deep6 AI begin to utilize artificial intelligence and natural language processing to “crawl” medical records to find patients for trials. Not only is this a smarter way to find qualified patients for existing studies, it may pave a pathway for discovering whole cohorts of patients before a trial is even designed.

2. Seamless patient “CRM” and retention management. 

With new access to programmatic browsing behavior, we are beginning to see the advent of data-driven retention and adherence solutions. Using individual-level targeted insights and trigger-based digital media, researchers and payers are more empowered to manage patient needs more proactively. For example, we can now assess a risk for dropout score for patients in trials that require rescue medications by watching online signals that expose when a patient may be experiencing early warning signs of an event (relapsed, refractory episode, AE, etc.)

3. Incorporation of real-world data into trial protocols.

 The more we see the advancement of digital biomarkers from social media, wearables, electronic health records (EHRs), and smartphones, the more we can begin to include real-world data in our protocol design and not just as “ancillary” or anecdotal information. There has been huge focus lately on tracking social determinants of health, for example, and Stanford has recently outlined its 400,000-person Apple Watch heart rate study. The impact of this could be enormous when you consider that right now, we only seek “perfect patients” for our trials, rendering many drugs potentially less safe or effective for those in demographics (age, race, gender) that the study was not designed for. 

 It is safe to say virtual trials are moving out of the proof-of-concept space into the full-implementation space. There was an interesting panel at CNS that examined all of the different virtual trial models at play. While there was no shortage of debate about what makes a trial ideal for a virtual model (heavy PK sampling need not apply!), it was clear that virtual trials are here to stay. Leaders in this space include Science37, Medable, Thread, and Medidata-each bringing something unique. Innovations include technology platforms that allow trials that operate via a single virtual meta-site, to telemedicine, and even in-home monitoring via BYOD (bring your own device). These companies are not alone-most of the top five CROs are working on virtual trial operations as well. 

5. Integrated research organizations usher in the clinical-research-as-a-care-option age.

 While CRAACO has been a theme over the last few years, ACOs and ICOs are starting to implement frameworks within large healthcare systems that aim to bridge clinical research with clinical care. Javara, out of Winston-Salem, NC, recently announced a partnership with Wake Forest Baptist Medical Center, with the goal of marrying comprehensive clinical research services with population health initiatives. As the industry continues its shift from fee-for-service to outcomes-driven, value-oriented care, this is a logical next step and one to keep an eye on.

 is R&D Practice Lead, Life Science, Capgemini Invent

A Closing Thought

Right now, we only seek “perfect patients” for our trials, rendering many drugs potentially less safe or effective for those in demographics the study was not designed for. 

Five Trends in Clinical Trials We’ll See in 2019

Issue PDF

Click the title above to open the Applied Clinical Trials January/February 2019 issue in an interactive PDF format.

Applied Clinical Trials, January/February 2019 Issue (PDF)

A top priority for FDA leaders in the coming months is to maintain, and continue to improve, the efficient and effective drug development and review system the agency has established in recent years. It will be a challenge for FDA to match or exceed its success in 2018 in approving record numbers of innovative new medicines, but agency leaders aim to try. In January, FDA made it official: The Center for Drug Evaluation and Research (CDER) approved 59 novel drugs and biologics in 2018, surpassing previous high marks and improving on recent gains (see 

,  for complete list of new drugs). Additional new products approved by the Center for Biologics Evaluation and Research (CBER) further boost the total.

	While analysts may define innovative new products differently, the ability of biopharma companies to capitalize on important advances in science, plus strong regulatory support, have combined to move many important new medicines to market efficiently to provide more treatments for a range of critical conditions. The “class of 2018” includes 16 new cancer drugs, with several indicated for genetically defined conditions. There also was a number of new treatments for rare diseases, along with the first cannabidiol-based medicine and the first drug to treat smallpox. Nearly all the new drugs were approved in one review cycle, and three-fourths were approved first in the U.S.

	Many of the new approvals benefited from one or more expedited development or review pathway, a trend that has prompted some observers to question whether FDA may be moving some experimental products too quickly to market. However, only 24% of the new drugs were designated as breakthrough therapies, and fast-track review applied to less than half.

	An important goal for CDER director Janet Woodcock is to complete the overhaul of the new drug review process to better manage a growing volume of applications. She announced at the FDA/CMS Summit in December that the Office of New Drugs (OND) in CDER would be led by OND deputy director Peter Stein, and that long-time CDER guru Bob Temple will become OND senior advisor, positioned to address more controversial and difficult drug development and review issues. Woodcock hopes to finalize the OND reorganization by next summer, but it has been delayed by difficulties in gaining Congressional approval of a new user fee program for improving the regulation of over-the-counter drugs. And the recent government shutdown only delayed legislative action further.

	Modernizing the review process will involve implementing new automation tools for managing drug applications, study data, and review documents under a “multi-disciplinary, issue-based review document” system. CDER also will continue to carry out provisions of the 21st Century Cures Act and reauthorized user fee programs to further advance patient-focused drug development, expanded use of real-world evidence, novel clinical trial design, and added authorities to hire more experts needed to carry out these multiple drug regulatory programs.

	To continue advancing innovation in drug development, FDA Commissioner Scott Gottlieb recently announced plans to establish a new office of drug development science at the agency. The aim is to develop and authorize new tools and methods able to reduce the cost and uncertainty in testing and marketing new medicines. This Office of Drug Evaluation Science will be part of OND and will work to implement new methods for clinical outcome assessments, biomedical informatics, and biomarker development. Additional efforts will aim to devise more standardized tools for capturing and formatting safety data to achieve more consistent review of this information.

	Amid advances in new drug development and review, Woodcock also recognizes the importance of addressing the nation’s deadly opioid epidemic as a top priority for the FDA and pharma companies. The immediate need is to reduce the over 200 million outpatient prescriptions for these drugs each year. Recently enacted legislation instructs FDA to develop evidence-based prescribing guidelines and to explore how manufacturers can provide medications in more secure packaging, Woodcock pointed out. At the same time, FDA will craft guidelines on developing non-opioid drugs for acute and chronic pain to improve treatment for patients.

 is the Washington Correspondent for Applied Clinical Trials 

FDA prioritizes to maintain, and continue to improve, the efficient and effective drug development and review system the agency has established in recent years.



FDA Sets Goals for Improvement in Drug Development and Approval

	Electronic health record (EHR) systems are being used increasingly by clinical investigative sites. This results in a growing expectation, by regulatory agencies, for clinical trial sponsors to verify that EHR systems comply with requirements to protect confidentiality and provide accurate and reliable data for use in a regulated research environment. The FDA recent guidance titled “Use of Electronic Health Record Data in Clinical Investigations,” released July 2018, encourages modernization and efficiencies in clinical investigation by using new technologies and EHRs in the clinical trial setting. The guidance recommends the collaboration between clinical trial sponsors and healthcare organizations toward interoperability and integration of EHRs and electronic data capture (EDC) systems. The legal framework for data protection also covers EHRs; and national legislative provisions or equivalent set of requirements on health data exist, but they are primarily focused on medical care practices and do not always include the use of computerized systems for EHRs. Disparities exist among countries and among investigational sites within countries on the deployment of EHRs for clinical trials. 

	The FDA provided recommendations on use of EHRs as a source of data in clinical trials to “ensure there are appropriate security measures employed to protect the confidentiality and integrity of the study data.”¹ In August 2010, guidance from the EMA went into effect that indicated that the EHR systems in use at clinical trial investigative sites must be assessed 

“to determine how well they meet the requirements of GCP. The assessment should include consideration of the potential harm to trial subjects and patient rights and to the data integrity of the trial. If the systems do not meet the GCP requirements, then mitigating actions should be taken as necessary prior to trial site initiation.”

	Regulatory agencies expect EHR systems in use at sites to comply with GCP requirements and are requesting access to the EHR during inspections. The FDA has indicated, 

“All relevant information in the EHR pertaining to the clinical investigation must be made available to FDA for review upon request.”

¹ If data from EHR systems are transmitted or manually transcribed into EDC (or similar) systems to sponsors, it is clear that regulators expect those data from the EHR to be  within the scope of their guidance.

	Identifying all global requirements for EHR systems can be a daunting task for sites. To add to the complexity, sponsor companies have varied ways of assessing and documenting an EHR system’s readiness for use in clinical research. The eClinical Forum, a non-commercial global organization with over 45 company members active in regulated clinical research processes and technologies, has been involved since 2006 in the establishment of conformance criteria to allow compliant integration of EHR data in clinical development. Their work resulted in approved functional standards by ANSI/HL7

 and their eSource Readiness Assessment (eSRA) is a practical way to apply these functional standards in the clinical setting. eSRA enables sites to evaluate and report on regulatory risk when their EHR systems contain source data. 

	eSRA is a standardized approach, free of charge, for assessing and documenting the compliance of EHR systems with regulations and guidance documents. This allows research sites, EHR system vendors, and sponsors to determine if data from a healthcare system are appropriate for use as source for clinical research.

	eSRA includes questions related to site and system information, handling of records for clinical research, date and time information in the system, access control, data review, backup, retention and recovery, system development, and maintenance. The outcome of a site’s self-assessment could be reused across their clinical trial sponsors. If adopted by sponsors, sites could avoid being assessed by multiple sponsors and would receive uniform questions for the evaluation of their EHR system. EHR vendors can add benefit for their customers by providing documentation of their systems’ compliance with the requirements so that sites would only need to assess their office practices and not the vendor’s system(s) functionality. Additionally, the adoption of eSRA standardized approach enables sponsors to decrease site assessment efforts as the site has the possibility to have already completed the self-assessment for each of their systems. If so, the site can re-use evidence of the eSRA documentation, even if completed initially for a different sponsor or clinical trial. 

	The eClinical Forum’s eSRA was first introduced as a free standard tool in August 2015. As of March 2018, it had been used by over 1,220 sites in 49 countries. 

	Feedback on usage of this first version of eSRA, along with updated regulations and guidance, were discussed at length by the eClinical Forum team resulting in the release of an updated eSRA v2018.1 in March 2018.  The eSRA (eSource Readiness Assessment) Handbook and Template can now be downloaded via 

	Observations and lessons-learned from reviewing the eSRA Version 1 assessments:

		Overall, the majority of assessments indicated sites’ systems had a high degree of compliance (>80%) to the subset of questions a system would need to comply, to be recommended for use as the source of data for clinical research (those that are critical based on assessment of regulations and guidance). This subset of questions is designated with * in the current version of the eSRA.

		Many assessments were incomplete, indicating that a sponsor/CRO might not have reviewed the assessment. After completion of the eSRA, the sponsor would need to work with the site to further assess any responses indicating incomplete compliance with the requirement to determine any actions needed by the site in order to ensure the reliability of the data from the EHR to be used in clinical research. 

		Occasionally, sites indicated a workaround was necessary to comply with the requirement; however, the action appeared inadequate or inappropriate to satisfy the requirement. It was unclear how the site was instructed to mitigate the risk and how it was subsequently documented by the sponsor (e.g., a workaround of “unsure” or “ask vendor” is not appropriate as a viable mitigation for compliance to the statements).

		A completed eSRA cannot be used to exonerate sponsors from doing and documenting their risk assessment for sites’ systems. The sponsor must still review the completed eSRA and assert that the system and associated processes, as indicated in the eSRA, are suitable to originate data for their study.

		To complete an eSRA, it is typically necessary to gather information from both the study coordinator and the IT support staff. Because the eSRA is quite comprehensive, completion of the assessment is not an activity that can be completed in haste. Sufficient time should be provided for site and other healthcare provider support staff and subcontractors to ensure the accuracy and appropriateness of the eSRA responses.

	There is growing scrutiny of EHR systems used in clinical research evidenced by regulators requesting EHR assessment documentation, such as eSRA. Although EHR systems have been in use for a while at sites, that those systems need to comply with requirements to ensure data integrity and protection of confidentiality when the data contained therein are used in clinical research. Use of eSRA could provide reliable, efficient means for assessing systems accessed for risk-based monitoring (RBM) and source data verification (SDV) without imposing on the sites the outdated hurdle of maintaining paper (or other forms of manually transcribed) source records.

	The standardized assessment approach and documentation of eClinical Forum’s eSRA is a viable mechanism for sites, sponsors, and EHR vendors to demonstrate that the EHR system meets requirements for providing trustworthy data for regulated clinical trials. The use of eSRA is strongly recommended for all sponsors who strive to present a harmonized and efficient compliance assessment in mutual recognition of quality standards.

		“Use of Electronic Health Record Data in Clinical Investigations: Guidance for Industry”, U.S. Department of Health and Human Services, Food and Drug Administration (July 2018)	

		“Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials,” European Medicines Agency, GCP Inspectors Working Group (June 9, 2010)	

		HL7 Specifies Functional Requirements for Clinical Research in an EHR-S (Nov. 6, 2009)	

		EHR/CR User Requirements Document (Release 1, Feb. 24, 2008)	

		Practical Considerations for Clinical Trial Sites using Electronic Health Records Certified for Clinical Research, 

Suzanne Bishop (eClinical Forum), Cheryl Tucker McClung (Lilly), Yvonne Rollinger (OmniComm), Yiannis Karageorgos (Bristol-Myers Squibb), Kelley Olree (Pfizer), Ludger Bührmann (GSK), Martijn Griep (Johnson & Johnson)

An increase in use of EHR systems at clinical investigative sites adds more expectations for clinical trial sponsors to comply by regulatory requirements. 

Can Data from EHR Systems Be Trusted in a Trial Setting?

Approaches centered around risk-based monitoring (RBM) have become widely accepted in the clinical trials industry. Key milestones include the amendment to ICH-E6(R2) guidelines on Nov. 9, 2016

 and the adoption of the same by regulatory agencies in various regions-Europe (European Medicines Agency) and Canada (Health Canada).

 Other countries, including Japan and Australia, are expected to follow suit. The FDA has also indicated its encouragement of the new model through its guidance document issued in March 2018.

	In general, large organizations-sponsors and contract research organizations (CROs)-have begun implementing risk-based approaches to monitoring. It is fathomable that since large companies command a greater share of drug development and outsourcing of clinical studies, the necessity to initiate and adopt new strategies and innovative models is a strategic imperative for them.

	In contrast, due to limited financial and operational resources, smaller organizations face multiple challenges when adapting to rapid industry changes. Process modification, reengineering SOPs, and operational change management can be daunting. Moreover, the development of new technologies requires upfront capital expenditure, which is risky when the model is evolving. All of these factors contribute to the slower adoption of RBM in smaller organizations.

	However, the need to comply with regulatory requirements leaves little room for companies to delay implementation of ICH E6(R2). Hence, organizational readiness is important when such a time comes into effect. This case study seeks to share the experience of a sponsor organization that decided to utilize RBM in a limited, but effective manner.

	A global mid-sized biopharmaceutical sponsor developed a promising anti-infective drug into a Phase II trial. In 2016-2017, the sponsor conducted a global study, spanning 40 sites across seven countries and several continents, including North America, Europe, and Africa. The sponsor team believed in RBM and the potential benefits of that approach, but was not in a position to fully implement RBM for a variety of reasons, including a reliance on the processes and procedures established by the CRO. Since RBM was in its nascent stages at that time, and the industry had not fully understood the impact, the sponsor utilized RBM in a limited manner, albeit with clear expectations and expected benefits.

	Given the importance of the clinical study and its potential impact on business strategy and growth, the sponsor’s primary focus was to ensure that the trial conformed to the highest standards. Hence, monitoring the overall study from a risk and quality perspective was the key objective of adopting RBM.

	Given the limited scope of the exercise, a simple yet objective risk-planning process was performed by the clinical operations and medical monitor from the sponsor team in consultation with a subject matter expert (SME).

	For the global mid-sized sponsor, risks that could significantly affect the outcome of the study conduct were considered. Key performance indicators (KPIs) and key risk indicators (KRIs) were developed to objectively monitor the data in an efficient manner on a regular basis. The RBM SME facilitated and advised the risk-identification process, and developed the risk-assessment and monitoring framework.

Acuity Analytics-an integrated data analytics and RBM enablement platform-was utilized for developing data visualization and custom analytics for monitoring the KPIs, KRIs and specific critical data. The custom dashboards and analytics were designed in alignment with the sponsor’s specific needs. Data sources included an electronic data capture (EDC) system and the CRO’s clinical trial management system (CTMS). These were integrated with the Acuity Analytics platform. Data from the EDC and CTMS were refreshed every 24 hours and once a week, respectively.

	During the start-up phase of the study, the team selected three key performance indicators to monitor the progress of study setup at the sites. These included:

		Site selection to site initiation visits at different stages

		Essential site document finalization and approval

		Site contract finalization and execution

	Validated clinical operations dashboards from the platform’s data visualization library were used to understand the delay points at various process operational touch points with sites during study start-up. Delays (at the site, CRO, or sponsor level) were monitored so that proactive and timely intervention could be taken. The sponsor’s operations team and central monitor performed periodic reviews. Outputs were shared with different stakeholders, and timely action was taken on a regular basis.

	Figures 1-3 (see above) depict a few of the operational dashboards utilized.

	From the risk-planning process, six KRIs were developed and used during the active phase of the study. Validated dashboards from the Acuity Clinical Insights library, along with analytics related to efficacy endpoints were aligned to the sponsor’s needs. The following are the KRIs and critical data points that were developed:

		Missing clinically acquired bacterial pneumonia (CABP) data points

		Potentially ineligible subjects utilizing surrogate data points

		Pharmacokinetic (PK) sampling deviations

	Figures 4-6 (see below) depict a few of the dashboards utilized for the central monitor’s review.

	A central monitoring plan was created during the initial phase of the study. The central monitor reviewed the KRI and critical data dashboards at frequent intervals (every four weeks) as per the central monitoring plan. The outputs were shared with the sponsor on a regular and timely manner. For certain types of complex data review that are prone to human error, custom algorithms were utilized to automate initial identification of issues. This was specifically applied for detecting potential eligibility violations and non-congruence of CABP symptoms in subjects as per the study protocol. In the event of specific issues, trends, or patterns, more frequent reviews were performed. The outcome of the review was also recorded in a data-driven approach as per the criteria below.

	Subsequently, the recorded outcomes from central monitoring were fed into a site-risk assessment model designed for Acuity’s Decision Factory. The model included the risk thresholds and risk categorization identified in Table 1 (see below).

	Additionally, the number of subjects enrolled at the site was factored into the model. The assumption is that more subjects at a site are directly proportional to increased operational and scientific workload for the site study teams and, therefore, higher error rates. If such a scenario occurs, the impact on the safety of subjects at the site and scientific outcomes can jeopardize the overall study outcome. Needless to mention, additional operational and financial resources are required to bolster quality control and monitoring efforts. Hence, it was deemed fit to factor this into the risk-assessment model.

	Since this was a short study, the RBM approach to monitoring was conducted over a period of 11 months. The start-up phase was monitored for four months, and the active recruitment and follow-up phases spanned more than six months. The study was successfully executed as per the originally planned timeframe.

	The three KPIs were reviewed using relevant data visualizations. Figures 1 and 2 illustrate some of the data visualizations that were utilized.

	Continuous monitoring of study start-up delays were performed by the sponsor’s operations team that enabled it to review process delays at the site and enhance governance of the CRO that managed the conduct of activities at the sites.

	The six KPIs were reviewed using relevant data visualizations. Figures 3 to 5 illustrate some of the data visualizations that were used for review by central monitors.

	After the review by central monitors against the central monitoring plan, the review outcomes were recorded in the site-risk assessment framework and factored in the subject load at the site, as depicted in Figure 6 (see below). The time-to-date, site-risk assessment was performed over the duration of central monitoring by comparing from the baseline to understand progress over a period. The team used a normal data distribution test to segregate sites that had significant increase and decrease of risks from baseline. Key outcomes of the analysis were:

		Initially, 27% of sites (eight sites) were identified as high, 45% (15 sites) as medium, and 28% (nine sites) with low-risk propensity. At the end of implementation, there was an overall reduction of risk propensity at sites. About 63% of the sites reduced in risk propensity from baseline.

		Two sites (6.25%)-categorized as high risk at baseline-continued to remain high risk until the last central monitor review cycle.

		Five sites (15.65%) from the high-risk category were downgraded to medium or low risk.

		The KRIs that contributed to significant change from baseline were:					

		Outputs from risk assessment were utilized for assigning resources for co-monitoring visits and quality monitoring visits. The sites were selected on a more objective basis than on purely subject recruitment reasons or issue escalations.

	This study had limited scope to implement RBM. However, this exercise proved to be a precursor for future clinical studies, and the observed significant benefits. The operational benefits included:

		Data visualization and custom analytics helped in central data monitoring.

		Data visualizations afforded an intuitive and significant efficiency in data review. The data review process time frame was reduced by about 45% from the first data review process.

		The site-risk assessment approach was objective and driven by data. The congruence of the model to ground realities observed by site monitors was validated by seeking inputs from the sponsor’s clinical operations team.

		The approach gave impetus to formally adopt and conduct risk-based approaches to monitoring in future studies in a scalable manner.

	Stakeholders from this exercise also learned certain lessons. To a great extent, this could not have been incorporated, as the expectation was to keep it light from an execution perspective.

		Laboratory data transfers into EDC need to be more frequent to enable better and close to real-time central data review of subject data.

		Ensuring a formalized and more comprehensive risk-planning approach so that other cross-functional inputs, as well as risks, can be captured.

Feature Articles

Case study demonstrates how one global mid-sized sponsor is testing the RBM waters.

Weighing the Benefits of RBM Implementation

The clinical trials sector has done a good job tracking costs associated with study execution, but costs linked to study start-up are conspicuously absent. A quick look at industry research confirms this gaping hole in cost accountability. For Phase III clinical trials, for example, it is widely recognized that total costs are substantial, ranging from an estimated $11.5 million to $52.9 million, depending on therapeutic indication.¹ Upon closer examination, it is apparent that identified costs are almost exclusively tied to study execution, and rarely include details on the hefty costs incurred early on, namely those tied to study start-up and site overhead. In fact, data from an often-quoted 2014 study published by the Department of Health and Human Services (HHS) indicate that unaccounted for costs may represent upwards of 43% of the total cost of conducting a clinical trial.

	This statistic is calculated from the HHS study, which focused on numerous cost drivers in each phase of clinical trials and the percentage they represent of total costs.² For Phase III studies, among the many known costs, the top three were:

		Administrative staff (11.7% of total costs)

		Clinical procedure costs, such as electronic data capture (11.3%)

		Site monitoring costs, such as risk-based monitoring (8.2%)

	Collectively, these top-known costs, plus all of the other known factors, rang it at $11.38 million, yet the average of all Phase III costs totaled $19.89 million. It is concerning that the large $8.51 million difference between these two figures lacks detail and remains unaccounted for.

	This lack of accountability is indicative of how the industry in general underestimates the activities associated with the initiation of clinical trials. The Trial Master File (TMF) Reference Model,³ a DIA document and records management community project, corroborates the significance of study start-up activities. It categorizes 53% of artifacts that ultimately end up in the TMF as being associated with study start-up, including ongoing maintenance activities (e.g., protocol and contract amendments, principal investigator [PI] changes and related submissions, etc.).

	It can be hypothesized that the undefined study cost elements in the HHS study are caused by an industry largely tracking significant elements (i.e., operational and regulatory) of a study in paper and unsophisticated spreadsheets and not dedicated systems/databases. In fact, the tracking of regulatory elements truly begins at investigational product release, at which point all systems/databases are traditionally implemented, hence there is cost-associated data for these elements. This article explores the factors that make up unaccounted expenses, especially those tied to the many complex steps that define study start-up, such as protocol writing, site identification, selection, and activation, budgets and contracts, institutional review board (IRB) and ethical reviews, and more. As part of this effort, the following issues are considered:

		How a workflow-based system helps gather quality data and metadata from multiple sources to help track costs and improve study start-up operations

		Why it is critical to consider using a parallel approach to the multi-step study start-up process to cut timelines and costs

	By embracing a systematic, data-driven approach, it is possible for metrics to identify more accurately the best sites, steps causing delays, the associated costs, and why this is happening.

	An effective study start-up process is fundamental to the overall operational success of a clinical trial, as it is the most impactful phase of the trial’s lifecycle. The impact comes from a thoughtful upstream study start-up approach that influences downstream performance through the use of purpose-built workflows. Essentially, workflows streamline and organize operational aspects in real-time, enable the collection of data and metadata, help identify problems quickly, and allow detailed study start-up costs to be calculated.

	This represents a sea change, as the HHS study shows study start-up costs as vague and incomplete, basically reflecting that, traditionally, these costs have not been clearly identified. Instead, they have been overlooked or lumped into a handful of categories, such as the cost of IRB approvals, site recruitment and retention costs, and patient recruitment costs. As a result, critical multi-step tasks ranging from pre-study visits through to contract or budget execution have relatively unknown costs, although they embody almost 60% of study start-up cycle time.

	In the DHHS study, the largest category is “Site Overhead,” which lacks detail entirely, but is still estimated as responsible for nearly $2.6 million of the $19.89 million in Phase III study costs. Similarly, the “All Other Costs” category is also undefined yet is associated with almost $6 million in Phase III expenditures. Moreover, the report states that these two large expenses were determined by extrapolation from cost components with available estimates-not task-specific data (see Table 1 below).

	A good example of how workflow technology can bring greater precision to study start-up costs includes taking a look at how it can impact tightly guarded siloes. Typically, stakeholders rarely interact with others along the clinical trial continuum, so they are often unaware of which delays are throwing the timeline off course or what the associated costs might be. With workflows, however, pinpointing tasks need to be completed and when, and by whom, can be more easily achieved, improving the likelihood of the study’s timeline staying on track.

	This approach is a major departure from the still-too-common department-to-department handshakes and continued reliance on departmental metrics meant to evaluate performance while generally lacking the ability to measure costs and activity at the granular level. For instance, attempting to measure “time to execute all site-level contracts” is not an accurate metric unless it is based on its sub-steps, all of which play a role. These include:

		When first contract language package is sent

		When last contract language is ready for execution

		Completion of regulatory submissions and approvals

	There are additional benefits to a workflow approach. It optimizes the handling of amendments, which determines their subsequent impact on study start-up functions, such as site activation and study execution. Research from the Tufts Center for the Study of Drug Development (CSDD) indicates why the limiting of amendments and their proper management are vital actions. More than half of all protocols, 57%, across all phases, have at least one substantial amendment, which results in a study taking three months longer to complete, on average, as compared to studies without amendments.

 Moreover, approximately 40% of all amendments occur before the first study volunteer receives the first dose in the clinical trial.

	It is noteworthy that workflow solutions align with the 2016 Good Clinical Practice (GCP) guideline from the
	International Conference on Harmonization [ICH-GCP E(6)R2)], which calls for a heavy emphasis on quality from the beginning of a study, and extending it throughout the life cycle.

 Specifically, the guideline has an entire

 section dedicated to quality management, which states that to achieve quality, efficient tools are essential for proper data collection and decision-making (see box on right).

	This regulatory input and the impact of workflow-based tools are driving significant improvements to how study progress is often tracked, namely Excel spreadsheets, Word documents, and PDFs that generally are department-specific and lack access to upstream and downstream information. As a result, reliable metrics are usually difficult to calculate, and stakeholders cannot easily track costs or study status, or make improvements.

	Taking the steps to improve data collection from study start-up activities means figuring the best way to maximize the impact of workflows. Certainly, implementing workflows drives efficiencies by automating manual processes, but significant reductions in timelines and cost require more-they require changing the underlying processes. For starters, this may involve switching to working in parallel, instead of continuing with the traditional sequential method of study conduct. In particular, working to evaluate sites, then selecting and activating them, are activities that can take place in parallel.

	The parallel approach has become somewhat of a battle cry in recent years, as industries from computer science to manufacturing to chemical synthesis have adopted it to shorten timelines. A simple Google search reveals a cadre of articles describing use of parallel technique dating back 20 years, but for the life sciences sector, it is a fairly new endeavor. A 2004 article in 

 comments on the then-growing use of electronic data capture (EDC), which would change the trial process from a series of sequential tasks to a parallel process, enabling continual review of data to improve data quality.

 Other articles describe conducting Phase I and Phase II trials in parallel to evaluate the safety and efficacy of combination dose levels simultaneously, to more quickly select the optimal combination dose.

	What role does a parallel technique play in creating a better study start-up experience? A system that will help identify high-quality sites that are most likely to meet enrollment targets and ensure adherence to GCP is one area. Research suggests that for site identification, there is no single source of data used to find the right sites, and the approaches are not evidence-based. Consequently, it is hardly surprising that 11% of selected sites are never activated, a statistic that has not changed meaningfully in more than a decade.

	With the help of data analytics, sponsors and contract research organizations (CROs) can do more than identify the best sites. They can activate them quickly, allowing them to start patient recruitment activities, while continuing to identify sites in parallel.

The need for parallel processes is evident. A survey conducted in 2017 by the Tufts CSDD, with 591 respondents, found that average cycle time for study start-up is quite lengthy, averaging more than 36 weeks (~ 8.4 months) for new sites and 26 weeks (~6 months) for existing ones (see Figure 1).

 This timeframe poses challenges. In particular, with a sequential approach, the “Sites Selected” list is typically finalized months before any sites are activated. And for sites that have identified patients as part of the feasibility process, expecting those patients to wait months to enroll is a poor management strategy.

	A parallel approach allows those sites that have already completed a confidentiality disclosure agreement, and are ready to recruit and enroll, to actually do so. Using a system to collect and analyze study start-up data, this approach can enable site feasibility and activation to take place in parallel. This also can help ensure that patients can be found and enrolled much closer to release of the investigational product to the site (see Figure 2).

	The clinical trials sector has invested heavily in recent years in cloud-based tools for study execution, such as EDC, clinical trial management system (CTMS), risk-based monitoring initiatives, interactive response technology, and more. As a result, there are more opportunities for parallel operations associated with these established solutions, given the existence of metrics-related data and good cost information, unlike the vague cost landscape that has typified study start-up.

	Part of the challenge in improving operations stems from the fact that until recently, there have been no systems to enter study start-up performance data, so it has been quite difficult for companies to perform a data-driven analysis to identify high-quality sites most likely to meet enrollment targets. Study start-up workflow systems with open application program interfaces (APIs) can exchange data from tools such as EDC, CTMS, and others, allowing data to flow among them. This enables stakeholders, including sites, to enter data that can eventually be used for performance analysis. With the help of these systems, a sponsor or CRO can identify sites, and continue identifying them until analytics indicate with 90%–95% probability that they will meet the enrollment target. As discussed, those sites that are ready to activate can do so. This creates an environment in which sites can compete to be selected and compete to reach enrollment targets.

	Figure 3 depicts how this end-to-end workflow system works, using a parallel model.

	This analytical approach is more exacting than defaulting to arbitrary milestones based on sequential methods that not only lack the data to find the best sites, but also cannot indicate when sponsors or CROs should stop identifying sites. There are available databases of site information, but those data do not necessarily lead to use of parallel process. A better approach for stakeholders looking to start a study is to review the protocol, and then use a parallel-based systemic approach to generate data and metadata specific to that study.

	With nearly half of clinical trial costs unaccounted for, including known administrative costs, and much of those linked to study start-up, it is telling that the industry is finally moving toward identifying those costs with the help of purpose-built tools that can determine, with a high degree of accuracy, when it is time to stop identifying sites. The vast wealth of data that reside in technologies designed for study execution, such as EDC, CTMS, and more, can be integrated with workflow-driven solutions for study start-up, allowing stakeholders to gain insight at a granular level into how a study process is unfolding and the associated costs.

	The parallel method, using data and applying data analytics, could possibly shave up to two months from the timeline. This change in approach will continue to highlight how improving the study start-up process is a great opportunity, yielding a major impact on quality, timeline, and overall cost of clinical trials.

		Sertkaya A, Wong H-H, Jessup A, Beleche T. Key cost drivers of pharmaceutical clinical trials in the United States. 

http://journals.sagepub.com/doi/pdf/10.1177/1740774515625964

		Sertkaya A, Birkenbach A, Berlind A, et al. Examination of clinical trial costs and barriers for drug development. US Department of Health & Human Services. July 2014. 

https://aspe.hhs.gov/report/examination-clinical-trial-costs-and-barriers-drug-development

		Lamberti MJ, Brothers C, Manak D, Getz KA. Benchmarking the study initiation process. 

Therapeutic Innovation & Regulatory Science

		The gateway to study startup (SSU) optimization: Performance metrics. goBalto. May 2017. 

https://www.gobalto.com/whitepaper_study_startup_optimization_performance_metrics

		Peters S, Lowy P. Protocol amendments improve elements of clinical trial feasibility, but at high economic and cycle time cost. Press Release. Tufts Center for the Study of Drug Development. January 14, 2016. 

Using data analytics to introduce parallel processes to optimize and accelerate clinical trials.

Applied Clinical Trials-01-01-2019