Beth Harper 

The results of several studies published in 

 and elsewhere have demonstrated the impact of growing clinical trial data volume and the increasing diversity of clinical data sources on study timelines and inefficiencies.

New benchmarks show that the increase in data volume and diversity is accelerating. The typical late-stage protocol, for example, now collects on average 3.6 million data points, three times the number collected 10 years ago.

In collaboration with and funding from IBM Watson Health, Tufts CSDD has now completed a study assessing how the clinical data management function—internal teams in collaboration with their electronic data capture (EDC) partners—are managing, collecting, organizing, storing and analyzing their clinical data. This article summarizes the results of this collaborative study and provides more granular cycle time measures, experience with EDC solutions, insights into challenges and opportunities to optimize clinical data management practice.

A global survey was conducted between September and October 2020.In total, 194 verified unique individual responses were received and analyzed. Respondents had an average of 10.8 years of experience working in a variety of roles including clinical operations, data management, executive leadership, clinical development and others. The mean number of clinical trials performed per year was 54. In the discussion that follows, we’ve classified organizations into two primary subgroups: those conducting 50+ trials/year are designated as “Larger” and those conducting fewer than 50 trials/year are designated as “Smaller.”

Clinical data management and EDC solution pain points

More than twelve different EDC solutions were identified by respondents. Most organizations (73.9%) report using two or more and the remaining 26.1% of respondents report using only one EDC solution.

Table 1 below shows the wide range of clinical data management pain points reported. The typical respondent noted 3.3 different pain points associated with their primary EDC solution. Managing mid-study updates and their associated delays topped the list followed by flexibility and customization challenges.

 Pain Points Experienced with Primary EDC Solution

Nearly one third of respondents reported challenges with database go live delays and the lack of integrated patient engagement and electronic clinical outcome assessment (eCOA) applications. Slightly more than 30% reported challenges with the cost of their primary EDC solution. Pain points that were reported the least frequently included data incompatibility between platforms, customer support problems and solution complexity (slow user learning curve).

Although a relatively high percentage of respondents cited database go live delays as a key challenge, it is interesting to note that the cycle time between final protocol approval and the database go live milestone has not changed during the past three years.

 Figure 1 below shows that the average duration from final protocol approval to the database go live milestone has increased by one day.

The results of this study also show that the average study close-out cycle time—from last patient last visit to database lock—remains unchanged between 2017 and 2020 with organizations reporting an average duration of 36.8 days. Mid-study updates represent a time intensive challenge.Managing a mid-study update requires about half of the total time needed to achieve the initial go live release of the database, as shown in Figure 2 below. This helps explain why planned and unplanned study updates are the top reported pain points.

Granular Data Management Cycle Times Between Small and Large Companies

Looking at each of these cycle times individually, and comparing smaller vs. larger organizations, there were some notable differences. The results show that smaller organizations (<50 trials/year) are faster at early-stage data management processes whereas larger organizations (≥ 50 trials/year) are faster in late-stage processes.

Despite the challenges noted, respondents report a high level of satisfaction with their EDC solutions, as shown in Table 2 below. Database design and data collection tasks received the highest satisfaction levels with over 90% satisfied. Database closeout, the overall end-to-end process, and data processing and cleaning abilities also received high levels of satisfaction. Managing mid-study updates and the overall study cycle time received the lowest relative satisfaction scores.

Satisfaction with EDC Solution Performance Across Data Management Tasks

As noted in Table 3 below, EDC solutions received high marks with 84.1% and 82.3% of respondents satisfied with the solutions’ ability to reduce the number of manual errors and avoid costs in managing mid-study updates. Fewer respondents (75.8%) were satisfied with the ability of their EDC solutions to help them reduce delays in managing mid-study updates.

Satisfaction with EDC Solution's Management of Mid-Study Update Challenges

EDC solutions also received high satisfaction ratings for electronic data capture, randomization and endpoint adjudication (100% of respondents reported being satisfied). Over 90% were satisfied with medical coding, data integration, data management and eConsent capabilities. Electronic patient-reported outcomes (ePRO), eCOA and Study-Level Reporting received the lowest satisfaction ratings. Nonetheless, over 70% report being satisfied in these areas as detailed in Table 4 below.

Satisfaction with EDC Solution Functionality

Table 5 below shows how respondents ranked aspects of their EDC solutions and solution providers that they like least. Lack of responsiveness and poor individualized attention were among the top criticisms of EDC providers.

Least Liked Aspects of EDC Solutions and Solution Providers

Functionality was by far the top factor that influences respondents’ decision when selecting an EDC solution as shown by Table 6 below. A distant second was Ease of Use with 57.4% of respondents including this factor among their top three. The ability of a solution to handle clinical trial scope and complexity was also noted as a top factor influencing selection decisions. Customer support, pricing and reputation were among the least influential factors.

 Influence of Factors When Selecting an EDC Solution

Looking ahead – will EDC solutions and providers maintain their high marks?

Organizations look for their EDC solutions to support a wide range of data management tasks and provide a variety of functionality. This study found that—although cycle times are not improving—the majority of organizations are satisfied with their providers and EDC solutions on all accounts.

Cycle times from final protocol approval to database go-live and last patient visit to database lock—now average 69.4 and 36.1 days respectively—represent opportunity areas to accelerate. Similarly, the time required to implement mid-study updates (28.5 days for planned and 29.9 days for unplanned) against a backdrop of frequent amendments and an increasing number of adaptive design studies also represent a major area of opportunity requiring attention and improvement.

One issue highlighted by the contradictory relationship raised between challenges and satisfaction rates is the close dependency that data management has on other study tasks. Evaluating EDC solutions for efficiency or time savings remains impractical due to the many external confounders—such as the protocol approval process and amendment resolution—that impact data management cycle times.

How EDC solutions and providers can add or integrate new functionality, manage mid-study updates and reduce overall cycle times remains to be seen. This study serves as a baseline from which to measure future progress. We look forward to monitoring the progress in this highly important aspect of drug development.

The authors would like to thank Dilhan Weeraratne and Van Willis at IBM for their contributions to the project and their support in developing this manuscript.

Indupuri R, Rocchio S, Getz K, Harper B, Wilkinson, M. Enabling Digital Transformation: Managing External Clinical Data Sources to Advance Drug Development. 

https://www.appliedclinicaltrialsonline.com/view/enabling-digital-transformation-managing-external-clinical-data-sources-to-advance-drug-development

.Published November 13, 2020. Accessed January 4, 2021.

Society for Clinical Data Management White Paper. 

The Evolution of Clinical Data Management to Clinical Data Science (Part 2: The technology enablers).

. Published March 5, 2020. Accessed December 20, 2020.

Rising protocol design complexity is driving rapid growth in clinical trial data volume.

 Volume 23, Number 1. January/February 2021.

Wilkinson M, Young R, Harper B, Machion G, Getz K. Baseline assessment of the evolving 2019 eClinical Landscape. 

Therapeutic Innovation & Regulatory Science.

Articles

2020 Tufts CSDD – IBM Watson Health benchmarking study highlights need for new functionality from EDC solutions and providers.

Characterizing Clinical Data Management Challenges and Their Impact

Data has truly become the currency in our everyday lives, and the volume and speed at which it is being collected, processed and exchanged is staggering. For instance, every minute, 188 million emails are sent, 4.49 million Google searches are conducted, and 4.5 million videos are watched on YouTube.

 This proliferation of information can also be seen in the world of clinical research, where the amount of clinical data per trial has increased by 183% in the last decade.

The growing volume of clinical data illustrates one significant way in which the drug development landscape is evolving. As clinical trials are becoming more decentralized, the variety of data types and sources, including genomics, biomarkers, wearables and images, that are being used in drug development is also expanding. With the amount of diverse data that must now be collected and analyzed, the life sciences industry is faced with increasingly complex studies that present new challenges in clinical data management.

From October 4 to December 30, 2019, eClinical Solutions and the Tufts Center for the Study of Drug Development (Tufts CSDD) conducted a survey to understand the trends in clinical data management and assess how sponsors are responding to current challenges. 149 individuals in senior-level data management, data sciences, biostatistics, information technology and clinical research operations roles participated in the study. Of the respondents, 48% were one of several individuals within an organization responsible for clinical data, 33% were primarily responsible for clinical data and 19% were frequent users of clinical data systems. 

The majority of respondents came from pharmaceutical (41%) and biopharmaceutical (37%) sponsors, though CROs (10%) and independent contracts (6%) were also represented. The sizes of respondents’ organizations varied from small sponsors (30%) that had initiated an average of 2.9 trials yearly, medium sponsors (29%) that had initiated an average of 23.8 trials yearly and large sponsors (17%) that had initiated an average of 112.4 trials yearly.

Sponsors are collecting more data types than ever. Though non-CRF data is the most commonly used in clinical trials among 87% of respondents, a variety of other data types and sources are also being used. Nearly two thirds of sponsors are incorporating direct data capture, apps, devices and medical images as other data sources, while roughly one-third are using electronic health records (43.4%) and omics data (35.5%).

Data sources contributing to significant delays

The growing number of external data sources is associated with longer database lock cycle times with more than half of the study respondents citing non-CRF data as a primary contributing factor. Since 2017, this has resulted in a 40% increase in time from Last Patient Last Visit (LPLV) to database lock. Specifically, large pharmaceutical companies using four or more sources are experiencing a three-week increase in Last Patient Last Visit (LPLV) to database lock.

Last Patient Last Visit (LPLV) to Database Lock Cycle Times

Labor intensive data management activities

Although the availability of richer data sets is essential for gaining deeper insights to advance drug development, sponsors are faced with challenges in all data management activities related to organizing and reviewing the immense volume of data. Among all participants in the Tufts-eClinical Solutions study, analyzing and consuming data was rated as the least challenging of these activities, though over fifty percent of respondents still noted this activity as somewhat or extremely difficult. Of the seven key data management tasks noted in the survey, participants highlighted the process of initiating relationships with data providers as the most time-consuming and labor intensive. This highlights the opportunity for improving vendor relationships as a crucial element in alleviating difficult data management duties.

Data management activity labor intensiveness

Unsophisticated tools to centralize and standardize data

Even though the variety and volume of data continues to evolve, making data management activities more difficult and time consuming, the tools used for data integration and analytics remain relatively unchanged. 75% of companies are still using SAS as the primary tool to convert, explore and analyze data.

With the changing landscape of clinical research presenting challenges in how companies organize and review disparate data types from a multitude of sources, all participants in the Tufts-eClinical Solutions study recognized the need to find solutions. Potential areas for improvement that were highlighted included handling more external data sources, decreasing cycle times through streamlined processes, encouraging patient-centered data acquisition strategies and developing better opportunities for real-world data uses. Life sciences organizations identified several ways in which they are responding to evolving clinical data management requirements in these specific areas.

When an organization outlines a formal data strategy to determine how data will be collected, processed and analyzed, there are clear benefits to clinical research. The greatest impact is to database lock times. Database lock is accomplished 10 days faster with sponsors that have implemented a data strategy compared to those that have not. Since large companies are managing more data sources, they are also more likely to have implemented a data strategy, especially given that 53% of organizations that are using five to six data sources experience longer times to database lock.

Though there are significant benefits, only one-third of the survey respondents have implemented a defined data strategy. Almost half of the respondents were either still examining or planning a data management initiative in the next few months. This means that there is still a great opportunity for all life sciences companies to leverage the benefits of a formal data strategy in reducing trial delays.

To keep pace with the growing number of disparate data, organizations are turning to more advanced tools. The survey findings show that clinical data hubs or repositories are being used more often than other systems, including EDC and SAS-based infrastructure. However, EDC is still being used most often for specifically integrating and organizing direct data capture. Tools like clinical data platforms with robust capabilities to help data managers import, transform and review data will become powerful assets as data sources continue to increase.

Sponsors using a data hub or lake also had more confidence in their analytics capabilities in areas associated with analytics dashboards, interactive visual exploration, the ability to publish and collaborate on analytic content, and augmented data discovery.

Integrating new technology into current systems is just one element in establishing mature analytical capabilities. The survey also revealed a strong association between these capabilities and the implementation of data strategies. Over 50% of respondents who have a defined data strategy noted that their analytical capabilities are more developed.

Another benefit to having definitive data strategies can be seen in the areas of artificial intelligence (AI) and data science competencies. Organizations that have implemented a data strategy are more mature in terms of researching or implementing AI, machine learning and deep learning capabilities. These companies are also more likely to be developing new data sciences functions or specific data sciences roles. In fact, 47.2% that have implemented a data strategy have also established a data sciences function.

Developing analytical and AI capabilities is crucial to preparing for the digital transformation that is sweeping across the drug development industry. Life sciences companies must now meet the increased demand for clinical data and real-time insights from a variety of distinct sources. By automating time-consuming and labor-intensive data tasks, sponsors are able to support greater access to and deeper insights into data to drive innovation and accelerate drug submissions and discoveries.

Respondents to the Tufts-eClinical solutions survey agreed that there are tangible benefits to laying the foundation for AI and data science through formal data strategies. 85% of all respondents rated these qualities as somewhat or very beneficial:

Increased visibility of data to stakeholders for faster decision making

Decreased cycle times through more collaboration and shared analytics

Cost-savings through resource optimization and reduced rework

Faster acquisition and benefits from new real-world data sources that contribute to development


Conclusion: a new era of clinical research

In this modern era of drug development, clinical data has become one of the most valuable assets for life sciences organizations in powering digital transformation. As the Tufts-eClinical Solutions study demonstrates, delays in clinical trials, measured through database lock cycle times and through manual efforts and time-consuming processes associated with assembling data sets for review and insights will continue to increase without the implementation of new strategies. The diversity and volume of clinical data is also growing exponentially, and the tools for managing these must improve to keep pace.

Industry leaders are recognizing that evolving their approach to the research process is critical. By adopting formal data strategies, implementing modern technologies like clinical data platforms and expanding data science competencies, analytics and AI capabilities, sponsors can gain more control over the research process and drive actionable insights from new data streams to continue to push and accelerate drug development.

Anticipating the Impact of the Patient Engagement Movement on Clinical Operations

, Presentation at CROWN conference, Slide #9, January 202

With an increasing amount of diverse data that must now be collected and analyzed, the industry is faced with increasingly complex studies that present new challenges in data management.

 Enabling Digital Transformation: Managing External Clinical Data Sources to Advance Drug Development

In the battle for the war on talent, biopharmaceutical companies, particularly those in competitive markets, are having to find creative strategies to enhance their staff recruitment, engagement, and retention practices.1,2 In the 2018 Employee Tenure Survey conducted by the U.S. Bureau of Labor and Statistics, it was reported that the median tenure for wage and salary workers was 4.2 years and this has not changed in a decade.3 The tenure is even lower in the healthcare industry, clocking in at 3.5 years for the category of “health services, except hospitals.”

While these trends may not directly correlate to roles in clinical research, the numbers highlight the challenges that biopharma managers and organizations face when it comes to developing and retaining talent. Several key questions arise:

Should companies play an active role in investing in their employees’ growth and development?

What practical approaches should employees take to ensure they have a targeted plan for growing the knowledge and skills most needed to be successful in their role?

What is the best way to communicate the competency expectations of employees and set them up for success, so they perform and grow in their roles and stay with the organization?

In a large organization with multiple managers, how can consistency in the interpretation of these expected competencies be ensured?

These were but a few of the questions that the clinical operations group at Genentech has tackled over the last few years. In this article, we provide a case study for how we went about addressing these workforce challenges, where we are currently, and what we learned along the way.

Genentech, a member of the Roche Group, is headquartered in South San Francisco, a hotbed for high-tech and home to many biopharmaceutical companies. Roche, like many big pharma organizations, functions around a set of core competencies that outlines the cultural and organizational behaviors that are expected of all employees, including teamwork, collaboration, decision-making, and innovation.

These competencies include both technical (e.g., achieving results) and leadership (e.g., communication) competencies. They provide expectations for the skills, behaviors, and knowledge (SBKs; sometimes also referred to as knowledge, skills, and abilities or KSAs) that employees should demonstrate and master to fulfill Roche’s overall corporate objectives.

As part of a larger initiative focused on developing, retaining, and shaping our study management leaders to build the culture of tomorrow, Genentech’s early research and development (gRED ECD) clinical operations group wanted to address long-standing feedback from employees and managers on the lack of a consistent framework for mapping the Roche core competencies. Additionally, the group struggled with the lack of a standardized way of defining these competencies across all jobs and job levels.

To address these issues, gRED partnered with the Association of Clinical Research Professionals (ACRP) and embarked on a journey in 2018 to ideate and develop a solution.

The “Competency Framework Initiative” (herein referred to as “the initiative”) was started with some of these immediate needs in mind along with a recognition of the larger challenge of securing, growing, and retaining talent in a highly competitive market.

Specifically, the goals of the initiative were to:

Provide a standardized and consistent way of tying the Roche Core Competencies to key job responsibilities across all job levels within study management

Develop a multi-purpose guidance tool for all managers and employees for:

Clinical operations managers and their direct reports to take a more targeted approach in identifying areas of strength, development, and growth (goal setting).

Clinical operations employees and managers to have a systematic and standardized way to evaluate the competencies at their existing job level and assess the talent pipeline.

Provide learning “playlists” that match the descriptions defined in the core competencies to aid in easy identification of learning resources.

Update job descriptions to align with the SBKs outlined in the framework.

More broadly speaking, the competency framework and associated training and competency-based job descriptions were expected to be a powerful tool to empower and motivate employees to take charge of their development and use the framework to refine their development plans. In essence, the initiative was designed to help Roche employees identify areas of focus and to generate clarity on the competencies in each role.

The study management function within gRED consists of four different job roles ranging from an associate clinical trial leader (CTL) to a senior and principal CTL.

Specifically, these roles span a broad range of clinical trial leadership and technical responsibilities. The initiative was a multi-step approach that involved the following (also see Figure 1 below):

 In collaboration with ACRP, a small group of managers (herein referred to as “core team”) reviewed the competency domains and developed definitions for theskills, behaviors, and knowledge (herein referred to as SBK) requirements for each of the competency domains outlined in the Roche Core Competencies for each of the three job roles in study management. Due to the monumental nature of the task and ACRP’s experience in developing role-specific competency guidelines4, gRED contracted with ACRP to provide consultation and support throughout the project.

 For thoroughness and robustness of the framework, the core team compared the technical and leadership competencies against the harmonized core competency framework developed by the Joint Task Force for Clinical Trial Competency (JTF)5, ACRP’s Competency Guidelines4, and added additional technical and leadership SBK statements.

 All managers participated and were engaged in a sprint workshop to review the exhaustive list of SBK in each competency domain and consolidated them into a manageable and succinct list.

This list was translated into an easy to digest spreadsheet format.

In partnership with the Learning and Development (herein referred to as L&D) group, a small subset of managers and their direct reports (at each of the three levels of study management) piloted this framework and provided feedback.

 The framework was further refined based on the feedback from the pilot attendees, and finalized.

The finalized framework was shared with managers; an FAQ was developed to aid managers in the roll-out and “office hours” held to answer any questions/

Note: The framework was shared with study management via email and discussed at a forum.

 Check-in meetings were held with managers to answer any questions and incorporate any feedback post-launch.

Updating the job descriptions to ensure that talent is hired to the same standards defined in the framework and to ensure that our job descriptions reflect the skills, behaviors, and knowledge of our workforce.

Developing the training “playlists” to give employees an easy access to all available learning resources.

We plan on updating the defined SBKs within the framework based on feedback from managers and employees. Additionally, we recognize that this effort was unique for the industry, in terms of creating a competency framework for clinical trial leaders. ACRP plans to build upon aspects of the CTL competency framework as they expand their competency resources.

Transitioning from traditional job descriptions and performance assessment tools, to a competency-based roadmap for staff development and assessment was both challenging and rewarding for the core team. Some key takeaways from our experience include:

While some industry-standard competency guidelines exist for other clinical research professional roles (clinical research associates, clinical research coordinators, and investigators)4, there were very few competencies that we could incorporate into the more advanced SBKs required of a clinical trial leader.

It’s easy to go overboard—adding too many sub competencies can make the framework cumbersome and can lead to issues with adoption.

It is very important to bring all managers along in the process; the core team continuously solicited feedback from all managers in sprint sessions along the way.

It is prudent to run a small pilot to gather feedback from direct reports and managers to:

Gather frequently asked questions about the framework and its implementation.

Refine the defined sub-competencies (e.g., delete repetitive or add missing sub-competencies).

Creative approaches are needed to address the clinical research workforce “talent wars.” A business-as-usual mindset won’t solve the problems, but a competency-based approach offers many benefits to the employer as well as employee. While every organization will adopt their own approaches to developing their talent, we hope that the methodology discussed here provides some practical guidance for those embarking on a similar journey.

While innovative hiring and engagement tactics may offer a competitive advantage, making investments in staff is the right thing to do for the patients we are trying to serve.

We look forward to sharing updates and lessons learned as we gain further experience with this initiative.

Harper, B. Talent Talent Wars: How To Develop A Competent Clinical Research Workforce In A Competitive Environment. 

https://www.clinicalleader.com/doc/talent-wars-how-to-develop-a-competent-clinical-research-workforce-in-a-competitive-environment-0001

. Published Dec. 18, 2018. Retrieved March 6, 2020.

Henderson, L. Catch (& Keep) a Rising Star: Clinical Research Talent Trends. 

http://www.appliedclinicaltrialsonline.com/catch-keep-rising-star-clinical-research-talent-trends

. Published March 1, 2018. Retrieved March 6, 2020.

Bureau of Labor Statistics Employee Tenure 2018. U.S. Department of Labor. 

https://www.bls.gov/news.release/pdf/tenure.pdf

. Published Sept. 20, 2018. Retrieved March 6, 2020.

ACRP Partners in Workforce Advancement; Define, Align and Vali- date Competence Standards. 

https://acrpnet.org/acrp-partners-in- workforce-advancement/acrp-partners-in-workforce-advancement- define-align-validate-competence-standards/

Joint Task Force for Clinical Trial Competency. 

https://mrctcenter.org/clinical-trial-competency/

Association of Clinical Research Professionals

Competency-based and other creative approaches are needed to address the battle for talent in the clinical research workforce. 

Beth Harper

Articles

Characterizing Clinical Data Management Challenges and Their Impact

Enabling Digital Transformation: Managing External Clinical Data Sources to Advance Drug Development

Combating the ‘Talent Wars’ in Biopharma