Kenneth Getz is a senior research fellow at the Tufts Center for the Study of Drug Development.

Kenneth Getz

The COVD-19 pandemic and changing global economies have impacted the clinical research workforce resulting in attrition, shortages, and staffing challenges. Current workforce shortages have posed numerous hurdles for clinical research stakeholders, including sponsors, investigative sites, contract research organizations (CROs), and industry associations.

 The role of the clinical research associate (CRA) has been especially affected as high rates of turnover and vacancies proliferate. CRA turnover rates have been reported as high as 30% in the US with similar rates noted globally.

Recent research has examined current approaches to CRA training, performance assessment, and experience. The results of a global survey of 661 clinical research professionals, including 52 CRAs, assessing their own competencies indicated that respondents with more post-secondary education reported higher levels of self-confidence in their research skills and those professionally certified by the Association of Clinical Research Professionals (ACRP) and/or Society of Clinical Research Associates (SOCRA) compared to those who were not. Self-confidence also increased with the level of academic degree.

 Researchers used a Joint Task Force (JTF) for Clinical Trial Competency Framework to assess respondents on eight competency domains. The results are consistent with prior research conducted in 2016.

The JTF for Clinical Trial Competency framework as discussed by ACRP “objectively defines the knowledge, skills, and attitudes necessary for conducting safe, ethical and high-quality research.”

 Although common practice is currently to hire CRAs with two years of experience, high demands, vacancies, and attrition in the industry have led researchers to recognize a need to address the two years of required experience for those entering the clinical research workforce. Proposed solutions include conducting outreach to raise awareness of the profession, capturing measures of experience and transferable skills as well as add to competency-based training and innovative recruitment and retention approaches to attract candidates to the field.

An examination of the entry-level shortage of CRAs cites several factors including the requirement of a minimum of two years of work experience, high CRA turnover, increasing numbers of open CRA positions, and clinical trial growth and complexity. An assessment of a monitoring simulation of 579 CRAs from a global CRO indicates no difference in performance regardless of CRA seniority and years of experience.

 The research also discusses that addressing the requirement of two years of experience for CRA roles would alleviate the current shortages in the workforce.

Given the impact of the changing clinical research industry on CRAs, Tufts CSDD investigated the current role of the CRA and critical issues including shortages and turnover, and approaches to CRA training and required experience. Also, the study examined the industry practice of two years’ experience as a hiring requirement and its relationship to CRA performance and recruitment and retention practices.*

*This research study was funded by Virb, Inc.

The research used a dual methodology comprised of a web survey of pharmaceutical, biotechnology and CRO executives and an executive roundtable with industry experts. Tufts CSDD conducted an online survey prior to the meeting to gauge perceptions and insights about the role of the CRA based on a small group of clinical research professionals from the US Areas examined included years of experience required to become a CRA and type and effectiveness of training provided. In addition, the survey gathered perceptions about CRA shortages and attrition, most successful recruitment approaches and frequently used retention strategies. CRA assessment and performance were also investigated.

A roundtable meeting was held in October 2022 with 33 participants from biotechnology and pharmaceutical companies, CROs, academia, consulting, and technology. Participants provided their insights and perceptions across a broad range of topics including CRA assessment and training and the industry standard of requiring two years of work experience as a prerequisite to hiring. Other areas explored included recruitment and retention practices; the investigative site perspective; and the impact of outsourcing models, technology, and economic factors on the CRA role.

The survey was distributed via a link to an e-mail invitation using Qualtrics software to industry executives in the US primarily working in clinical operations or clinical development. A total of 55 responses were collected between August and September 2022. Respondents represented a mix of company types with over half (56%, n=31) representing pharmaceutical or biotechnology companies; more than one-fourth (27%, n=15) from CROs and 16% (n=9) from other company types. The mean number of company employees was 17,581 with companies ranging in size from 4 to 95,000 employees. Respondents had 21 years of experience on average and an overwhelming majority (91%, n=50) identified their functional area as clinical operations or clinical development.

Survey responses varied regarding the number of years of experience required to become a CRA at respondent organizations with less than half (44%, n=15) indicating less than two years of experience, while more than one-third (35%, n=12) reporting two years of experience. A smaller percentage (15%, n=5) noted more than two years were needed, and a few respondents (6%, n=2) provided other responses. A majority (69%, n=22) reported that their organization has hired a CRA with less than two years of experience, while fewer (28%, n=9) did not or were considering this option (3%, n=1).

One area that emerged from the discussion was whether the current metrics used by organizations are useful and how to accurately measure CRA competency. The group discussed the implications of published data revealing that newer CRAs performed better on a simulation than experienced CRAs.

 The results also indicated that when comparing more experienced CRAs to newer CRAs, experienced CRAs may demonstrate complacency in their roles. A meeting participant discussed the need for CRA performance metrics:

“We don’t measure much coming from CRAs. The simulation is an exciting tool. It may not be perfect, but we need something like that to give a Pass/Fail, or at least a target. Those results are miserably low. You want to see over a two-to-three-year period that your CRAs moved up in performance and then you’re looking at performance in terms of FDA audits and CRA breaches. If we don’t start, we’ll never get to that point. We’ve been looking at this for years and years.”

Using years of experience as the primary criteria for promotions and not performance was brought into question and participants discussed replacing years of experience as an overall performance metric.Although years of experience is currently the primary determinant of promotions, 61% (n=14) of respondent organizations have not examined CRA performance by CRA experience or job level, while fewer (9%, n = 2) have, with 30% (n = 7) planning to in the future. Enhanced pressure by organizations to promote top performing CRAs is common to prevent attrition. Focusing on other key performance indicators (KPIs), such as site satisfaction or evaluating soft skills acquired through maturity and experience were also proposed.

The group acknowledged that various therapeutic areas and phases of studies may pose different challenges for entry-level CRAs. For less experienced CRAs, trials in oncology, rare disease and gene therapy as well as larger phase III trials could potentially be difficult. Therefore, evaluating CRAs within the same therapeutic areas or phases or within one protocol might be more effective. Attendees also discussed giving CRAs access to their assessments, and building them into the CTMS system, so they can use them as a guide for improving their performance.

The investigative site perspective on the role of the CRA

Sites face multiple hurdles in evaluating CRA performance, one of which is achieving consistent performance given the difficulty with gathering metrics from newer CRAs. An added burden is that CRAs and site staff may lack training or specific skill sets required for their roles. Sites expect a CRA to provide a single point of contact to ensure responsiveness to inquiries despite CRAs often being overburdened with work.

One participant discussed the challenges to the CRA role on-site and the impact on studies:

“Study de-prioritization is a real thing that happens at sites. I can tell you this by experience. Calling CRAs and they never pick up the phone, it always must escalate to a director or manager and even then, they’re overwhelmed. Talking about site relationship managers, I have not seen one on any of our studies and the reason why sites de-prioritize a study is the sponsor or the CRO are absent. There are other sponsors who are more connected to the sites than other sponsors. If they aren’t responding or it’s taking too much effort, just put it in the back burner and let it sit.”

CRA turnover also impacts sites and may slow recruitment or cause study delays or terminations. Estimates of the average annual turnover rate of CRAs at organizations varied according to survey respondents. One-quarter (n=8) projected turnover between 15-20%; 22% (n=7) between 0-10%; and 16% (n=5) indicated 20-25% turnover. More than three-quarters (77%, n=23) reported that average time before turnover was more than two years.

A participant discussed “the challenge with sites is that the CRA must be the expert at all things, that resource, and it takes a CRA nine months to be proficient but they’re at a two-year cycle then that turnover is disruptive.” Senior coordinators support new CRAs and are influenced by turnover placing an even greater burden on the investigative site. A proposed solution is to have line managers work with CRAs on-site to improve rapport and teamwork.

In examining where CRAs typically receive training, a greater percentage of survey respondents indicated CRAs were trained by academia (62%, n=21) or company training programs (59%, n=20). Nearly 40% (38%, n=13) reported that ACRP provided training, while fewer (36%, n=12) noted specific programs or other types. The remainder (21%, n=7) mentioned SOCRA or Regulatory Affairs Professionals Society (RAPS). More than one-third of respondents (33%, n=11) reported that new CRAs receive three to four months of training at their organization.

The survey also asked respondents to evaluate the most effective forms of training. An overwhelming majority of respondents (84%, n=26) reported that peer support or mentoring was the most effective training while more than three-quarters (77%, n=24) indicated in-person training was most valuable.Over half noted (52%, n=16) either observational or web-based training models were most beneficial, with 42% (n=13) reporting scenario based and 39% (n=12) experiential training. Three percent felt other trainng types were effective.

Entry-level CRA training varied at participant organizations. Some comprised large in-house training programs, while others included transitioning new college graduates from clinical study administrators to CRAs or shifting clinical research coordinators into CRA roles. Programs ranged from 12 weeks to 9 months and emphasized co-monitoring, mentoring, and training with both senior CRAs and line managers. The training also included quality assessments and use of ACRP assessments.

One participant related the details of their in-house training program:

“The demand for CRAs is growing. There’s definitely an appetite for creative ways to bring people in, we are doing that through partnership programs where you’re turning global study administrators into CRAs through a training program, including a classroom portion that is taking about three months and then actually turning into their day jobs and in another three months they’re doing on-site visits and getting that on-scene perspective.”

Companies have also established global recruitment programs for US based CRA jobs through partnerships with Mexico and Japan. A newer focus of company training emphasizes trust in partnerships. Softer skills such as communication or collaboration also comprise training and may be transferable from other roles if a candidate is new to the field.

A few alternative training approaches were proposed. One strategy included the use of chatbots for CRA questions to clarify issues and conduct training through use of behavioral data. Other methods discussed were utilizing the protocol and training documents as resources or experiential support and training.

Some general training techniques can be specifically applied to train CRAs including critical mistakes analysis of performance. This approach involves training learners on real world challenges with high stakes. Another was the use of spiral design, a training strategy intended to begin with the least complex version of a task and continue to challenge the learning with similar tasks that increase in complexity.One participant noted that “the CRA role isn’t one where individuals improve over time, but rather through training.” Attendees considered that quality of monitoring could potentially be improved through spiral learning.

Industry guidance (ICH E6R2) does not specify number of years of work experience for a CRA but stresses clinical trial specific training and experience. Nevertheless, a two-year requirement has been the industry standard and often incorporated into sponsor-CRO contractual agreements. One participant discussed that the “threshold of two years is a barrier if sponsors are not willing to take the time to train so they can provide their own help with the deficit today.” Many CROs do not have the resources to fill positions and training requires a large investment. Participants unanimously agreed that a commitment to industry training was critical.

In evaluating primary CRA recruitment approaches, larger numbers of survey respondents reported using recruiters (86%, n=25), websites or social media (69%, n=20) and staffing companies (59%, n=17). Additional approaches included professional networks or associations (45%, n=13); universities or colleges (41%, n=12); and other methods (28%, n=8). Lesser numbers recruited CRAs through training programs (24%, n=7) or conferences (17%, n=5).

The greatest number of respondents indicated recruiters (52%, n=15) were the most successful methods of recruiting CRAs while 38% (n=11) used referrals from internal staff, recommendations or appplications. Fewer(31%, n=9) reported that staffing companies, training programs (21%, n=6), professional networks or associations (17%, n=5), universities or colleges (17%, n=5) and conferences (7%, n=2) were successful.

Academic-Industry partnerships are significant to CRA recruitment and provide extensive training through internships and fellowships. Several universities have also established clinical research programs and with demonstrated success in placing students in permanent positions. A participant from a university noted:

“We focus on getting organizations to offer cooperative education opportunities which are paid internships for preparing students not just for the working environment, but also for acquiring soft skills such as preparing their resume, write their cover letters, getting in front of companies, and engaging in the dialogues of what their experiences were.”

There is, however, a pervasive lack of awareness among undergraduate students about clinical research as a profession and of the role of a CRA. In a survey of more than 200 university students in both the US and UK, not a single one had heard of clinical research as a career.

 Recruitment by companies on college campuses over the past five years has led to some increased awareness. A meeting participant in support of recruiting new graduates explained that “the next generation wants to work for a higher purpose and what greater higher purpose than helping humanity and saving lives?” Mentoring students is also critical however, and some universities are unable to fill faculty positions due to the massive pay disparity between industry and academia.

CRA retention and perceptions about industry shortages

The primary reasons for CRAs leaving their positions were for higher salaries (74%, n=23); followed by better growth opportunities (45%, n=14) and burnout (29%, n=9).The top two strategies implemented by organizations were flexible work schedules (65%, n=20) or higher salaries (55%, n=17).

The growth opportunities that CRAs seek are in part impacted by the rapidly changing clinical research industry and by economic factors. Both factors influence retention of CRAs and may inhibit growth opportunities. Yearly salary increases are not sustainable, and they typically leave these roles for higher paying positions.An attendee suggested that other career paths allow a CRA to stay in the industry yet be able to advance:

“We should think about as an industry, career opportunities. What else is next? Central monitoring, data monitoring, regulatory.All require quality whether or not the role is a CRA. We want to keep CRAs while promoting, because that skill set is very valuable as a monitor.”

A CRA may experience disconnectedness from staff on-site for a number of reasons: burnout from having a large work burden; working on complex studies too soon; and managing multiple technologies and systems. One participant described the gap in the support that may contribute to a CRA feeling disconnected: “there is more separation between MDs and clinical teams, and they are not connected with the CRAs on the ground.” In addition, a CRA may currently spend less time on site and more on remote monitoring.

Company outsourced models can also influence retention of CRAs. While CROs are the primary organization that staff CRAs, pharmaceutical organizations are increasingly using internal or headcount CRAs. Participants offered mixed opinions on full service, functional service provider and freelance models. One attendee with headcount CRAs reported that turnover was less than 5%, and another indicated that functional service providers (FSP) resulted in high turnover. A third participant found no differences in performance between headcount and FSP CRAs.

An overwhelming majority of survey respondents (88%, n=28) reported a shortage of CRAs in the industry while 13% (n=4) did not.Opinions were mixed about whether the shortage was getting worse (43%, n=14) or staying the same (57%, n=18), however no respondent felt the situation was improving.

The results of the survey and executive roundtable confirm that organizations are currently accommodating CRAs with less than two years of experience and establishing novel approaches to training. In addition, solutions are currently being implemented or planned to alleviate the impact of CRA shortages and turnover, and these include the following:

Re-examining experience in relationship to performance through using innovative strategies to gather alternative metrics on CRA performance, such as evaluating site satisfaction or assessing soft skills.

Organizations are also planning to link specific metrics such as FDA audits to experience or job level. In addition, evaluating CRAs with similar workloads and providing access to their assessments for self-improvement are strategies being considered. Novel approaches to gathering metrics are still in the early stages of implementation, however, and follow up studies will be critical.

Addressing the shortage of CRAs through raising awareness of clinical research as a profession at the university level.

Recruitment by companies on college campuses and offering university internships have led to permanent placements for students after graduation. Academic-industry partnerships also provide new graduates with valuable real-world experience before entering the workforce.

Mentoring and working collaboratively on-site with managers can address CRA disconnectedness and facilitate teamwork as well as build trust.

Mentoring programs provide an increased level of support for CRAs beyond training programs. In addition, dedicated line managers working on-site offer critical training. These approaches can alleviate burnout and provide the critical groundwork for encountering challenging studies.

Providing growth opportunities to increase retention and address issues of complacency among senior CRAs as well as offering varied incentives to remain within the industry.

A competent CRA can easily transition into other positions where comparable skills are valued and would be able to fill roles in regulatory, trial management, and central or data monitoring. Many organizations support career development of CRAs to increase staff retention.

Given increasing use of technology and growth of decentralized trials, proposing sustainable solutions that tackle industry and global change is critical.

Solutions should address changing organizational models as well as economic factors. In response, some companies are developing in-house training programs. Other organizations are examining the impact of outsourced models on CRA performance as well as their onboarding processes and training programs to implement crucial changes to current practices.

Consider experiential training approaches based on spiral learning that may prepare CRAs to perform effectively without two years’ experience.

The aviation industry trains pilots using simulation, scenarios, and real-time coaching by expert pilots and may be a useful model from which we can learn when designing curricula for training and preparing CRAs to actually monitor clinical trials in the field.

, PhD, associate director and research assistant professor, 

, executive director and research professor; all with Tufts CSDD

Hill M. Clinical research sites share cash and workforce shortage concerns. 

. Published online July 21, 2022.Available at: 

https://www.clinicalleader.com/doc/clinical-research-sites-share-cash-workforce-shortage-concerns-0001

Wieland A. What is really behind the CRA shortage in clinical trials? A root causes analysis. Published online May 18, 2020. Available at: 

https://www.clinicalleader.com/doc/what-s-really-behind-the-cra-shortage-in-clinical-trials-a-root-cause-analysis-0001

Sonstein S, Samuels E, Aldinger C et al. Self-assessed competencies of clinical research professionals and recommendations for further education and training. 

Therapeutic Innovation and Regulatory Science

. 2022; 56:607-615. Published online March 31, 2022. doi: 

Sonstein S, Silva H, Thomas-Jones C et al. Global self-assessment of competencies, role relevance, and training needs among clinical research professionals. Clinical Res. 2016;30(6):42-9. Available at: 

https://acrpnet.org/2016/12/01/global-self-assessment-of-competencies/

Barriers to Bridges: Addressing the Urgent Need for a Diverse, Research-Ready Workforce within the Clinical research profession. ACRP Whitepaper. September 2022 p. 7. Available at: 

https://acrpnet.org/barriers-to-bridges-addressing-the-urgent-need-for-a-diverse-research-ready-workforce-within-the-clinical-research-profession/

Alsumidae M. Entry-level CRA shortage still crippling trials. 

. Published online March 2, 2022. Available at: 

https://www.appliedclinicaltrialsonline.com/view/entry-level-cra-shortage-still-crippling-clinical-trials

Alsumidae M. CRA Skills Lack in Critical Areas. 

. Published online May 22, 2018. Available at: 

https://www.appliedclinicaltrialsonline.com/view/cra-skills-lacking-critical-areas

Survey conducted by Connelly Partners Health for Virb, Inc., March 2022.

Articles

Study by Tufts CSDD uncovers potential reasons behind CRA shortages, turnover, experience requirements, and more.

An Examination of the Role of the Clinical Research Associate and Factors Impacting Performance and Experience

Although we have witnessed increased inclusion of diverse populations in COVID-19 vaccine

 trials over the past two years, in nearly all other disease indications, representation of BIPOC

 populations—black, indigenous, and peoples of color—in clinical trials remains comparatively low, particularly in world regions where most clinical trials are currently conducted

 and where demographics are becoming increasingly more diverse (i.e., North America and Western Europe).

A variety of factors help explain the lack of racial and ethnic representation among clinical trial volunteer participants. Low clinical research literacy, poorly targeted communications, and limited access to clinical research facilities are among top factors mentioned.

 Many note that racial and ethnic representation among the research workforce is essential in reducing barriers to clinical study volunteer participation among historically underrepresented populations.

 A 2008 study conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) found that a physician’s race and ethnicity can influence the race and ethnicity of clinical trial volunteers. The study also found that incidence of participation in clinical research among BIPOC physicians was well below that observed among white physicians. Although interest in becoming a clinical investigator was high among all physicians surveyed, BIPOC investigators tended to conduct and initiate fewer trials annually. In that study, BIPOC investigators also reported fewer infrastructural resources and support compared to white investigators.

More than a decade has passed since the 2008 study. During that time, public and clinical research stakeholder awareness and interest in improving the proportional representation of volunteer participants by race and ethnicity in clinical trials has intensified. Several recent initiatives underscore these changes. The FDA Reauthorization Act of 2017, for example, expanded the FDA’s authority to address barriers to participation in clinical trials (SEC.610.Expanded Access).

 In 2020, FDA issued guidance for the broadening of volunteer eligibility criteria and the enhancement of diversity in clinical trial participation.

 In April 2022, FDA issued draft guidance recommending the development and submission of plans—with each new IND application—for achieving diversity targets, as well as retaining study participants.

 Separately, The Lancet issued a diversity pledge to reinforce its commitment to diversity

 and the New England Journal of Medicine conveyed the importance of diversity in clinical research in a recent editorial.

In late 2020, Tufts CSDD conducted a new study to revisit, characterize, and more deeply examine the relationship between investigative site personnel diversity and study participant diversity.

This research began with the formation of a large Working Group

 of biopharmaceutical companies, contract research organizations (CROs), and one global professional association. The Working Group supported the development of two extensive online global surveys that were distributed between April and July of 2021. These were reviewed and approved by the Tufts University IRB and the university’s General Data Protection Regulation (GDPR) expert reviewer.

The first survey targeted medical professionals, regardless of their involvement in the clinical research process. Its purpose was to gather data about medical professionals regarding their awareness of, access to, training, and involvement in clinical research. The second survey targeted clinical research professionals working at various types of clinical research sites. Its purpose was to gather data on staff diversity at investigative sites, the infrastructure and support available to clinical research staff, and staff perceptions about Diversity, Equity, and Inclusion (DEI).

Definitions of race and ethnicity were presented to respondents participating in both surveys as shown in Table 1. These definitions are consistent with those reported in a 2021 study looking at race and ethnicity of COVID-19 patients participating in 68 studies worldwide.

 Race and Ethnicity—Definitions Presented to Respondents

Survey one: Awareness of, access to, training in, and involvement in clinical research among medical and allied health professionals

In total, 611 respondents participated in the online survey aimed at medical professionals regardless of their involvement in clinical research. Of these, 54% were in North America (US & Canada) and 46% were in Rest of World (ROW). Regarding their education levels, 62% had a doctoral or medical degree, 24% had a nursing degree, and 14% had achieved other educational levels, such as a bachelors or high school diploma. Respondents to the survey were 56% female and 44% male. Racial and ethnic identities were reported as follows: 68% White, 15% Asian, 10% Other (including mixed race/ethnicity), 4% LatinX, and 3% Black.

The survey results show that most medical and allied health professionals who are involved in clinical research believe that mentors made them more comfortable with (1) the clinical trial process (93%), and (2) referring and screening patients for clinical trials (91%). Most medical and allied health professionals involved in clinical research (87%) also believe that mentors helped them find greater access to clinical trials.

The survey findings also indicate that a high proportion of medical and allied health professionals first became exposed to clinical research at the invitation of a peer or mentor. Differences in exposure through a mentor or peer were also observed between BIPOC and white professionals, with 42% of white professionals reporting becoming involved in clinical research in this manner compared to only 30% of BIPOC professionals. Other differences in exposure among BIPOC and white professionals were found when respondents were asked if they first became exposed to clinical research by proactively contacting a pharmaceutical company about opportunities to participate as staff supporting or leading clinical trials. Thirteen percent of BIPOC professionals reported becoming involved in clinical trials by being proactive compared to 6% of white professionals.

Further, differences in access to clinical research opportunities and infrastructure were identified between BIPOC and white medical and allied health professionals who have not been involved in clinical trials. Table 2 shows the barriers to involvement in clinical trials as reported by Physicians, PhDs and nurses who have not been involved in clinical research worldwide.

 Barriers to Involvement in Clinical Research as Reported by Professionals Without Clinical Trial Experience (Global)

Survey two: Investigative site diversity, infrastructure, and support available to clinical research site staff

The primary aim of the second survey was to understand current racial and ethnic representation among research professionals at clinical research sites supporting clinical trials funded by pharmaceutical and biotechnology companies, and the elements that influence staff diversity at these sites. These elements include (1) the types of communities served (i.e., urban, suburban, and rural), (2) the income levels of the patient population served, (3) staff’s access to technologies, (4) training and support available to site staff, and (5) the site’s infrastructure. To that end, in addition to the definitions shown in Table 1, respondents to this survey were also provided with the definitions included in Table 3 below.

 Site Characteristics—Subgroup Definitions

In total, 3,462 respondents—representing an estimated 3,340

 unique sites—participated in this global

 online survey. Of these respondents, 40% were principal investigators, 36% were site directors and managers and other administrative staff, and 25% were clinical research coordinators (CRCs) or clinical research nurses. Respondents to this survey were 61% female and 39% male. Racial and ethnic identities were reported as follows: 69% White, 12% LatinX, 8% Asian, 6% Mixed, 4% Black, and 2% Other.

In addition, survey respondents were asked to estimate the racial and ethnic breakdown/composition of the staff working at their respective sites. Table 4 shows this breakdown by world region and type of research institution. Overall, BIPOC populations were not widely represented among staff at clinical research sites globally. When comparing BIPOC percentage representation by type of institution in North America, higher representation was observed in private sites and site networks compared to academic medical centers and large healthcare systems. When comparing BIPOC percentage representation by type of institution in Europe, significant differences in representation were not observed among private sites, site networks, academic medical centers, and large healthcare systems. BIPOC representation was higher among staff at research sites located in ROW.

 Distribution of Investigative Site Staff by Race and Ethnicity

Clinical research sites were also classified into one of two groupings: ‘High’ Staff Diversity Sites and ‘Low’ Staff Diversity Sites. Sites in the US were classified as High Staff Diversity if their staff was at least 60% BIPOC. Sites in the US were classified as Low Staff Diversity if their staff was 40% or less BIPOC. Sites in locations outside of the US were classified as High Staff Diversity Sites if representation by any one race or ethnicity was 40% or lower; they were classified as Low Staff Diversity Sites if representation by one race or ethnicity was 60% or higher. Sites that fell outside of these parameters (i.e., the 50th percentile) were excluded from supplementary analyses. It is important to note that less than 1% of European sites met the criteria for the High Staff Diversity grouping—this includes sites in Eastern Europe as well as the UK, Germany, Spain, France, and other parts of Western Europe.

Clinical trial volunteer diversity is strongly associated with diversity in the clinical research workforce

Racial and ethnic representation among the clinical research site workforce is associated with racial and ethnic representation in clinical trial volunteer enrollment. To explore the strength of this relationship, respondents were asked to estimate the racial and ethnic breakdown of patients enrolled in their recently conducted clinical trials. These estimates were then cross tabulated against the groupings for High Staff Diversity Sites and Low Staff Diversity Sites, and chi-square tests were conducted to determine statistical significance. As shown in Table 5, the enrollment of BIPOC patients was significantly higher at High Staff Diversity Sites, including all site types. This result was statistically significant.

 Relationship Between Staff Diversity in Diversity in Patient Enrollment in Trials (Global)

The relationship between diversity in patient enrollment and investigative site staff diversity has important implications for the development of DEI plans and policies. If the aim is to increase representation among clinical trial volunteers, policies and plans must also prioritize improvements in representation of BIPOC populations among investigative site staff; thus, understanding the basic characteristics of sites that show High Staff Diversity is a meaningful starting point for future initiatives in this space.

Characteristics of investigative sites reporting higher staff diversity

This study sheds light on the characteristics of sites reporting higher levels of staff diversity—though it also uncovers barriers faced by those sites reporting lower levels of staff diversity. Sites reporting higher racial and ethnic diversity among their staff tended to be in urban areas, service low-income communities, and have centralized clinical trials offices to support research teams—in greater proportion than sites reporting lower staff diversity levels. As shown in Table 6, two of these findings were statistically significant.

 General Characteristics of Investigative Site Staff by Diversity Subgroup

Clinical research sites reporting High Staff Diversity levels also reported greater use of technologies and infrastructure to support the execution of clinical research compared to Low Staff Diversity sites, as shown in Table 7. The difference in use of Patient Management Systems at High Staff diversity sites compared to Low Staff Diversity sites was statistically significant. Additionally, provision of adequate access to Patient Management Data was higher at High Staff Diversity sites. The analysis also revealed that, prior to the pandemic, remote communication platforms were used in higher proportion at High Staff Diversity sites compared to Low Staff Diversity sites, and that the overall current use of these platforms was higher at High Staff Diversity sites.

 Use of Technology and Infrastructure by Diversity of Investigative Site Staff

Not least, this research revealed that High Staff Diversity Sites tended to offer better access to diversity and inclusion, unconscious bias, and inclusive behavior SOPs to its staff. The offering of inclusive behavior SOPs was notably higher at High Staff Diversity sites.

 Presence of Diversity SOPs by Diversity of Investigative Site Staff

The Tufts CSDD Investigative Site Personnel Diversity Study results demonstrate association between reported levels of racial and ethnic diversity among personnel at investigative sites and racial and ethnic representation among volunteer participants enrolled in clinical trials. This finding suggests that increased diversity within the clinical research workforce facilitates the racial and ethnic representation of study volunteers in clinical trials at individual research sites.

This study also offers new understanding about shared characteristics among sites reporting comparatively higher staff diversity and offers insight into the paths followed by medical and allied health professionals toward becoming involved in clinical trials. These elements are key considerations and may be important strategies supporting improvements in racial and ethnic representation among clinical trial volunteers.

Specifically, our results suggest that expanding exposure to clinical research opportunities to all medical and allied health professionals can lead to greater overall staff participation in clinical investigation. Our results show that 87% of all professionals involved in clinical research believe that mentors helped them find greater access to clinical trials, and over 90% of all medical and allied health professionals involved in clinical research report that a mentor made them feel more comfortable with the clinical trial process, as well as in referring and screening patients for trials. However, only 30% of BIPOC medical and allied health professionals first became involved in clinical research by invitation by a mentor or peer compared to 42% among their white counterparts. This last result highlights a disparity in access to mentors in clinical research and the benefits that accompany such connections. Mentorship and other research socialization programs are therefore expected to be effective in increasing overall and BIPOC staff participation in clinical research.

The study findings indicate that the presence of DEI-specific materials and SOPs may play a role in supporting workforce diversity and, in turn, foster diversity and inclusion among study volunteers. First, a strong relationship was identified between High Staff Diversity sites and the availability and use of Diversity and Inclusion, Unconscious Bias, and Inclusive Behaviors SOPs. Second, High Staff Diversity Sites reported access to these SOPs in higher proportion compared to Low Staff Diversity sites. Third, approximately half of staff at clinical research sites—across site types and locations—believe that SOPs for Diversity & Inclusion, Unconscious Bias, and Inclusive Behaviors help increase participant diversity at sites. Our results set the stage for drug development stakeholders to collaborate in the development and dissemination of materials, policies and procedures that elevate awareness and understanding of the importance of inclusive behaviors and minimizing biases. This collaboration is not just isolated to the level of the investigator site, but also can benefit from involvement of sponsoring organizations and CROs to support such efforts to increase DEI awareness.

Altogether, our research demonstrates that a diverse clinical research workforce is associated with diversity among clinical trial volunteers, shows that diversity in the clinical research workforce still falls short of reflecting the diversity of the global patient population, and identifies key areas that can support increased racial and ethnic representation among site staff: offering mentorship in clinical research to medical and allied health professionals, as well as SOPs and educational materials related to diversity and inclusion, unconscious bias, and inclusive behaviors.

, MA, Research Consultant and Senior Analyst, Tufts CSDD, Tufts University School of Medicine, 

, BA, Research Analyst, Tufts CSDD, Tufts University School of Medicine, 

, PhD, Strategic Feasibility Manager, UCB, 

, Senior Director, Global Feasibility, Labcorp Drug Development Inc., 

, PhD, Senior Director Patient Insights and Solutions, Portfolio Transformation and Insights, Biopharmaceuticals R&D, AstraZeneca, Gothenburg, Sweden, 

, Director Oncology Hematology, Janssen Research & Development, LLC,

, MBA, Executive Director and Professor, Tufts CSDD, Tufts University School of Medicine

 for his contributions. The authors also thank representatives from the following organizations for providing research grants and support to this study: AbbVie, Amgen, Association of Clinical Research Professionals, AstraZeneca, Biogen, Bristol Myers Squibb, Covance (now Labcorp Drug Development), CSL Behring, Eli Lilly and Company, EMD Serono, ICON, IQVIA, Janssen, Merck, Otsuka Pharmaceuticals, Parexel, Pfizer, Sanofi, Seagen, Syneos Health, Takeda, UCB, and Genentech, a member of the Roche group. Additionally, the authors wish to thank the following for their assistance: Philippine Nurses Association of America, Asian American Pacific Islander Nurses Association, Taiwan Clinical Research Association, Academy of Physicians in Clinical Research, European Society for Clinical Investigation, ARMA UK, ARCS Australia, Hong Kong Medical Association, the Israeli Medical Association, and the Society of Clinical Research Sites.

Artiga S, et. al. Racial Diversity within COVID-19 Vaccine Clinical Trials: Key Questions and Answers. Kaiser Family Foundation. 2021.

Stewart J, Krows ML, Schaafsma TT, et al. Comparison of Racial, Ethnic, and Geographic Location Diversity of Participants Enrolled in Clinic-Based vs 2 Remote COVID-19 Clinical Trials. JAMA Netw Open. 2022;5(2):e2148325. doi:10.1001/jamanetworkopen.2021.48325

An article published by the New York Times states that the earliest use of the term BIPOC was in 2013. The acronym stands for black, indigenous, and peoples of color.

https://www.nytimes.com/article/what-is-bipoc.html

Getz K, Smith Z, Pena Y.Quantifying Patient Subgroup Disparities in New Drugs and Biologics Approved Between 2007 - 2017. TIRS (2020); 54(6): 1541-1550.

Bierer B, White S, Maloney L, Ahmed H, Strauss D, Clark L.Achieving Diversity, Inclusion and Equity in Clinical Research.(2020). 

https://mrctcenter.org/blog/resources/achieving-diversity-inclusion-and-equity-in-clinical-trials-principles

Holden L and Richardson LD. Disparities and Diversity in Biomedical Research. Chapter 18. Diversity and Inclusion in Quality Patient Care (Martin et al). Springer International Publishing. Switzerland. (2016)

https://www.congress.gov/115/plaws/publ52/PLAW-115publ52.pdf

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial

https://www.fda.gov/media/157635/download

Editorial. Striving for Diversity in Research Studies. N Engl J Med 385; 15. October 7, 2021.

The Tufts CSDD Investigative Site Personnel Study Working Group included the Association of Clinical Research Professionals (ACRP), five contract research organizations (Covance—now Labcorp Drug Development, ICON, IQVIA, Parexel, Syneos Health) and 17 biopharmaceutical companies (AbbVie, Amgen, AstraZeneca, Biogen, Bristol Myers Squibb, CSL Behring, Eli Lilly and Company, EMD Serono, Janssen, Merck, Otsuka Pharmaceuticals, Pfizer, Sanofi, Seagen, Takeda, UCB, and Genentech, a member of the Roche Group).

Magesh S, John D, Wei, et al. Disparities in COVID-19 Outcomes by Race, Ethnicity, and Socioeconomic Status, November 11, 2021. JAMA Network Open. 2021;4(11):e2134147. doi:10.1001/jamanetworkopen.2021.34147 (Reprinted)

Racial and Ethnic Categories and Definitions for NIH Diversity Programs and for Other Reporting Purposes. April 8, 2015. National Institutes of Health. Retrieved 1 March 2022 from 

https://grants.nih.gov/grants/guide/notice-files/not-od-15-089.html

United Nations Ethnocultural Characteristics. United Nations Statistics Division. Retrieved 1 March 2022 from 

https://unstats.un.org/unsd/demographic/sconcerns/popchar/popchar2.htm

Cunningham, W, Jacobsen, J. Earnings Inequality Within and Across Gender, Racial, and Ethnic Groups in Four Latin American Countries. World Bank Policy Research Working Paper. April 2008. Retrieved 1 March 2022 from 

https://openknowledge.worldbank.org/bitstream/handle/10986/6724/wps4591.pdf?sequence=1&isAllowed=y

Jensen, E, Jones, N, Orozco, K, Medina, L, Perry, M, et al. Measuring Racial and Ethnic Diversity for the 2020 Census. August 4, 2021. United States Census Bureau. Retrieved 1 March 2022 from 

https://www.census.gov/newsroom/blogs/random-samplings/2021/08/measuring-racial-ethnic-diversity-2020-census.html

Standards for the Classification of Federal Data on Race and Ethnicity. August 28, 1995. Executive Office of the President, Office of Management and Budget (OMB), Office of Information and Regulatory Affairs. Retrieved 1 March 2022 from 

https://obamawhitehouse.archives.gov/omb/fedreg_race-ethnicity

This estimate is based on an analysis of 12 unique variables included in the dataset and on the application of a 3.4% duplication rate. The number of sites represented here accounts for approximately 40% of the total number of unique investigative sites involved in FDA-regulated clinical trials in 2020.

Sixty-two countries were represented. Fifty percent of sites were in the US and Canada, thirty-two percent were in Europe, ten percent were in Central and South America (including Mexico), four percent were in Asia and Pacific, and four percent were in ROW (rest of world).

2020 Tufts CSDD study examines relationship between investigative site personnel diversity and study participant diversity.

Improving Diversity in Clinical Trial Volunteer Participation by Addressing Racial and Ethnic Representation Among the Clinical Research Workforce

It is widely recognized that the clinical research enterprise embraced numerous changes and adaptations during the COVID-19 pandemic to maintain patient safety, accelerate the development pathway, and ensure clinical trial continuity. Topping the list of changes is the unprecedented number of protocol deviations that have been implemented since the global lockdown.

Sadly, we do not have empirical data at this time characterizing the expected sharp increase in the average number of deviations per protocol. However, my team at the Tufts Center for the Study of Drug Development (Tufts CSDD) has recently completed a study capturing pre-pandemic protocol deviation metrics from which future comparisons can be made.

Typically, protocol deviations are executed by study staff while the clinical trial is underway in an effort to accommodate study volunteers encountering difficulties complying with the schedule of visits or adhering to study medication administration requirements. Investigative site personnel may also deviate from the protocol when facing difficulties finding eligible patients, and following and performing clinical and administrative procedures dictated by
the protocol.

Most deviations have a minimal impact on protocol integrity and execution. A subset of all deviations, however, are more significant and may harm the quality of the research, may put the rights and safety of study volunteers at risk, and may detract from the completeness, accuracy, and reliability of the study data. There are a number of useful resources to help organizations identify, classify, and manage protocol deviations including those by the Drug Information Association’s (DIA) Good Clinical Practice and Quality Assurance community and by TransCelerate BioPharma.

Finally, its important to note that protocol deviations are the top cause of FDA inspection warning letters. Approximately one-third (30%) of all warning letters are due to the failure to follow the investigational plan. Site failure to adequately maintain case report records is the second most common reason, accounting for 17% of all warning letters.

To our knowledge, little-to-no data exists quantifying and benchmarking protocol deviation experience overall and by clinical phase. For this reason, Tufts CSDD engaged a working group of 20 major and mid-sized pharmaceutical companies and contract research organizations (CROs) to gather hard data on deviations associated with traditional clinical trial protocols (i.e., master protocols and adaptive designs were not included). Our study sample included protocols that had received final approval between January 2013 and December 2018 and had a primary completion date or database lock date before December 31, 2019. The data-collection process deployed in this study followed an approach that Tufts CSDD has been using to evaluate trends in, and the impact of, protocol design practice since 2008.

Data on a total of 187 protocols targeting a range of disease conditions was analyzed. One-quarter of the total were Phase I protocols, 39% Phase II, and 36% were Phase III protocols.

The table on the facing page provides high-level baseline measures of protocol deviation experience by clinical phase. Phase I protocols have the lowest mean number of deviations and involve half the proportion of patients than those observed in Phase II and Phase III protocols. Each Phase III protocol has a mean number of 119 total deviations, involving approximately one-third of all patients participating in that protocol. The coefficients of variation around the mean number are very high—between 1.7 and 2.0—indicating that the average deviations per protocol are widely dispersed and difficult to predict.

Wide variation in protocol deviation experience was also observed by therapeutic area. Phase II and III oncology protocols average almost 20% more total deviations than the average for non-oncology protocols, at 108.8 and 91.9, respectively. Protocol deviations in combined Phase II and III oncology protocols involve 47% of the total study volunteers, nearly double the proportion observed in non-oncology protocols. Rare disease indications average a lower relative number of total deviations (78.1) among a smaller proportion of study volunteers (27.7%) compared to that of non-rare disease indications.

Tufts CSDD also found that the incidence of protocol deviations was significantly, positively correlated with all clinical trial milestone durations, with the exception of study start-up and study close-out. The duration to complete all first-patient visits (first patient first visit to last patient first visit), the overall treatment duration (first patient first visit to last patient last visit), and the total clinical trial duration from protocol approval to database lock showed weak but significant positive correlations with the number of protocol deviations per protocol.

We were not surprised to find that no association existed between the incidence of protocol deviations and study start-up and close-out cycle times. Protocol implementation burden is generally low during the study initiation process. This burden, however, for both study staff and study volunteers, becomes more pronounced as these parties must comply with protocol implementation requirements over time.

A relationship between select protocol design complexity variables and the incidence of protocol deviations was observed. The number of endpoints and the number of procedures per visit were modestly positively associated with, and predictive of, the incidence of deviations per protocol. Adjustments to these select design variables may play a role in preventing protocol deviations and in minimizing their risk to the study and data integrity.

Two operational variables were also positively associated with protocol deviations. The number of countries showed a modest relationship. A positive relationship was also demonstrated between the number of investigative sites and the number of protocol deviations, suggesting that decisions and behaviors that diverge from the investigational plan may be a function of variation in investigative site personnel training, site infrastructure, operating environment, and culture.

Tufts CSDD plans to build on this body of research. We will be looking at the root causes of protocol deviations and how to address them. Future research will also look at deviations with more granularity to understand the incidence and impact of minor and major deviations. Whereas some deviations may improve participation convenience and clinical trial executional feasibility, other deviations may pose substantially greater risk to study volunteer rights and safety and to the quality and integrity of the clinical trial data. We also hope to understand the relationships between types of deviations and patient characteristics—including demographic subgroup and geographic location—and investigative site characteristics.

The results of this initial study provide useful measures that sponsors and CROs can use to benchmark their own protocol deviation experience and identify factors most associated with protocol deviations. 

Study uncovers pre-pandemic deviation levels.

Quantifying Protocol Deviation Experience by Clinical Phase

During the past decade, there has been unprecedented focus on patient-centric initiatives in support of drug development planning and execution. The original intent of these initiatives was to amplify patient input, address patient needs and preferences, and ensure that patients have the best possible experience participating in clinical trials. As investment and experience with patient centric initiatives has evolved, a growing number of organizations have looked to measure the impact of patient-centric activities on a variety of clinical trial outcomes includingspeed and quality.

 Although these outcome measures inform project management, resource allocation and return-on-investment decisions, they are ‘trial-centric’; they are not patient centric.

Key indicators of patient experience—most notably satisfaction—are largely missing from the outcome measures that organizations are systematically gathering. Such routine assessment of patient satisfaction is essential to measuring the successful implementation of patient-centric initiatives and to informing continuous improvement of these initiatives.

The purpose of this article is to remind clinical research stakeholders to re-prioritize routine assessment of patient satisfaction as a primary outcome measure and to inform how to approach that assessment.

Patient satisfaction is not a new concept. It has been covered extensively in the medical science literature. Studies have consistently shown that there exists a positive link between patient satisfaction and a variety of patient-oriented clinical care outcomes, including in-patient mortality, treatment adherence, and improved rapport with care providers.

 Given this compelling and consistent association, it is only fitting to consistently measure and evaluate patient satisfaction.

Yet, based on a recent literature review of patient satisfaction in clinical research specifically, only a small fraction of studies (nine out of several thousand) reported systematically evaluating patient experiences during a clinical trial.

 Further, of the nine studies that reported measuring patient satisfaction, only one study used a widely accepted and validated patient satisfaction instrument (Modified Client Satisfaction Questionnaire).

 The remainder used ad hoc, investigator-developed patient satisfaction instruments.

These findings were recently corroborated at a meeting among patient engagement stakeholders from sponsor companies and contract research organizations (CRO) where the majority of companies conceded that they are not consistently measuring patient satisfaction in their clinical trials. The inconsistency and sparseness in how patient satisfaction is measured should not come as a surprise given the limited number of patient satisfaction scales developed exclusively for clinical research.

There is clearly little to no consensus on what is the most appropriate instrument for measuring patient satisfaction. This by no means suggests that organizations should be satisfied with the current state of affairs. Rather, lack of a consensus measurement-method signifies an opportunity to test out and validate different approaches for capturing patient experience in clinical research. To begin, there already exists a recognized and validated non-clinical research patient satisfaction instrument, the 8-item Modified Client Satisfaction Questionnaire. This instrument is commonly used in many different industries and has been adopted in the medical science and healthcare literature.

This measure could provide an appropriate starting point from which organizations can expand and refine (i.e., adding additional questions about different aspects of the clinical research study).

There are numerous factors that impact a patient’s clinical trial experience ranging from their relationship with clinical research staff and healthcare providers to their perceptions of the research facility, location of the study site, ease or difficulty adhering to protocol requirements and the ease with which technology solutions can be used. It is important to note that the domains that are assessed are likely to change depending on the unique and custom design of each clinical trial and the unique patient population served (e.g., demographics, social determinants). Generally, it is understood that there is no one-size-fit-all-approach and this is certainly true in the endeavors to measure the patient experience. However, there still needs to be more of a unified effort and approach among sponsor and CRO companies to capture patient satisfaction across different domains covering the clinical research experience.

Reminding clinical research stakeholders to reprioritize routine assessment of patient satisfaction as a primary outcome measure in patient-centric activities.

Kenneth Getz

Articles

An Examination of the Role of the Clinical Research Associate and Factors Impacting Performance and Experience

Improving Diversity in Clinical Trial Volunteer Participation by Addressing Racial and Ethnic Representation Among the Clinical Research Workforce

Quantifying Protocol Deviation Experience by Clinical Phase

Measuring Patient Satisfaction as a Primary Outcome for Patient-Centric Initiatives

Tracking Change in the Global Investigative Site Landscape

Assessing Sponsor and CRO Awareness