Jennifer Kim

Healthcare Provider (HCP) and Patient Advisory Boards (PAB) or Community Advisory Boards (CAB) have become common practice

 among clinical trial sponsors to evaluate the validity and feasibility of study protocols. The expertise of HCPs is essential to the outcomes of these advisory boards (AdBoards); however, recruitment of HCPs for research studies can be challenging, and not much has been written about how to tackle these challenges. The nature of the patient-facing healthcare profession often means that carving out time for AdBoard participation may be more of a barrier for HCPs than the average patient targeted for such discussions.

The COVID-19 pandemic has further impeded attempts at gathering data from HCPs, including physicians, nurses, and physician assistants. The well-documented burnout effect of the pandemic and the increase in requests for surveys or focus groups investigating COVID-19 experiences,

 further contribute to survey and videoconference fatigue. Such difficulties in the recruitment of HCPs can hinder successful implementation of AdBoards. Hence, recommendations and strategies to attract more eligible HCPs to participate in these AdBoards are essential to the support of drug development research.

The increasing prominence of the internet, and by extension the ability to host AdBoards, focus groups, and interviews virtually, has reduced geographic and time-related barriers that previously prevented recruitment of a wider, more diverse sample of participants.

 Although research around the transition from in-person to virtual AdBoards began well over a decade ago,

 the uptake in the medical field has been relatively slow and publications around recruitment strategies and methodology for virtual studies have remained noticeably thin.

 The COVID-19 pandemic, however, forced a rapid shift towards virtual interactions, catalyzing a few studies around the pandemic’s effect on qualitative research methods and best practices. According to a 2021 study by MphaR, 20% of advisory boards had already moved online prior to the COVID-19 pandemic, but 2020 saw this number jump to 80%.

 Despite some research emerging on the topic, indicating increasing interest, the literature on best methodologies for conducting virtual qualitative research in medical science remains scant. The shift in how we recruit and incentivize HCP research participants, in particular, requires additional research for adaptive strategies.

Thus, the purpose of this article is to share learnings related to our effort to recruit HCPs to a virtual AdBoard during the pandemic. We outline the challenges to recruitment, the adaptive strategies used in response to these challenges, and suggest recommendations for future studies.

The aim of this study was to gather a group (n = 4 – 6) of US-based patient-facing healthcare providers with experience treating patients with multiple myeloma to provide input on a Phase III clinical trial protocol for an experimental drug. The inclusion criteria were: (1) US-based patient-facing HCP (2) Currently treating patients with multiple myeloma. Recruitment began following exemption by an independent ethical review board.

Recruitment efforts were done virtually and lasted approximately 2 months. A personalized email containing a screener link to determine eligibility was created. This email contained logistical details related to the duration of the AdBoard (two 2-hour videoconference sessions), participants’ responsibilities (i.e., reviewing the study protocol and providing feedback), and compensation (market rate). In the same email, participants were asked to share the study with qualified colleagues who may be interested in participating. Participants’ input was reviewed by the research team to determine their eligibility. Eligible participants were then contacted regarding their availability.

To ensure that we reached a diverse population of HCPs, we used several outreach outlets. During the first outreach effort (Wave 1), we used an existing purchased list of 733 US-based HCPs comprised of physicians, nurses, and physician assistants in oncology. Recipients who had not clicked on the link or who had not finished the screener were sent a reminder email one week after the initial email, and a second follow-up two weeks after the first. Individuals who chose to click the “unsubscribe” link, or reached out directly to request removal, were immediately removed from the contact list.

The above recruitment strategy was supplemented with a more targeted strategy (Wave 2a), which involved searching for email addresses for potential participants from publicly available sources, including Pubmed, 

, and multiple myeloma research and healthcare institution websites. We considered Wave 2a a pilot run with the intention to expand the strategy depending on the response rate. Our initial search using this method yielded 106 contacts. No other elements of the recruitment strategy were altered for Wave 2a. The following search criteria was used on both PubMed and 

: “multiple myeloma” [or] “hematology.” Filters were applied on Pubmed to exclude any article published before 2017. We also noted the participant’s gender and racial background where possible. This targeted approach resulted in a larger, more diverse pool of eligible participants and an increased enrollment rate, which we will detail in the results section.

Following the initial trial run (Wave 2a), we expanded our effort using this method, which yielded 677 contacts who were sent an email via Qualtrics. In this wave, we made two adjustments: 1) the introduction text was revised to include session dates and was considerably shortened and 2) as opposed to a personalized link, which was used in Wave 1 and Wave 2a, an anonymized link was sent, making it easier for recipients to share the screener with their network. One reminder email was distributed one week after the initial invitation to those who had not unsubscribed from the list.

Lastly, a small number (n = 20) of professional contacts within the research team’s immediate network were also contacted for recruitment.

Participants who qualified for the study were sent an official invitation, consent form, and a Zoom link. A reminder was sent 24 hours prior to the first session to ensure attendance. Overall, two 2-hour AdBoard sessions were held among 5 US-based multiple myeloma healthcare providers. Sessions were recorded and transcribed.

A pre-purchased list of 733 physicians, nurses, and physician assistants in oncology were contacted in late October 2021 during the first phase of recruitment. Of these 733 recipients, 20 unsubscribed (2.7% unsubscribe rate) and 51 emails bounced (7.0% bounce rate). An additional 4 recipients reached out to the research team directly to clarify that they did not treat multiple myeloma and were therefore not interested in the study.

Twelve invitees opened the screener link (“clicks”), all of whom completed the screener. Two additional screener responses were recorded from referrals. One screener respondent specified that they were unable to participate due to scheduling restraints. Of 14 potential participants, 3 were deemed eligible and 2 were enrolled in the study.

Wave 2a was sent to 106 contacts collected from publicly available sources, resulting in 2 clicks and 1 screener completion. The individual who completed the screener was deemed eligible and was enrolled in the study.

Wave 2b resulted in 677 contacts gathered from Pubmed and 

 who were contacted between early to mid-November 2021. Of the 677 contacts, 9 invitees opened the screener link and 7 finished filling out the screener. Six additional recipients reached out via email to inform the research team that they were not available during the session dates, or that they would not be able to devote 4 hours to the project regardless of the scheduled times. These participants did not click on the screener link but expressed interest in participating. One additional screener response was recorded from referrals. A second referral never filled out the screener but was deemed eligible based on known background and experience, however, this individual was not able to participate due to scheduling conflicts. Of the 9 potential participants, 6 were deemed eligible and 2 were enrolled in the study.

Four recipients unsubscribed in this wave, and 1 reached out to the research team requesting removal from the list. See Table 2 and Figure 1 below for recruitment summary for both waves.

Overall, 5 individuals were identified for AdBoard participation and completed all 4 hours of the AdBoard. The original recruitment goal was 6 to 8 participants total, making the final group fall short of meeting this quota. However, this number of participants was sufficient for this AdBoard. Enrolled participant characteristics are provided in Table 1 below.

The screener distribution through the purchased list received a slightly higher volume of responses and a slightly higher response rate overall compared to gathering contacts from publicly available sources. However, the higher response rate did not yield a higher participation rate such that only 21.4% of those contacted via the purchased list were eligible compared to 70.0% of those identified on Pubmed and 

. This problem could be avoided by purchasing a more specialized list; however, such lists can be harder to generate depending on how narrow eligibility requirements may be. For non-physician HCPs, the availability of specialized lists narrows, as many nurse and physician assistant contact lists are not specific to treatment specialty.

In terms of cost, many list providers require a minimum expenditure, the lower end of which typically falls between $1,000 and $2,000 for an annual license, with the cost per contact varying depending on number of uses, type, and size of overall purchase. Specialized contact lists are difficult to use for future projects for general research teams, and the requirement of a larger number of contacts without the ability to repurpose them for other projects further decreases the benefit of purchasing a contact list. While some companies offer a single-use option, which can decrease the overall cost, this gives the researchers less control over the database, and may restrict the ability to send reminder or follow-up emails. Another limitation of purchased lists is that their viability cannot be guaranteed despite their high cost. For example, in this study, the purchased list carried a 7.0% bounce rate.

The distribution using contacts manually collected from Pubmed and 

 proved to be more fruitful. This method allowed the research team to generate a contact list of highly specialized HCPs using targeted keywords (i.e., “multiple myeloma”), which increased the eligibility rate of respondents despite a comparable click rate and referral rate to the purchased list distribution. Further, we were able to track the participant’s demographics (i.e., gender, race, and provider type – physician, nurse, etc.), allowing us to create a more diverse pool of potential participants. The slightly lower click rate and screener completion rate for the Pubmed/

 distribution may be a result of decreased availability from recipients since individuals from Wave 2b were contacted closer to the session dates and were therefore more likely to have scheduling conflicts. This supposition is supported by two factors:

Wave 2a (publicly available sources), which was distributed during the same time period as Wave 1 (purchased list), carried a slightly higher click rate (1.9%) than the purchased list (1.6%)

An increased number of individuals during Wave 2 directly contacted the research team regarding scheduling conflicts

Although Wave 2 also resulted in a high bounce rate (17.0%), the fact that the research team was still able to enroll more eligible participants with this method suggests that a high bounce rate may not heavily affect the success of this recruitment method. However, to avoid collecting out-of-date contact information, we recommend applying a 5-year “Publication Date” filter to the Pubmed search criteria to account for turnover within the institution. 

 also has a filter that only includes studies that are currently recruiting in the search results, which would produce more up-to-date contacts as well. The likelihood of turnover within target institutions is especially high within the last few years due the COVID-19 pandemic, making the application of these filters important for the viability of contacts.

During recruitment, manually gathering contacts from Pubmed and 

 was more time-consuming than using a purchased list, as it required combing through webpages and hand-recording each entry. The research team conducted further research following the conclusion of this study and found that there are software packages (i.e., R and Python) that allow for easier gathering of emails. One such package is R’s 

whichallows researchers to retrieve Pubmed records, including names, affiliations, and emails where available. However, this is only possible for US contacts, as data protection regulations in some ex-US countries (primarily those within the European Economic Area) prevent the availability of this data scraping method internationally. By using appropriate keywords, this program can conduct similar recruitment methods as searching through the platform manually by automatically gathering names and contact information.

Another method includes adding keywords to the Pubmed advanced search function and downloading the full Pubmed text file. From there, it is possible to extract emails from the text file. This method, however, does not include first or last names,

 making it difficult to personalize emails, which some studies deem more effective in reaching potential participants,

 though others have found that personalization has little to no effect on recruitment and enrollment.

 This method is simpler and may be preferable for research teams without experience in R programming.

Utilizing publicly available resources such as Pubmed and 

 can be helpful in AdBoard, focus group, or other qualitative studies where eligibility requirements are strict; the research team can manually comb through records or scrape the webpage for contact information; and/or the study budget does not allow for purchased lists or other common recruitment methods such as social media advertisements.

We note that automatic extraction of email addresses from publicly available sources can be controversial and that some may perceive unsolicited emails from any source as spam.

 However, we believe that using such methods for academic purposes that are directly related to the HCPs’ field of interest is less likely to result in complaints from email recipients compared to the use of a general market list. Further, the low unsubscribe rate from our own study is a potential indication that our recruitment efforts were not perceived as spam by most of the screener recipients. Notably, this effort can be a viable way for academics and scholars operating under a limited budget to achieve enrollment goals given that purchased lists tend to be expensive and, at times, controversial as well.

Nevertheless, it is important to contact only those whose expertise is directly related to the given study and therefore may be most likely to benefit from participation. The study team was mindful to use a targeted approach by only recording emails and contacting individuals from Pubmed and 

 who would be most relevant for our study.

Recent survey research by MphaR suggests that utilizing asynchronous discussion forums—defined within this research as the ability for participants to engage with the content (i.e., relevant publications and materials) on their own time and individually consider their comments and feedback in place of or in addition to video interaction—may be a more productive AdBoard medium. In this survey, 50% of HCPs preferred asynchronous discussion only, while 30% preferred asynchronous discussion in addition to video interaction. Ten percent reported a preference to return to face-to-face, and 10% preferred video only for AdBoard formats.

 Although our AdBoard was conducted synchronously, asynchronous discussion may be an advantage for recruitment to overcome one of the most difficult aspects of HCP recruitment: scheduling. Multiple recipients of invitation emails in this study showed interest in participating but stated that they could not allocate 4 hours to the project due to limited bandwidth or schedule conflicts.

An asynchronous discussion eliminates many of these issues, as participation can take place at any time. With a hybrid method that combines synchronous and asynchronous discussion, the synchronous session could be pared down to a much more manageable length (i.e., 1 hour instead of 4) that may be more accommodating to the average HCP. Several studies have been successful in implementing an asynchronous focus group method for HCP qualitative data collection, further supporting the use and validity of this method among this population.

Though using social media for study recruitment has been studied extensively, few studies concern the recruitment of HCPs.

 Additionally, quantifying the effectiveness of social media on recruitment has proved difficult because previous studies have defined and used different metrics to measure success (i.e., views, clicks, or participant enrollment).

 Pizzuti et al. (2020) found that Facebook was the most commonly used social media platform among US-based HCPs compared to seven other platforms (Snapchat, Google+, Pinterest, Twitter, Reddit, LinkedIn, Instagram).

 Yet, Facebook Ads did not always yield high success rates compared to other social media platforms.

 In fact, our review of the literature revealed that the platform’s success often depended on the target population. Thus, when the recruitment strategy involves social media advertisements, the research team should have a thorough understanding of their target audience, particularly which platform(s) their audience most commonly uses (i.e., Facebook vs. TikTok).

As with the type of platform and general effectiveness of social media recruitment, its cost effectiveness varies widely across studies and target populations. According to Whitaker et al. (2017), in a review of 35 global studies between 2012 and 2017, the average cost of Facebook recruitment for health research was $14.41/participant at $0.51 per click.

 In a separate, randomized controlled trial, the reported cost/randomized participant reached $172.76/participant before the researchers discontinued the social media advertisement due to low engagement.

 Studies that were successful in recruiting their target sample reported a low cost overall and improved cost effectiveness

 compared to other strategies such as email and print ads, while those unsuccessful in social media recruitment often canceled their paid social media advertisement once the effort proved unfruitful24 or hit a cost limit,

 making it difficult to estimate an exact cost if social media were to be used until the full sample size was achieved.

Few online sources reported recruitment methodology for focus group studies, with those reporting cost data instead consisting primarily of cross-sectional or randomized controlled clinical trials. Although some studies identified during this literature review reported on recruitment costs of HCPs via social media,

 and none addressed both. That said, given the review of studies reporting the cost of social media as a recruitment method, accuracy rather than cost seems a more significant limitation.

To get around this limitation, researchers could post on professional groups or associations relevant to their target participants. However, as with any cold call method, the response rates may be low. Professional associations receive a large number of requests and are limited in how many research studies they can promote. During the initial phase of our study, for example, five hematology and oncology associations were contacted regarding the possibility of advertising our study, yet none responded. Often, having professional ties to someone in those organizations can be an effective way to recruit niche populations for qualitative studies. For example, Dalessandro et al. (2018) cited difficulties recruiting LGBTQ participants for a qualitative study but managed to ask a contact to share the study on a private Facebook group, which helped the research team to reach their recruitment numbers.

 Another study cited “partner organizations’ social media outlets” as a recruitment tool.

This professional network requirement to research recruitment is a noted barrier, with McRobert et al. (2018) stating that social media recruitment “relies on pre-existence of a diverse and functioning social network.”

Our institution conducts a wide array of research in a variety of fields and topics, making it difficult to have such contacts available, particularly for studies on niche topics. However, as we demonstrated in this study, recruiting HCPs on a specialized topic without the research team having pre-existing professional ties to HCPs in particular healthcare specialties is still possible. It simply requires additional time, effort, and a bit of creativity. Additionally, though initial recruitment using cold emailing methods can be time-consuming, once the relationships between the research organization and the HCPs have been established, they can lead to improved future recruitment opportunities.

Lastly, using a combination of recruiting methods (i.e., cold emailing; word-of-mouth; social media) provides more viable ways to recruit a diverse pool of participants compared to using one method. For example, social media recruitment is associated with a less diverse sample, namely young, female, white, educated, and economically advantaged, compared to traditional recruitment methods.

 Thus, a combination of the different strategies may be ideal to produce a diverse sample that reaches the target sample for the study.

In this study, collecting multiple myeloma HCP emails from publicly available sources helped the research team to find more eligible participants compared to using purchased lists. Additionally, packages such as R’s 

as well as Khalifa’s strategy to scrape emails from Pubmed make it possible to generate contact lists more suited to the needs of researchers at little to no cost to the institution. Scheduling was a major challenge faced by researchers during this study, which could potentially be mitigated by utilizing asynchronous virtual AdBoards to improve the scheduling process and make AdBoard or focus group participation easier for HCPs from all backgrounds. Although social media recruitment has been studied as an alternative recruitment medium, not much is known about recruiting HCPs with this method, and most studies conclude that its usefulness and cost-effectiveness depends on the target population and the professional contacts of the research team and institution. For small, non-profit institutions, particularly those that focus on a range of research topics and target populations, cold emailing using contacts gathered form publicly available sources is one of the most cost-effective and accurate options for the recruitment of HCPs.

, PhD; both with the Tufts Center for the Study of Drug Development, Boston, MA, USA

Michaels, DL, Lamberti, MJ, Pena, Y, et al. Assessing biopharmaceutical company experience with patient centric initiatives. 

2019; 41: 1427-1438. DOI: 10.1016/j.clinthera.2019.07.018.

Eder, M, Evans, E, Funes, M, et al. Defining and measuring community engagement and community-engaged research: CTSA institutional practices. 

Progress in Community Health Partnerships: Research, Education, and Action

 2018; 12(2): 145–156. DOI: 10.1353/cpr.2018.0034.

De Koning, R, Egiz, A, Kotecha, J, et al. Survey fatigue during the COVID-19 pandemic: an analysis of neurosurgery survey response rates. 

Morgantini, LA, Naha, U, Wang, H, et al. Factors contributing to healthcare professional burnout during the COVID-19 pandemic: A rapid turnaround global survey. 

 2020; 15(9): e0238217. DOI: 10.1371/journal.pone.0238217.

Boydell, N, Fergie, G, McDaid, L, et al. Avoiding pitfalls and realising opportunities: reflecting on issues of sampling and recruitment for online focus groups. 

 2014; 303-311. DOI: 10.1177/001112876100700402.

Kenny, AJ. Interaction in cyberspace: An online focus group. 

 2005; 49: 414–422. DOI: 10.1111/j.1365-2648.2004.03305.x.

Archibald, MM, Ambagtsheer, RC, Casey, MG, et al. Using zoom videoconferencing for qualitative data collection: perceptions and experiences of researchers and participants. 

International Journal of Qualitative Methods. 

Denisova, N. Virtual advisory boards: what pharma learned in 2020 and new trends for the Post-COVID Era. 

https://m-phar.com/virtual-advisory-boards-what-pharma-learned-in-2020-and-new-trends-for-the-post-covid-era/

Harrison, JM, Germack, HD, Poghosyan, L, et al. Surveying primary care nurse practitioners: an overview of national sampling frames. 

 2021; 22(1): 6-16. DOI: 10.1177/152715442097608.

Mitchell, EJ, Ahmed, K, Breeman, S, et al. It is unprecedented: trial management during the COVID-19 pandemic and beyond. 

 2020; 21(784). DOI: 10.1186/s13063-020-04711-6.

Khalifa, M. Using PubMed to generate email lists of participants for healthcare survey research: a simple and practical approach. 

Health Informatics Vision: From Data via Information to Knowledge

 2019; 262: 348 – 351. DOI: 10.3233/SHTI190090.

Short, CE, Rebar, AL, and Vandelanotte, C. Do personalised e-mail invitations increase the response rates of breast cancer survivors invited to participate in a web-based behaviour change intervention? A quasi-randomised 2-arm controlled trial. 

 2015; 15:66. DOI: 10.1186/s12874-015-0063-5.

Hennrich, P, Arnold, C, and Wensing, M. Effects of personalized invitation letters on research participation among general practitioners: a randomized trial. 

 2021; 21: 247. DOI: 10.1186/s12874-021-01447-y.

Thomas, B. E-mail address harvesting on PubMed--a call for responsible handling of e-mail addresses. 

 2011; 86(4):362. DOI: 10.4065/mcp.2010.0817.

Kowalewicz, R. Why you should never buy an email list. 

https://www.forbes.com/sites/forbesagencycouncil/2020/12/03/why-you-should-never-buy-an-email-list/?sh=17f308836511

LaForge, K, Gray, M, Stack, E, et al. Using asynchronous online focus groups to capture healthcare professional opinions. 

International Journal of Qualitative Methods

 2022; 21: 1 – 9. DOI: 10.1177/16094069221095658.

Ferrante, JM, Friedman, A, Shaw, EK, et al. Lessons learned designing and using an online discussion forum for care coordinators in primary care. 

 2016; 26(13): 1851-1861. DOI: 10.1177/1049732315609567.

Williams, S, Clausen, MG, Robertson, A, et al. Methodological reflections on the use of asynchronous online focus groups in health research. 

 2012; 11(4): 368-383. DOI: 10.1177/160940691201100405.

McRobert, CJ, Hill, JC, Smale, T, et al. A multi-modal recruitment strategy using social media and internet-mediated methods to recruit a multidisciplinary, international sample of clinicians to an online research study. 

 2018; 13(7):e0200184. DOI: 10.1371/journal.pone.0200184.

Arigo, D, Pagoto, S, Carter-Harris, L, et al. Using social media for health research: Methodological and ethical considerations for recruitment and intervention delivery. 

2018; 4: 2055207618771757. DOI: 10.1177/2055207618771757.

Pizzuti, AG, Patel, KH, McCreary, EK, et al. Healthcare practitioners’ views of social media as an educational resource. 

 2021; 15(2): e0228372. DOI: 10.1371/journal.pone.0228372.

Topolovec-Vranic, J, Natarajan, K. The use of social media in recruitment for medical research studies: a scoping review. 

 2016; 18(11):e286. DOI: 10.2196/jmir.5698.

Whitaker C, Stevelink S, and Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review.

 2017; 19(8):e290. DOI: 10.2196/jmir.7071.

Heffner, JL, Wyszynski, CM, Comstock, B, et al. Overcoming recruitment challenges of web-based interventions for tobacco use: the case of web-based acceptance and commitment therapy for smoking cessation. 

 2013; 38 (10): 2473-2476. DOI: 10.1016/j.addbeh.2013.05.004.

Geist, R, Militello, M, Albrecht, JM, et al. Social media and clinical research in dermatology. 

 2021; 10(4): 105-111. DOI: 10.1007/s13671-021-00350-5.

Kapp, JM. Peters, C, and Oliver, DP. Research recruitment using Facebook advertising: big potential, big challenges. 

 2013; 28(1):134-7. DOI: 10.1007/s13187-012-0443-z.

Khatri, C, Chapman, SJ, Glasbey, J, et al. Social media and internet driven study recruitment: evaluating a new model for promoting collaborator engagement and participation. 

 2015; 10(3): e0118899. DOI: 10.1371/journal.pone.0118899.

Townsend, A, Leese, J, Adam, P, et al. eHealth, participatory medicine, and ethical care: a focus group study of patients’ and health care providers’ use of health-related internet information. 

 2015; 17(6):e155. DOI: 10.2196/jmir.3792.

Dalessandro, C. Recruitment tools for reaching millennials: the digital difference. 

 2018; 17(1): 1 - 17. DOI: 10.1177/1609406918774446.

Benedict, C, Hahn AL, Diefenbach, MA, et al. Recruitment via social media: advantages and potential biases. 

2019; 5:2055207619867223. DOI: 10.1177/2055207619867223.

Articles

Tufts CSDD study addresses barriers in recruitment. 

Methods to Recruit Healthcare Providers for Virtual Advisory Boards in Drug Development

Women in drug development continue to be underrepresented in leadership roles. Although various reasons have been given for this disparity, one possible explanation concerns gender-based microaggressions experienced by women in the workplace. To investigate this proposed cause of gender disparities in the drug development workforce, Tufts Center for the Study of Drug Development conducted 26 interviews among a diverse group of women professionals in drug development. These interviews focused on participants’ experience with microaggressions in the workplace and the effect of these microaggressions on teamwork and communication. The research team found that gender-based microaggressions in the drug development workplace can negatively impact collaboration among teams and can affect retention of women employees.

Drug development is among the many areas within Science, Technology, Engineering, and Math (STEM) where women professionals continue to be underrepresented. Although the gender diversity in the drug development industry has improved over the years, women, and particularly women of color, are still noticeably underrepresented in leadership roles. Only 30% of all C-suite positions are filled by women, with a mere 4% of this group representing women of color.

 There have been many reasons put forth to explain this gap, including perceived role incongruity between women and male-typed jobs,

 and deeply entrenched systemic barriers.

 Another contributing factor to this gap are gender microaggressions, defined as subtle behaviors and comments that most often unintentionally demean or denigrate individuals based on their gender.

 Though subtle, microaggressions are ubiquitous, occurring daily in the lives of women in the workplace. Though some argue that microaggressions are harmless, amounting to nothing more than minor insults,

 the emerging research on workplace gender microaggressions has demonstrated their deleterious effects on both cognitive and affective outcomes on women professionals.

Naturally, we wondered how gender microaggressions might manifest in drug development and what impact they might have on women professionals navigating this industry. To shed light on this topic, Tufts Center for the Study of Drug Development recently conducted a qualitative study to examine how gender microaggressions manifest among different work teams in drug development. We interviewed a total of 26 women across different areas, including sponsor organizations, contract research organizations (CROs), vendors, and academic research centers, representing a variety of different functions, levels, and work experience. Importantly, our sample was also racially diverse: 50% were women of color representing Asian, Black, Latina and multi-racial women, allowing us to capture nuanced ways gender microaggressions manifest for women of color.

Of the different types of microaggressions that we uncovered, devaluation emerged as a common form of slight that women encountered on a regular basis. Devaluation describes behaviors or comments that put down or diminish a woman’s input or contribution, the most common example being interrupting women while they speak or misattributing their work to a male colleague. These experiences were shared unanimously across the diverse sample of women we interviewed.

Our study’s racially diverse sample provided important nuance to how gender microaggressions manifest for women of color. Women of color not only had to contend with the usual forms of gender microaggressions, such as devaluation and misattribution of work, but also had to navigate an added layer of microaggressions that intersected their gender and racial identities. These included unsolicited comments that otherized different aspects of their racial and ethnic identity (i.e., colleagues repeatedly mispronouncing their names and being mistaken for another woman of color). Important distinctions also emerged within our racially diverse sample. Black women shared how they felt otherized by belabored comments from colleagues about their hair that made them feel uncomfortably spotlighted. Assumptions of incompetence and lack of intelligence for Black and Latina women also emerged. Conversely, Asian women were assumed to be overly competent, which was often accompanied by the assumption that they do not need resources or help or that they are naturally experts in data analysis even if they were not.

The potential impact of gender microaggressions on team productivity

Regularly experiencing these types of slights were linked with cognitive depletion, emotional exhaustion, and behavioral changes, including overcompensation as well as withdrawal. Importantly, constantly navigating these dynamics was also associated with lowered productivity and team performance, stemming largely from lowered trust towards and a tendency to limit one’s interactions with the aggressor. Naturally, this can pose a challenge to team productivity and effectiveness, which we detail below.

Lowered trust towards one’s colleague can lead to decreased information sharing and communication, which are hallmarks of effective teams.

 This is particularly worrisome within the context of drug development given the collaborative nature of many cross-functional teams whose success depends on effective communication and idea sharing.

Moreover, several of the behaviors that women encountered, such as interruptions or misattribution of their work were directly linked to lowered productivity. In one example, a project manager (PM) described a situation in which the Principal Investigator (PI) would constantly email her male colleague about her project. Repeated incidents such as this created redundancy and delays in the project, all of which could have been avoided had the PI reached out to the correct PM, in this case, the woman.

Such interactions that we have described so far may seem minor at first blush. However, when experienced on a regular basis at work among one’s colleagues, the psychological and emotional toll can be taxing. In fact, the growing research on workplace microaggressions has linked these interactions to emotional and cognitive exhaustion as women manage their emotions while they decipher the meaning and intent behind many of these often-unintentional slights that are committed unbeknownst by the aggressor.

 Though unintentional, these micro interactions have the potential to have macro effects if left unchecked. In fact, many of the women in our sample talked about their intent to leave the organization, and many of them had, at the time of the interview, recently left what they viewed as toxic and exclusionary teams and organizations. Losing talent is the ultimate cost to teams and organizations as it can result in a “brain drain” of diverse, high potential talent.

What then can organizations do to address these all-too-common forms of team dynamics? The solution is creating an inclusive organizational culture where individuals, regardless of their background, feel welcomed and have the potential to develop and grow. However, creating such an environment is easier said than done. While many organizations have heeded the calls for more inclusive workplaces by organizing Diversity, Equity, and Inclusion (DEI) panels, talks, and workshops, etc. such efforts will fall short if they are not accompanied by an intentional and systematic way of benchmarking progress. After all, how does one know there is a problem when one does not measure it? How does one know there is improvement if there are no metrics to track over time? While tracking personnel data, such as turnover statistics, is vital, such statistics would be well paired with antecedent indicators that can help predict and prevent the loss of diverse talent.

We propose measuring the occurrence of these dynamics within organizations and teams to identify areas that need resources related to training and dialoguing. We believe that microaggressions are best addressed through conversation and perspective-taking. The ability to measure and track these dynamics can serve organizations well as they can “get ahead of the problem.”

Cross collaboration is increasingly necessary for producing high quality science.

 Microaggressions, however, can hinder team effectiveness by stifling communication, trust, and cohesion among teams. Further, these dynamics can signal exclusion, ultimately contributing qualified women to leave such organizations or quit the industry altogether in search of more inclusive environments. Thus, the ability to identify, measure, and benchmark these dynamics can help organizations take a more proactive approach to ensure that they are creating an inclusive environment where all individuals, regardless of their social identity, can thrive.

Corresponding Author, Tufts Center for the Study of Drug Development, Tufts University School of Medicine, 145 Harrison Avenue, Boston, MA 02111. 

This study was conducted through a research grant from PhRMA.

Sandle T. Getting there: Strategies to improve gender equality in pharmaceuticals and healthcare. 

. Published online 2021. doi:10.1080/21507090.2732021ts-gtsige

Berlin G, Darino L, Greenfield M, Starikova I. 

Eagly AH, Karau SJ. Role congruity theory of prejudice toward female leaders. 

. 2002;109(3):573-598. doi:10.1037/0033-295X.109.3.573

McCord MA, Joseph DL, Dhanani LY, Beus JM. A meta-analysis of sex and race differences in perceived workplace mistreatment. 

. 2017;103(2):137-163. doi:10.1037/apl0000250

Hall WM, Schmader T, Aday A, Inness M, Croft E. Climate control: The relationship between social identity threat and cues to an identity-safe culture. 

. 2018;115(3):446-467. doi:10.1037/pspi0000137

Capodilupo CM, Nadal KL, Corman L, Hamit S, Lyons OB, Weinberg A. The manifestation of gender microaggression. In: Sue DW, ed. 

Microaggressions and Marginality: Manifestation, Dynamics, and Impact

Lilienfeld SO. Microaggressions: Strong Claims, Inadequate Evidence. 

. 2017;12(1):138-169. doi:10.1177/1745691616659391

Meister A, Sinclair A, Jehn KA. Identities under scrutiny: How women leaders navigate feeling misidentified at work. 

. 2017;28(5):672-690. doi:10.1016/j.leaqua.2017.01.009

Kim JY, Meister AL. Disrupting the effects of gender microaggressions for women leaders: Allies as triggers of redemptive sense-making. In: 

. ; 2021. doi:https://doi.org/10.5465/AMBPP.2021.281

De Jong BA, Dirks KT, Gillespie N. Trust and team performance: A meta-analysis of main effects, moderators, and covariates. 

. 2016;101(8):1134. doi:https://doi.org/10.1037/apl0000110

Holder AMB, Jackson MA, Ponterotto JG. Racial microaggression experiences and coping strategies of black women in corporate leadership. 

. 2015;2(2):164-180. doi:10.1037/qup0000024

Wernick B, Wojda T, Wallner A, et al. Brain drain in academic medicine: Dealing with personnel departures and loss of talent. 

. 2016;2(1):68. doi:10.4103/2455-5568.183332

Reardon S. Scientific collaborations are precarious territory for women. 

. 2022;605:179-181. doi:10.1038/d41586-022-01204-1

26 interviews conducted by Tufts CSDD found that gender-based microaggressions can negatively impact collaboration and affect retention of women employees.

The Impact of Gender Microaggressions on Team Performance in Drug Development

During the past decade, there has been unprecedented focus on patient-centric initiatives in support of drug development planning and execution. The original intent of these initiatives was to amplify patient input, address patient needs and preferences, and ensure that patients have the best possible experience participating in clinical trials. As investment and experience with patient centric initiatives has evolved, a growing number of organizations have looked to measure the impact of patient-centric activities on a variety of clinical trial outcomes includingspeed and quality.

 Although these outcome measures inform project management, resource allocation and return-on-investment decisions, they are ‘trial-centric’; they are not patient centric.

Key indicators of patient experience—most notably satisfaction—are largely missing from the outcome measures that organizations are systematically gathering. Such routine assessment of patient satisfaction is essential to measuring the successful implementation of patient-centric initiatives and to informing continuous improvement of these initiatives.

The purpose of this article is to remind clinical research stakeholders to re-prioritize routine assessment of patient satisfaction as a primary outcome measure and to inform how to approach that assessment.

Patient satisfaction is not a new concept. It has been covered extensively in the medical science literature. Studies have consistently shown that there exists a positive link between patient satisfaction and a variety of patient-oriented clinical care outcomes, including in-patient mortality, treatment adherence, and improved rapport with care providers.

 Given this compelling and consistent association, it is only fitting to consistently measure and evaluate patient satisfaction.

Yet, based on a recent literature review of patient satisfaction in clinical research specifically, only a small fraction of studies (nine out of several thousand) reported systematically evaluating patient experiences during a clinical trial.

 Further, of the nine studies that reported measuring patient satisfaction, only one study used a widely accepted and validated patient satisfaction instrument (Modified Client Satisfaction Questionnaire).

 The remainder used ad hoc, investigator-developed patient satisfaction instruments.

These findings were recently corroborated at a meeting among patient engagement stakeholders from sponsor companies and contract research organizations (CRO) where the majority of companies conceded that they are not consistently measuring patient satisfaction in their clinical trials. The inconsistency and sparseness in how patient satisfaction is measured should not come as a surprise given the limited number of patient satisfaction scales developed exclusively for clinical research.

There is clearly little to no consensus on what is the most appropriate instrument for measuring patient satisfaction. This by no means suggests that organizations should be satisfied with the current state of affairs. Rather, lack of a consensus measurement-method signifies an opportunity to test out and validate different approaches for capturing patient experience in clinical research. To begin, there already exists a recognized and validated non-clinical research patient satisfaction instrument, the 8-item Modified Client Satisfaction Questionnaire. This instrument is commonly used in many different industries and has been adopted in the medical science and healthcare literature.

This measure could provide an appropriate starting point from which organizations can expand and refine (i.e., adding additional questions about different aspects of the clinical research study).

Reminding clinical research stakeholders to reprioritize routine assessment of patient satisfaction as a primary outcome measure in patient-centric activities.

Jennifer Kim

Articles

Methods to Recruit Healthcare Providers for Virtual Advisory Boards in Drug Development

The Impact of Gender Microaggressions on Team Performance in Drug Development

Measuring Patient Satisfaction as a Primary Outcome for Patient-Centric Initiatives