Jim Kremidas

Site perspectives of clinical trial quality are rarely heard, even though they are the foundation of the clinical research enterprise. But sites are not alone in conducting trials. They depend on sponsors and contract research organizations (CROs) to provide services and resources that allow them to execute a trial. The critical question from the site’s perspective then becomes: What are the essential resources and services that sites need from sponsors and CROs in order to conduct high-quality trials? The purpose of this paper is to identify such services and resources that are associated with high-quality clinical trials. 

	This study focuses on three research questions (RQs):

 What are the coproduction activities that sponsors and CROs provide to sites that lead to high-quality trials?

 How do sites evaluate sponsors and CROs on these key drivers of quality? 

How do each of these coproduction activities drive clinical trial quality? 

	To answer these questions, we conducted focus groups to identify all the activities that are important to sites (RQ1). To evaluate RQ2 and RQ3, we loaded all of these activities, as well as a measure of clinical trial quality, into an online survey. We then solicited Association of Clinical Research Professionals (ACRP) members to evaluate a recent trial as to the quality of the study and then how CROs and sponsors were performing on these key drivers of clinical trial quality. Finally, we created a statistical model to give the appropriate weight to each of these quality drivers.

	Trials in which the site had a direct relationship with the sponsor were perceived as being higher quality compared to when the site relied on a CRO. The most critical driver of trial quality from the sites’ perspective was communication-being available for questions, timely responses, and being helpful in resolving problems. While sponsors and CROs are doing reasonable well in this area, improvements in communication will yield the greatest benefits to improving clinical trial quality. Other significant drivers of quality from the sites perspective included the quality of the protocol, budgeting processes, technology, and monitor performance. 

	The results of this study illustrate a pathway by which sites, CROs, and sponsors can improve the quality of clinical studies. These findings can also provide a starting point for achieving an ROI on quality investments. Perhaps, most importantly, the results of this study give clinical trial sites a greater opportunity to provide insights into quality improvement methods for the clinical research enterprise.

	Clinical trial sites are the underappreciated foundation of the medical research enterprise.

 Despite this importance, sites struggle to execute studies effectively and efficiently.

 But sites depend on sponsors and CROs to support them by providing services and resources that allow them to execute the study plan. This lack of support can contribute to trial delays, increased costs, and considerable site turnover.

 As a result, investigative sites are becoming a scarce resource that limits the ability to conduct clinical research.

	The purpose of this paper is to address this situation by identifying the things that sponsors and CROs do that allow sites to execute high-quality clinical trials. Clinical studies depend on multiple stakeholders-often broken into separate organizations-working together effectively to coproduce a high-quality research program. Coproduction refers to the phenomenon by which multiple actors must apply their knowledge, skills, services, and resources to cocreate a complex service.

 While the site may interact with the patient, delivering a high-quality clinical trial depends on sponsors, CROs, and sites working collaboratively.

	But there are potentially thousands of coproduction activities that could impact trial quality. What are the specific drivers that lead to higher quality clinical studies? We distill all of these issues into the three RQs mentioned. 

RQ1: What are the coproduction activities that sponsors and CROs provide to sites that lead to high quality trials?

	To examine RQ1, we conducted two focus groups of 12 to 15 study coordinators each at the 2015 ACRP national conference in Salt Lake City, Utah. We had no difficulties recruiting potential subjects. Clinical research professionals were anxious to participate. Several of the participants told us that they were delighted they could contribute “so our voices could finally be heard.”

	Our approach to these discussions was inductive. The participants all had about 10 years or more of trial experience, so their observations were naturally granular. Each of the sessions was opened by asking, “What are the things (or activities) that sponsors/CROs do that help you execute a high quality study?” This led to a long list of performance activities. The process was iterative as participants clarified and built on each other’s observations. Once we had collated this list of performance activities, we worked with the focus groups to organize all of them into the following eight distinct groupings:

	We tested these groupings with focus group members by challenging, for example, whether budgeting and reimbursement were really different groups.

 Once we confirmed that these were distinct groups and aligned the items within each one, we arranged the participants into subgroups of about three people each to organize the items. At the end of the session, each group presented their refinements and the larger group commented on their work.

	Representative items-framed in survey format-for each category are shown in Table 1 (click to enlarge). We also included a set of questions to assess the quality of the clinical trial from the site’s perspective, which served as the dependent variable, shown at the bottom of Table 1 and derived from the SERVQUAL measure.

 All of these items were edited for clarity, combined with similar items when appropriate, and loaded into the CRO Analytics’ Performer platform in preparation for a survey.

RQ2: How do sites evaluate sponsors and CROs on these key drivers of quality? 

	We solicited responses from the ACRP membership, excluding those members who participated in the focus groups. Respondents logged on to the Performer tool. They were instructed to think about a Phase II or III trial that completed recently and then to evaluate the items with that study in mind. We also asked them about characteristics of the trial and demographic items.

	We received a total of 278 responses from experienced research professionals. The respondents had an average of 10.7 years’ experience. About 94% of respondents characterized themselves as clinical trial coordinators or clinical research managers, with 3% describing themselves as principal investigators and 3% using a variety of other job descriptions (e.g., consultant). The trials involved various therapeutic areas. Cardiology only made up 8% and oncology 3% of the trials. Most of the studies were Phase III (75%); the rest (25%) were Phase II. The average trial consisted of 70 subjects (sd= 495), with a long right tail. Eighty percent of the studies in this sample met their enrollment goals, higher than typically reported.

	How did sites evaluate the overall quality and performance in the coproduction activities? The average overall perceived quality of the clinical trials was 7.1 (sd= 2.17) on a scale of 1 to 10 (1 = low quality; 10 = high quality). This average is high compared to similar studies we have conducted on clinical trial quality. Figure 1 below illustrates the overall quality ratings with the ratings for each of the performance areas. This graph is constructed so that quality is the first column on the left and then performance areas are ordered from least to greatest (left to right). Each of the performance areas demonstrated discriminant validity from the others, meaning that we demonstrated statistically that budgeting was distinct from reimbursement and all the other groups.

	It is not surprising, given our focus group discussions, that budgeting, reimbursement, and monitors were the lower-ranking performance areas and that communication (µ= 7.2) and protocols (µ= 7.7) were rated more highly. Given these averages, we next sought to identify the amount of variation with each of these averages. Are these ratings consistent, or is there wide variation around this average? Greater variation means that there is less consistency or agreement about the quality and performance ratings. We screened for variation by looking at the standard deviations. A high standard deviation would indicate significant variation in the responses around the average. Figure 2 below illustrates the findings on variation and standard deviations. Items with the highest standard deviation scores have the highest amount of variation (or less consensus) around that average (i.e., they are likely to have subgroups of very low or very high scores). Items with the lower standard deviation scores have less variation (greater consensus). Communication (µ= 7.2) has a relatively high rating, but there was also a lot of underlying variation (sd= 2.53) around that rating. This suggests that while many sites are rating communication very high, there is a lack of consensus about this performance metric.

	The findings regarding the protocols were more complex. In examining the means and variation of the ratings of the protocol sub-drivers, we found that some of the important ones-complexity of the protocols and the frequency of amendments-had low ratings with high variation, or a lack of consensus. This is a paradoxical finding: even though the sites rated the overall quality of the protocols highly, they thought they were exceedingly complex, with excessive amendments. We also found that the correlations between overall protocol quality to the complexity (r= .05) and amendments (r= .26) were very low. 

	There is a disconnect between how sites evaluate the quality of a protocol and its complexity and amendments. We interpret these findings as an example of lowered expectations. It may be that sites have simply accepted the complexity of protocols and frequent amendments as a norm, so it does not impact their ratings of the overall quality of the protocol (see Figure 3).

	Finally, we found that sponsor companies outperform CROs in overall quality and many performance areas, as shown in Figure 4 below. Labels that have a single asterisk (*) were found to be statistically significant at p= .05 in a multivariate ANOVA (analysis of variance) and the labels with a double asterisk (**) were significant at p< .001. We see two possible explanations here based on our focus group discussions. It may be that CROs are conducting a wide variety of studies with many different sites, thus an individual site does not feel like it is getting the support or attention from the CRO. It may also be that the CRO can be perceived as a “middle-man” that blocks or delays the site from the getting information or support it needs to execute the study. Further research is needed on this question.

RQ3: How do each of these coproduction activities drive clinical trial quality?

	To this point, we have assumed so far that each of the performance drivers all have an equal impact on quality. To drill down further and weigh the relationships between overall quality and the eight performance areas, we created a multivariate linear regression model. The sites’ perception of the quality of the trial was the dependent variable and the performance areas were the independent variables. We also included a set of 

covariates-variables that might influence the relationship between trial quality and performance-in the model. The covariates were the size of the trial (number of subjects); whether it met its enrollment goals and endpoints; the complexity of the protocol; the appropriateness of protocol amendments; whether the site’s contract was with a sponsor or CRO; and the length of the experience in clinical trials of the respondent.

	The regression model explained a statistically significant (F(15,244)= 48.5, p< .001) amount of variance (R2= .75) in clinical trial quality. The model met all of the regression assumptions and variable were centered to enhance interpretation. The coefficients with their statistical significance tests are shown in Table 2 (click to enlarge).

	 We compare the magnitude of the coefficients graphically in Figure 5 below. In interpreting the coefficients, each of the variables was mean-centered before running the model. Each coefficient gives an estimate of the unique impact of that variable on trial quality at the mean of all the other variables. Also, the variables for total subjects and years’ experience were skewed to the right and, thus, were log-transformed.

	The impact of the performance drivers on clinical trial quality fall into three distinct layers. In the top, most impactful layer, communication (b= .29, p< .001) and protocol quality (b= .24, p< .001) had the single greatest effects on quality in this study. Communication in common usage is usually a very broad term. The results from our focus groups offer drill-down insights. In the focus groups, sites mean very specific things when they speak of communication. When a study team is communicating well, that means that they provide information in a timely fashion; are available for questions as they arise; and are available and helpful in resolving problems. The impact of protocol quality is particularly impressive because we included protocol complexity and amendments as covariates in the model. Protocol quality has an impact on trial quality above and beyond complexity and amendments.

	In the middle tier of performance drivers, budgeting (b= .16, p< .001) and technology (b= .15, p< .001) had statistically significant and positive-although more moderate-impacts on trial quality. Budgeting refers to the process of establishing the resources for the site’s services. Within this category, fairness was the dominant theme. The recurrent theme that we heard in the focus groups was that site coordinators simply wanted to be paid for the requested work. They become very frustrated when the site must absorb the cost of adjusting to protocol amendments, training new monitors, handling serious adverse events (SAEs), or just budgeting extra time for the coordinator to do the work demanded by the protocol. Notice that this budgeting driver is distinct from the actual reimbursement, which was not a significant driver.

	Within the technology areas, sites often struggle with multiple clunky software systems that are not integrated. On average, attendees at the focus groups had to separately log into six or seven technologies for each trial. Sites want seamless, integrated trial technologies and improving technology will have significant and substantial effects on perceived clinical trial quality. Improving technology will have a significant, though moderate, impact on trial quality.

	The performance of the monitors (b= .10, p= .02) had a slightly less substantial, but still significant impact on clinical trial quality. Sites described in the focus groups that they want monitors who can serve as a resource to help them execute trials better and more efficiently. Inexperienced monitors-that the site must train-who don’t understand the protocol and are disruptive to site operations are an ongoing frustration for sites.

	Site initiation (b= .06, p= ns), closeout (b= .05, p= ns), and reimbursement (p= -.05, p= ns) were not significant performance drivers of quality in this study. Even though closeout has a negative coefficient, it is not statistically different from zero. In understanding these results, remember that this model estimates the unique effects of each performance driver, exclusive of all the other performance drivers. So if it seems like initiation (or closeout or reimbursement) should have been significant, remember that we are estimating the isolated effects of initiation on quality, excluding the effects of communication, the protocol, budgeting, technology, and monitors.

	Sites are the foundation of the clinical research enterprise, but they have surprisingly little input in the development or planning of trials. The purpose of this research was to provide sites the ability to give their perspective of how we can improve the quality of clinical trials. The recurrent theme that we find in both the focus groups and statistical analysis is that sites are looking for partners who can help them serve their patients and conduct excellent science. Their common experience is that they often struggle to be treated fairly. 

	The results of this research may not surprise those who work with sites. This data has been anecdotally available for some time. The real contribution of this study is not only in the systematic collection of all of the performance drivers, but, more importantly, in giving weights to each of these performance drivers. In doing so, we are able to think about an ROI on site relations. 

	Suppose you are a sponsor or a CRO that wants (or needs) to improve site relations. The results of this study provide a way for sponsors and CROs to assess where they stand compared to the rest of the industry. Companies can send out a survey and compare their results to the data illustrated in this report. Based on the 

results of their survey, companies can think about the ROI on investing to improve their scores using the results of the regression model. 

	Suppose a company’s survey shows that it rated a 6.5 on a 1-to-10 scale for communication and a 7.0 on overall quality. Congratulations-not a bad score. But if one looks at our data, they will see that our sample rated an average of 7.2 for communication. This means that the organization would have room to improve in this area. Is it worth investing to improve communication? To assess this question, examine the results of the regression. Communication had a coefficient of .29. That means that if a sponsor or CRO invests in communication and is able to raise its communication rating from 6.5 to 7.5, it would improve its quality score from 7.0 to 7.29. What specifically does one look at to improve communication? Examples of the sub-drivers are found in Table 1.

	Now imagine that a company’s survey shows a 5.5 for closeout and it is compared to our findings of 7.4. The rating gap is fairly significant. Should a company, therefore, invest to improve its closeout ratings? The results of our regression model would suggest no. The closeout driver had no significant effect on quality, so the results of this study would suggest an organization’s investment would be wasted. The firm certainly might make some non-monetary investments using guidance from Table 1, but it would not improve quality by investing heavily in this area. 

	Several limitations should be kept in mind in evaluating the results of these studies. First, some might argue that the ACRP membership is not a representative sampling. Someone who belongs to a professional organization like ACRP, attends the national conference, and even volunteers for a focus group is likely to be more engaged in their professional work that the average site coordinator. While this is possible, we would also point out that these are also likely to be key opinion leaders within the profession-the very group that sponsors and CROs need to reach in order to improve clinical trial quality.

	We would also note that the model estimates are general across a variety of therapeutic areas and phases of clinical trials. Although we would argue that these estimates would generalize across clinical studies, estimates may vary around our means within specialty areas.

PA-C, PhD, is Clinical Professor, LeBow College of Business, Drexel University, email: 

 is Executive Director, Association of Clinical Research Professionals (ACRP), email: 

* Acknowledgement: The authors wish to thank Jenna Rouse, Director of Business Development, ACRP, for her assistance in conducting this study.

	1. Califf, Robert M. “Clinical Research Sites - the Underappreciated Component of the Clinical Research System.”

 Journal of the American Medical Association

	2. Rhymaun, Farouk, “Sponsor - Investigator Relationships: A Crisis of Trust.” 

; published September 23, 2014. Available online at 

http://www.appliedclinicaltrialsonline.com/sponsor-investigator-relationships-crisis-trust-life-sciences

	3. Getz, Kenneth A. “Lifting Up a Fragmented Study Conduct Landscape.” 

	4. Scholars recognize coproduction as being pervasive across all service industries. For a fuller explanation, see Zeithaml, Valerie, Mary Jo Bitner, and Dwayne Gremler,

	5. We later statistically confirmed the discriminant validity of these groupings.

	6. Parasuraman, Anantharanthan, Valarie A. Zeithaml, and Leonard L. Berry. “Servqual: A Multiple - Item Scale for Measuring Consumer Perceptions,” 

	7. Parasuraman, Anantharanthan, Leonard L. Berry, and Valarie A. Zeithaml. “Refinement and Reassessment of the SERVQUAL Scale.” 

	8. Getz, Kenneth A, “Enrollment Performance: Weighing the “Facts,” 

http://www.appliedclinicaltrialsonline.com/enrollment-performance-weighing-facts

Articles

A model to identify the services and resources sites need to conduct high-quality clinical trials.


Site Perspectives on Clinical Trial Quality

Most clinical research professionals now acknowledge that there is a critical need for general, broadly based outreach and education to assist in reversing the erosion of public trust. But this need is being underserved by the realities of today's research environment. 

PHOTOGRAPHY: GETTY IMAGES    ILLUSTRATION: PAUL A. BELCI 

Facing finite resources and increasing difficulty recruiting study volunteers, sponsors, CROs, and investigative sites must address the intense and immediate pressure to fill their clinical trials. As such, communication with the public has focused almost entirely on attracting volunteers to a specific trial and not on improving the public's overall literacy in clinical research. Public outreach and education does not appear to have an attractive short-term return on investment. Until now. 

The results of a recent pilot test conducted by Eli Lilly and Company show that The Center for Information & Study on Clinical Research Participation's (CISCRP) new public education program, "Everyday Heroes," can boost monthly enrollment rates substantially. Lilly found that running CISCRP's rebranding initiative concurrently with its subject recruitment advertisements for specific clinical trials generated more than a 35% improvement in monthly enrollees. This surprising finding suggests that public outreach and education, in conjunction with trial-specific outreach, may provide short-term benefits to accelerating subject recruitment activity for study sponsors. 

This article is a sequel to our April 2007 

 article entitled "Mission Possible: Rebranding Clinical Research" and provides a discussion of this ground-breaking pilot study and its results and implication. As background, this article begins with a summary of the process undertaken to develop CISCRP's public education campaign. 

Large-scale public education campaigns have proven to be a highly effective strategy for changing public perception and behavior in other sectors of the economy, such as blood donation, substance abuse, and save the environment movements. CISCRP created the Everyday Heroes Campaign to encourage the public to think differently about clinical research and to recognize the act of participation as a profound gift and a public good. 

With an educational grant from Eli Lilly and pro bono collaborative support from Fast4wD Ogilvy—part of the health division of international advertising and marketing agency Ogilvy & Mather—CISCRP has spent the past two years developing and consumer testing a public education campaign. 

Table 1. The outcome for Eli Lilly when it paired the Everyday Heroes campaign with recruitment ads for two pain studies. 

In the fall of 2005, CISCRP, Lilly, and Fast4wD Ogilvy convened an initial meeting in Washington, DC, among a small, diverse group to discuss the difficulties of subject recruitment and retention and how to improve public education. That initial meeting brought together about two dozen people representing patient advocacy groups, the U.S. Food and Drug Administration, the Department of Health and Human Services, the National Institutes of Health, PhRMA (the Pharmaceutical Research and Manufacturers of America), the National Health Council, academic institutions, pharmaceutical companies, clinical research organizations, investigative sites, subject recruitment service companies, and the IRB community. The group reviewed extensive market research regarding why people choose to participate in clinical research studies. A brainstorming session was conducted to develop creative ideas to raise awareness and educate the public about volunteering for clinical trials. 

At the conclusion of the initial small group meeting, Fast4wD Ogilvy reviewed attendee input and ideas and developed 15 different campaign ideas called "treatments." These treatments cut across a broad spectrum of educational approaches, some serious, some shocking, and some humorous to brand the impact that participation has on public health. One campaign featured ordinary people wearing superhero capes with emphasis on their contribution through participation. Another showed remedies applied to common medical ailments hundreds of years ago in order to capture how far medical knowledge has advanced. 

One of several medical heroes featured in CISCRPs new public service ads for television. (PHOTOGRAPHY: CISCRP) 

In early 2006, another meeting was held among nearly 60 research professionals. Again, this group was comprised of representatives from a broad spectrum of stakeholders in the clinical research enterprise. A major goal of this meeting was to solicit buy-in and support for the campaign moving forward. The group spent a full day reviewing, discussing, and modifying the 15 treatments. At the end of the meeting, the group selected its choice of the top five campaigns. After incorporating suggestions and refining the campaigns to make them look polished and professional, the five treatments were presented in March 2006 to consumer focus groups. 

The focus groups included a representative mix of ages, genders, ethnicities, lifestyles, and occupations. Participants were grouped into one of three socio-economic classes. The Everyday Heroes Campaign scored significantly higher than the rest across the board. Consumers liked the concept and felt appreciative of the people featured in the campaign. Among the lower socio-economic group in particular, respondents identified with and admired the featured heroes engaged in everyday activities. The campaign also prompted all focus group participants to want to learn more and to acknowledge the gift that clinical trial participation gives to advancing medical knowledge. 

In early 2007, following the focus group responses, Fast4wD Ogilvy and CISCRP refined the Heroes Campaign and developed a series of printed images and radio versions of the campaign. These images and radio spots can be reviewed on the CISCRP web site at 

. A 60 second television spot, completed in early 2008, can also be viewed on the CISCRP Web site. 

It's important to remember that the Everyday Heroes Campaign is disease neutral, as it does not focus on any specific illness, and it does not promote any specific clinical research program. The purpose of this educational campaign is not to recruit volunteers into studies currently enrolling volunteers or to promote newly approved treatments. Its purpose is simple: to raise awareness of and appreciation for those who participate in clinical research. 

In June 2007, Lilly decided to pair the Everyday Heroes public outreach campaign with its efforts to recruit subjects for two pain studies—one focusing on back pain, the other on knee pain. These clinical studies involved approximately 30 investigative sites in 18 different geographic locations across the United States. 

Over a three month period, Lilly ran its subject recruitment advertisements alone in 12 markets. And in six markets it ran the Everyday Heroes Campaign on the radio—in 30-second and 60-second formats—concurrently with its mass media (e.g., radio and television) subject recruitment ads. The public education campaign was never run immediately before or after a specific recruitment ad. Rather, it was presented independently, airing at different times of the day, but targeting the same demographic populations as the subject recruitment ads. 

Across all 18 markets, when only a radio subject recruitment ad was used, investigative sites recruited an average of 2.9 subjects per month. During Wave 1 of the pilot test, however, in the six markets where recruitment ads and the Everyday Heroes Campaign aired on the radio concurrently, investigative sites enrolled an average of 4.0 subjects per month—a 38% increase. During this period, in the 12 markets receiving only a subject recruitment ad over the radio, sites continued to enroll an average of 2.9 subjects per month. 

In the second month of the pilot test, Lilly dropped both the subject recruitment ads and the Everyday Heroes Campaign. During this period, enrollment rates in all 18 markets dropped significantly, as expected, to low and comparable base levels. 

Then, during Wave 2 of the pilot test, Lilly observed a most dramatic effect. In this wave, Lilly ran television and radio subject recruitment ads in 12 markets. Investigative sites reported that their recruitment rates jumped to an average of 4.0 enrollments per month. But in the six markets where the television and radio recruitment ads were run concurrently with the Everyday Heroes Campaign, investigative sites saw an average of 9.6 enrollments per month: more than double the rate. Whereas enrollment rates for both cohorts increased during the subsequent wave, the cohort that included education and outreach saw a substantially higher response, suggesting a lasting impact, in part, from general awareness-building communication. 

The pilot test clearly has its limitations. It focused on a single therapeutic area and used only a radio version of the Everyday Heroes program. Additional testing on the impact of this educational branding initiative across a variety of therapeutic areas and patient populations is needed to enrich our understanding. Still, the results of this initial pilot test are promising and intriguing. Following its pilot experience, Lilly is now using the Everyday Heroes Campaign across its portfolio of studies that are actively using media advertising to recruit volunteers. 

At the present time, CISCRP has produced one television spot and two radio spots of the Everyday Heroes Campaign, as well as printed posters to hang onsite at investigative sites and health care provider offices. Print versions of the campaign are also available for local newspapers, magazines, and flyers. CISCRP is in the process of producing additional television and radio Everyday Heroes public educational spots. 

The results of this pilot study offer insight into the power of a well-developed public education initiative running concurrently with clinical trial specific subject recruitment advertisements. Here are a few key insights:  

Patients seeking clinical trials see a connection with the very public message of the Everyday Heroes Campaign.  The pilot test suggests the possibility that the more visible the public outreach campaign, the better the enrollment rates will be for clinical trials in general. Without overdoing it, broad and regular exposure of the Everyday Heroes Campaign benefits the whole clinical research enterprise and, ultimately, public health.

The Everyday Heroes Campaign not only provides public education, but also can be considered within the context of trial-specific initiatives. Clinical research professionals—sponsors, CROs, investigative sites, and subject recruitment service providers—can and should factor the Everyday Heroes Campaign into their subject recruitment strategies and tactics. Its use creates a synergist effect that provides a better return on investment for advertising dollars spent.

Investment in public education may indeed have a short-term, measurable impact on subject recruitment initiatives. As a result, requests for resources to support public education and outreach are justifiable. Moreover, the cost is minimal. CISCRP is making Everyday Heroes available—in print, radio, and television formats—to contributors of its national campaign for use concurrently with local subject recruitment campaigns. There is no incremental recruitment cost because sponsors, CROs, and sites are allocating a portion of their existing recruitment advertising budgets.

There is now a real opportunity to collectively use the Everyday Heroes educational campaign to both reverse the erosion of public trust in the clinical research enterprise and to promote appreciation and interest in clinical trial participation. It is our hope that the two year effort to create this re-branding initiative will culminate with its widespread adoption. There is nothing to lose and much to gain in collectively supporting public education and outreach in conjunction with clinical trial-specific promotion. 

,* MBA, is a senior research fellow at the Tufts Center for the Study of Drug Development and the chairman of the Center for Information and Study on Clinical Research Participation, both in Boston MA, email: 

 is director of global enrollment optimization at Eli Lilly and Company in Indianapolis, 

 is group managing director of Ogilvy Healthworld in London, UK.

*To whom all correspondence should be addressed.

To watch the Everyday Heroes commercial, please visit 

Pilot test to rebrand clinical research shows promise as a way to build public trust and promote interest.

Everyday Heroes Campaign Captures the Public

For the lay public, clinical research is not typically a topic of dinner table conversation. It's not discussed in high school science classrooms. And it's usually only discussed in a doctor's office when a patient has been diagnosed with an illness for which no available treatment exists or for which available treatments are unpleasant, unsafe, and ineffective. 

Yet arguably, clinical research touches every person's life regularly. According to the U.S. Census Bureau, more than half of all Americans (55%) are taking prescription medications at the present time. Most Americans (90%) report routinely using over-the-counter medications. And most Americans (85%) state that they recently read an article or heard press coverage of clinical studies recommending new approaches to improving health through diet, exercise, and lifestyle changes. 

The lay public simply does not realize or understand that all medicines it consumes—and many behavioral interventions—have gone through clinical testing, involving people who have volunteered their participation. Indeed, although the majority of people believe that clinical research helps advance medical knowledge, less than 15% believe they have even a rudimentary understanding of how clinical research affects them directly. 

Ubiquitous patient recruitment ads on metro trains, in big-city newspapers, on the radio, and in health communications bombard the public frequently. An estimated half a billion dollars is spent annually on mass media patient recruitment advertising and promotion. These ads are designed to recruit for specific studies, and yet it is clear that the broader understanding of the benefit of clinical research is neither widely considered nor appreciated by the public. For this reason, specific study messages are met with a large amount of indifference: They have no foundation on which to rest. The public has no context within which to understand the importance of clinical research. 

 This public service ad for CISCRP was rated highly among focus group members, beating out campaigns featuring sports stars.			

Extensive media coverage of drug withdrawals and their aftermath; conflicts-of-interest among FDA members and research professionals; questionable pricing practices; and tragic errors resulting in harm to patients have left the public confused and troubled. Public sentiment toward the clinical research enterprise is at an all time low. One recent public poll revealed that only 14% of the American public feels pharmaceutical companies are "Honest," a rating similar to that bestowed upon the tobacco and used car industries. In another 2006 poll, more than 70% of Americans said they believe drug companies put profits ahead of patient needs. 

The clinical research enterprise faces its greatest crisis ever: widespread distrust. Without public and patient support, there can be no realization of innovations in medical therapies. In the absence of study volunteers, clinical trials cannot be conducted or they are severely delayed and, ultimately, public health advances are not made. An alarmingly low 4% to 6% of eligible patients participate in U.S.-based clinical trials annually. 

The clinical research community has largely failed to respond to poor public perception and eroding public trust.  Professionals affiliated with research sponsor organizations, with service providers, educators, regulatory agencies, and human subject protection programs have been insular in their approaches to addressing the crisis. They have focused on better professional training, on accreditation and certification, and on improving the disclosure of conflicts-of-interest and the accessibility of clinical research information. But the magnitude of this crisis demands a response designed to educate the lay public and to "rebrand" clinical research as a valuable public good. 

Large-scale public education campaigns have proven to be a highly effective strategy for changing human perception. The long-running national milk campaign "Got Milk?" has used a string of celebrities to promote milk consumption. The "This is your Brain on Drugs" campaign, featuring those famous fried eggs, has affected at least two generations' views on the dangers of drug abuse. And who can forget the crying Indian, whose tearful face in the 1970s television commercials became a powerful symbol of the anti-litter, save the environment movement? 

Focus Group Reactions to Everyday Heroes			

Clinical research needs its own "Got Milk"-like campaign—one that is recognizable and easy to remember, and engages the public in thinking differently about clinical research. Such a campaign has been a top initiative at the Center for Information and Study on Clinical Research Participation (CISCRP) during the past 12 months. 

With an unrestricted educational grant from Eli Lilly and pro bono collaborative support from Fast4wD Ogilvy—a division of international advertising and marketing agency Ogilvy & Mather—CISCRP worked extensively with a large and diverse group of research professionals and patient advocacy organizations and conducted consumer focus groups to arrive at the most effective clinical research rebranding message: "Everyday Heroes." To a wide cross-section of the population, print advertisements that call out ordinary individuals as heroes piqued interest and curiosity in addition to sending the message that being a part of clinical research is a good and even noble thing to do. 

Other campaigns tested were less likely to get across the idea that an everyday person can help save lives. Those featuring study volunteer heroes alongside recognized sports heroes were polarizing and unpopular. Another campaign that used an eye-catching red superhero cape blowing on a clothesline was confusing to participants and some perceived the cape as trivializing the message of everyday heroes. A campaign picturing a long line of people coming together to advance medicine tended to raise more questions than it answered. Another showing remedies for everyday ailments was confusing and viewed by some as promoting homeopathic drugs. 

Like the "Got Milk?" campaign, "Everyday Heroes" will unfold nationally as a series of engaging, general educational messages. As such, one of the key challenges is to maintain a long-term perspective that the goal of the campaign is to build consumer awareness and trust in the important role that clinical research plays in advancing medical science, and to change public perception and appreciation of the clinical research volunteer. The purpose of this campaign is not to recruit volunteers into currently enrolling studies or promote newly approved treatments. This campaign must focus on early education and on establishing a relationship and a connection with the public. What follows is a detailed discussion about the development of the Everyday Heroes campaign. 

Eighteen months ago, CISCRP and Fast4wD Ogilvy convened a small diverse group in Washington, DC, to talk about the difficulties in patient recruitment and retention and how that related to public education failures. That initial meeting brought together about two dozen people representing patient advocacy groups, the U.S. Food and Drug Administration, the National Institutes of Health, PhRMA, the National Health Council, academic institutions, pharmaceutical companies, CROs, investigative sites, patient recruitment service companies, and the IRB community. Fortunately, the dialogue remained constructive, the views largely shared, and the mindset determined to move forward. 

This initial meeting highlighted the unified view that not only does the public feel disconnected to clinical research but also that those orchestrating trials often forget the process has an association with physicians in the community and involves real members of the public. There was also consensus that study volunteers deserve recognition and appreciation for the profound gift that their participation provides for others. Most study volunteers want to know that they're making a valuable contribution to science. Yet few research centers stay connected to volunteers after a trial has ended. 

At the conclusion of the small group meeting, Fast4wD Ogilvy reviewed attendee observations and suggestions and developed 15 different campaign ideas, called "Treatments." These treatments cut across a wide spectrum of educational approaches—some serious, some shocking, and some quite humorous, including one campaign that tried to capture how far advances in medicine had come. This campaign juxtaposed several old and new treatments, such as a pile of cow dung for severe skin burns against modern treatments made possible by millions of clinical research volunteers. 

Four months later, another meeting was held among nearly 60 research professionals representing all stakeholder groups involved in the clinical research enterprise. This large group spent a full day reviewing, discussing, and modifying the 15 treatments. At the end of the meeting, the group had winnowed the campaign choices to five. After incorporating suggestions and refining the campaigns to make them look polished and professional, the five treatments were presented in March 2006 to consumer focus groups. 

The focus groups included a representative mix of ages, genders, ethnicities, lifestyles, and occupations, and individuals were grouped into one of three socio-economic classes. The "Everyday Hero" campaign scored significantly higher across the board. Consumers liked the concept and felt appreciative of the people featured in the campaign. Also, the hero ads most prompted them to want to learn more. Among the lower socio-economic group, in particular, respondents identified with and admired the featured "Heroes." 

The copy on the final version of the campaign reads, "Medical heroes can be found in ordinary places. Together we can make a difference through medical research." Based on the request of many focus group respondents, the campaign also gives viewers a telephone number and a Web site where they can learn more about the clinical research process. 

"Everyday Hero" campaigns will make their debut in a broad public service campaign being targeted for launch in 2007. Heroes still under consideration for the other two ads include a person working behind a desk in an office setting, a teacher in an elementary school classroom, and a senior citizen in a social setting. The CISCRP logo and contact information will appear on the ads. The FDA has expressed interest in having its logo on the campaign material. 

Our goal now is to raise $750,000 to $1 million from leading pharmaceutical companies and government agencies to underwrite first-year campaign costs. The ads will run on a discounted basis in major consumer publications like 

 as well as on billboards, bus shelters, and bus backs in high traffic areas like Manhattan, Chicago, and Los Angeles. Banner ads will likely also be placed on commercial Web search engines and high-traffic consumer sites like Yahoo and WebMD. 

CISCRP and Fast4wD Ogilvy intend to conduct follow-up research on the campaign's impact on public perceptions about and public interest in clinical research. Over time, provided that this campaign achieves the reach and frequency planned, we're hopeful that ongoing public opinion will reflect a better-informed populace, greater appreciation of the value of clinical research to public health, and deeper recognition of research volunteers. 

The authors wish to thank David Davenport-Firth, director of Ogilvy Healthworld, for his creative leadership in developing this campaign. 

 MS, MBA, is a senior research fellow at the Tufts Center for the Study of Drug Development and the chairman of the Center for Information and Study on Clinical Research Participation, both in Boston MA, (617) 590-4175, email: 

 is group managing director of Ogilvy Healthworld in London, UK. 

 is the director of global enrollment optimization at Eli Lilly and Company, Indianapolis, IN.

*To whom correspondence should be addressed.

A new patient recruitment campaign depicts trial volunteers as heroes and aims to both educate and win over the public.

Mission Possible: Rebranding Clinical Research

Last year may well be remembered as a public relations nightmare for the clinical trials industry. 

Early in 2004, the American Medical Association and the publishers of prestigious, peer-reviewed medical journals such as the 

 called for a government-based clinical trial registry to provide greater disclosure of drug risks in order to address potential bias in industry-sponsored clinical trials. That same month, the New York attorney general filed a lawsuit against GlaxoSmithKline for deliberately suppressing negative clinical trial results for its antidepressant drug Paxil. Dr. Angell's controversial and highly publicized summer 2004 book, 

The Truth About Drug Companies: How They Deceive Us and What to Do About It

, argued that conflicts of interest guide industry-sponsored trials.  

Later in the year, Merck withdrew its blockbuster COX-2 inhibitor Vioxx due to the drug's elevated risk of heart attacks shown in a large clinical trial. The Office of the Inspector General next released a survey showing that one-third of FDA scientists have doubts about the agency's ability to monitor the safety of new therapies during the review and approval process. And late in 2004, clinical research results showed elevated risk of heart attack and stroke with Pfizer's widely used COX-2 inhibitors Celebrex and Bextra. The press also reported on one study suggesting increased risk of cardiovascular events with over-the-counter drug Aleve. 

Poor public relations in 2004 are arguably part of a longer, five-year downward spiral that the clinical trials industry has endured and largely ignored. Indeed, some of the media's scrutiny and skepticism has been warranted. In 1999 and 2000, several tragic deaths of study volunteers participating in government- and industry-funded clinical trials (e.g., Gage Stevens, Jesse Gelsinger, Ellen Roche) were widely publicized. In 2001, the Office of the Inspector General released a report documenting a stretched and failing human subject protection system. That same year, the Food and Drug Administration reported dramatic increases in the number of complaints filed against investigators for noncompliance and fraud. In 2002, the Institute of Medicine issued its report calling for major reforms to the informed consent process.  

An assessment of major newspaper coverage during the past five years shows that the frequency of articles on the clinical research enterprise is increasing, but the number of negative articles is growing at a disproportionately faster rate.

 published its now-famous cover story "How Medical Testing has Turned Millions of Us into Human Guinea Pigs." The media's reference in this article—and in dozens more since then—to study volunteers as "Guinea Pigs" fails to acknowledge the profound decision that millions of people make each year when they give their consent to participate. The article also neglects to report that volunteers are active participants in a study and that they have the option to withdraw at any time. Sadly, clinical research professionals in government, foundations, and industry have done remarkably little to address the media's incomplete, sensationalistic, and ill-informed portrayal. Yet the public and prospective volunteers derive most of their exposure to—and education about—the clinical research process from the media. 

Public and volunteer communities know nearly nothing about clinical research and the important role that each party plays in the process. Historically, the clinical research community has relied heavily on the informed consent (IC) process as the primary means of educating study volunteers. Not only is the IC process reaching a narrow subset of the communities that it must educate, but also recent assessments of the process show that it is failing to live up to its intended purpose. According to a 2002 report, for example, the Institute of Medicine found the focus of the IC process to be largely the review of lengthy and confusing documents designed to protect the investigative site from liability. 										

Although the informed consent process invites study volunteers to ask questions freely, more than 70% of volunteers say that they do not even know what questions to begin to ask. Following informed consent form review, more than 85% of volunteers claim to understand the duration of the study and that they can withdraw at any time. But one-third of volunteers report that they did not understand that their clinical trial may carry additional risk and discomforts compared to a standard treatment.

 Only 17% of volunteers report that they sought input from other trusted sources. And 14% of volunteers say that they did not even bother to read the informed consent document prior to signing it.

The public and prospective volunteer communities also derive some education about the clinical research process through seeing study advertisements and promotions. According to a 2004 poll of nearly 6,000 adults, more than two-thirds (69%) were exposed to information on clinical research studies through television, radio, print media, and via Internet advertising. One out of seven adult Americans are exposed to information about clinical research studies from their primary or specialty care physician.

 Without broad understanding and context, however, recruitment advertising and promotional messages are met with at best passing curiosity from the public and prospective volunteers. According to recent public opinion polls, 94% of Americans perceive that clinical research is important to advance medical knowledge, but less than 20% of those diagnosed with severe and life-threatening illnesses report considering clinical trials as a health care option.

 Despite a wealth of online information available today, less than 5% of the general public knows where to find information about relevant clinical trials.

The medical and health professional communities are largely absent from educating the public and prospective volunteer communities. Of the 700,000 board-certified physicians in active community practices, less than half report referring their patients into clinical trials with an average referral rate for each physician of less than one patient per year.

 In 2004, less than 10% of the total number of volunteers completing clinical trials came directly from physician referrals. This is primarily due to health providers lacking adequate information and context with which to make informed decisions on behalf of their patients.

Growing coverage of the clinical research enterprise in the media is highly problematic. Whereas research sponsors and investigators tend to communicate with journalists when they are ready to announce a medical breakthrough, the media tends to focus on human mistakes, concealment, fraud, and corruption. As a result, there is a long-standing disconnection between the media and clinical research professionals. 

Most journalists have little to no formal knowledge about clinical research and they have virtually no time to learn as they rush to meet tight editorial deadlines. Facing a crunch, and due in part to patient privacy and confidentiality, many journalists must resort to interviewing the same study volunteers featured in past articles—typically the ones who are most vocal and often the whistle-blowers. Reporters are often looking for a hook—a storyline that will catch the reader's attention, possibly even garner a journalistic award. Many have been trained to dig for the negative stories and exposés.  

Research sponsors rarely, if ever, respond to media coverage as government and corporate employees are usually instructed not to interact with the media for fear of bringing more attention to a story, or of appearing defensive and self-serving. As a result, the public is largely receiving a one-sided view of the clinical trials industry in the media. 

Popular culture is also unbalanced. In those rare instances when clinical research is depicted in novels and in film, it is typically portrayed as clandestine and corrupt.

 These portrayals provide the necessary vehicles to create thrilling and intense fiction. In John Le Carre's book 

, for example, a health care activist is murdered before she is able to expose a large pharmaceutical company that has been exploiting vulnerable African study subjects. And in the film 

, Harrison Ford plays a physician wrongly accused of the murder of his wife. Ford is able to exonerate himself when he proves that the murder was part of a cover-up by a colleague who has been falsifying clinical research data to support a drug for a company in which he is a major shareholder. 

Without public education, support, and involvement, research sponsors and investigative sites are spending ever higher levels of resources on patient recruitment and retention with marginal—if not diminishing—results. During the past decade, spending on patient recruitment programs by investigative sites and research sponsors has grown by 16% annually, reaching more than $500 million in 2003.

 In that same period, volunteer randomization rates have declined steadily. According to reports from several hundred investigative sites and analyses of study completion records, nearly 45% of volunteers were enrolled in clinical trials in 1984. In 2003, randomization rates declined to less than 25%.

 And whereas one out of every six enrollees dropped out of clinical trials in 1984, an average of 27% dropped out of clinical trials in 2003

 Figure 1. The recruitment and retention challenge.			

Presently, investigative sites report that more than 90% of all clinical trials must extend their original study timelines in order to enroll the requisite number of volunteers.

 Sites attribute these enrollment delays to both the unrealistic timelines set by research sponsors and to the high level of difficulty in recruiting and retaining study volunteers. 

Government and industry-funded research sponsors now expect significant project delays as they seek approximately 2.6 million study volunteers to complete their studies each year.

 It is estimated that only 4%-6% of eligible patients who suffer from severe and life-threatening illnesses take part in U.S. clinical trials today.

Clinical research sponsors have turned their attention to recruiting volunteers from markets abroad—typically from ascending regions that include Latin America, Central and Eastern Europe, India, and China. These regions offer well-trained professionals, large numbers of treatment-naÃ¯ve and motivated patients, and lower relative costs. During the past five years, clinical research spending in ascending regions has grown by 45%-60% annually.

 But sponsors have done a poor job of explaining these strategies and practices to the media and the public. As a result, the media tends to portray the conduct of clinical trials overseas as exploitive and financially motivated. 

Regulatory agencies and the clinical research community have responded to public confusion and concern by implementing reforms designed to assure higher levels of compliance and reduce errors that might result in physical and ethical harm to study volunteers. Since 2001, a growing number of universities have established clinical research degree programs. The Department of Health and Human Services has implemented new privacy and conflict-of-interest disclosure  guide lines. Two agencies—The Association for the Accreditation of Human Research Protection Programs and the Partnership for Human Research Protection—were established to accredit institutional review boards. In addtion, trade groups including the Drug Information Association and the Association of Clinical Research Professionals have developed investigator certification programs. 

This past year, prompted by GSK's alleged failure to disclose clinical research information about Paxil and Merck's withdrawal of Vioxx, public and policy-maker outcry for greater transparency and comprehensive disclosure of active clinical trials and study results came to a head. Industry sponsors, including Eli Lilly, GlaxoSmithKline, and Pfizer, announced programs to provide comprehensive listings of clinical trials online. The industry lobbying group PhRMA (Pharmaceutical Research and Manufactures of America) also launched a clinical trial results Web site. 

These reforms and measures are an important step, but they are not nearly enough. They begin to address the need for greater transparency and disclosure, and they satisfy regulatory agency and research sponsor requirements to minimize potential errors, noncompliance, and fraud. However, they fail to acknowledge and address the fundamental need to educate and engage the public and prospective study volunteers. As such, these reforms will only be marginally effective until these communities understand why they are necessary and how they will influence and inform decision-making about the risks and benefits of clinical research participation and new medical therapies. 

Recent polls conducted among more than 5,800 American adults suggest that public trust in the clinical research enterprise is waning, with the highest levels of distrust existing among minority communities.

 (see Figure 2). The clinical trials industry must act now if it is to better inform and engage the public and prospective volunteers as partners in this process. 

Figure 2. Ethnic differences in perceptions of clinical trial safety.			

Long overlooked, the clinical research enterprise must proactively raise the overall level of public awareness, must properly and fully inform the public and prospective volunteers, and must establish and build a stronger connection with the many vital stakeholders on the periphery. Broad outreach and advocacy are critical practices that must be utilized to raise public awareness and understanding of the clinical research process and the benefits it brings society. 

Outreach and advocacy programs are already being used by a small but growing number of investigative sites and contracted clinical service providers to support patient recruitment efforts. Examples of typical "grass roots" initiatives include clinical research professionals attending health fairs; providing free health screenings at community centers; listing trials in health association and health support group newsletters; and speaking at community events. 

Reports on the results of these efforts show that advocacy and outreach has the potential to dramatically impact prospective volunteer interest in study participation. Community outreach efforts have been shown to more than double the rate of prospective volunteer inquiries in a closed head injury study; nearly quadruple the inquiry rate in a recent study of ulcerative colitis; and increase physician referral rates by 71% on a study of a new medical therapy for type II diabetes.

Current outreach and advocacy programs are promising but limited. They are project specific and therefore not designed to provide a general education. They are short-term in their focus as they are intended to fill individual clinical trials. Once enrollment targets have been reached, these programs are terminated. Their design and implementation also vary from project to project. As such, they fail to provide a consistent and coordinated message. 

To be effective and successful, broad education, outreach, and advocacy programs must have the support of the entire clinical research professional community. These programs must be well-coordinated and integrated into all clinical research activities. They should have a consistent message and should focus on raising awareness and engaging stakeholders on the periphery of the enterprise, including the medical community, government officials, family and friends of potential volunteers, and the media. 

Specifically, a comprehensive, focused set of initiatives is needed to introduce the public, prospective volunteers, health professionals, the media, and policymakers to the large amount of clinical research activity being supported and conducted within the community each year. These initiatives should include: 

1) Educational material to assist volunteers in framing the questions to ask of the research community in order to protect their safety and rights, and to inform the profound decision to participate in a clinical trial. These materials should also address the unique educational needs of special populations (e.g., pediatric and geriatric populations; communities of color and gender-based populations). 

2) Broad-based outreach and advocacy programs to convey the important role that health care providers play as partners in the clinical research process. These programs should also assist the media and policymakers in understanding the processes and requirements of developing new medical interventions and participating in clinical research. 

3) Messages and images that put a face on the many professionals who dedicate themselves to managing and conducting clinical research and elucidate why R&D is a lengthy, costly, and labor-intensive process that holds the potential to significantly benefit and improve public health. 

4) Most importantly, an open acknowledgement of society's  appreciation for the many people who give the gift of participation each year. 

The National Institutes of Health and the Food and Drug Administration are increasingly recognizing the critical and timely need for broad education, advocacy, and outreach. This past year, the NIH announced the creation of a Public Trust Initiative to instill higher levels of public confidence in government-funded clinical research.

 The FDA recently announced its interest in raising overall awareness of the importance of clinical research to public health and the role that the agency plays in that process.

 In an effort to increase the transparency of industry-funded clinical trial results, PhRMA also recently launched a new Web site

 that includes educational information for the public. 

In March of 2004, the Center for Information and Study on Clinical Research Participation (CISCRP) was established to educate and inform stakeholders including the public, prospective volunteers, health providers, policymakers, and the media. Based in Boston, CISCRP is an independent, nonprofit organization that derives its funding from a diverse base of supporters that includes government agencies, foundations, corporations, and individual clinical research professionals. As a result, CISCRP is well-positioned to provide broad education, advocacy, and outreach across all areas of government- , foundation- , and industry-funded clinical research activity.  

Since its inception, CISCRP has produced and released the nation's first public service announcements for radio, television, and print to raise awareness and national pride in the clinical research enterprise, and to express appreciation for the millions of people who volunteer to participate in clinical trials each year. CISCRP has compiled contact lists of professionals and experienced study volunteers who have given their consent to speak at community events, lecture medical schools, and be interviewed by the media. It has hosted and participated in numerous advocacy, professional, and community events where broad-based educational materials have been distributed. CISCRP has organized and implemented a Clinical Research Public Perception Summit that brought together members of the clinical research community and the media. The organization was selected to collaborate with the FDA and the NIH on general outreach and advocacy projects. And CISCRP developed and launched a resource-rich Web site at 

. These are but a few examples of CISCRP's many programs and initiatives.

It is time for clinical research professionals to acknowledge that the public and volunteer communities have a severely limited and incomplete understanding of the clinical research process, and the important role that each party plays in that process. These stakeholders, and others on the periphery of the clinical research enterprise—including health care providers, the media, and policymakers—are not engaged in the process. Broad education, outreach, and advocacy are needed at both the national and local levels if we are to partner with these vital stakeholders in order to safely realize the profound benefits of biomedical research to society. 

An Industry in Evolution: Sourcebook of Charts and Statistics Providing Analysis and Insight into the Clinical Trials Industry

2. K. Getz, "The Informed Consent Process," 

3. K. Getz, "The Informed Consent Process," 

4. HarrisInteractive, "Views and Attitudes of Clinical Research Studies," 

During the past decade, spending on patient recruitment programs has grown 16% annually. In that same period, volunteer randomization rates have declined.

Jim Kremidas

Articles

Site Perspectives on Clinical Trial Quality

Everyday Heroes Campaign Captures the Public

Mission Possible: Rebranding Clinical Research

Educating the Public: A Critical, Unmet Need