Hugo Stephenson

	Be honest-would you really trade the technology of today for the technology of 20 years ago? Do you miss giant projection TVs, thousands of CDs, car phones the size of suitcases, and the revolutionary Windows 95?

	No one would blame you for wanting to stay current. Today’s technology is convenient, fast, and mobile in a way that few of us could have predicted even back in the 1990s. In fact, the advancements of the last 20 years have made many aspects of our lives easier than ever.

	So why, then, in my unique dual role as a principal investigator in Melbourne, Australia-and the Executive Consultant of technology provider DrugDev-do I long for the past?

	It may sound counter-intuitive, especially from me, but the processes we ran back then made it so much simpler for sites to run a clinical trial. To understand my mindset, let’s take a trip back in time to 1999:

	My study coordinator and I were tasked with enrolling, and treating patients. Back then, that meant filling out large binders of forms from sponsors-non-carbon triplicates. Every visit, we would fill out the forms supplied by the sponsor with the patients’ information and send them back. If there were any questions about the data we supplied, they faxed us back and we would record any changes as needed. That was the end of it.

	Sponsors would take the forms and send them off to a data processing facility to be double-data entered in a database. Training was focused on the protocol, supply management, and safety-it was still time consuming, but it was relevant to our responsibilities.

	Nowadays we understand the flaws of this system-the data quality and efficiency concerns to name just a few. It makes sense companies like DrugDev would come along to “fix” the process.

	Flash forward 20 years later and a clinical trial site is rife with computers, tablets, and smart phones. Today, instead of completing forms, we’re given access to online software that requires my study coordinator to remember multiple logins, become familiar with multiple (counterintuitive) user interfaces, and deal with multiple ways that our sponsors want us to things (e.g. enter these diagnoses here, but not here). She’s a wonderful coordinator, but she’s not a data specialist. We spend a tremendous amount of time training on each system’s differences and learning the different ways to input data. And we still get emails and calls weeks after “acceptable” data entry when sponsors change the way they want to deal with data as it starts streaming in.

	As sponsors jumped at the chance to implement new site-facing technologies and processes they ended up creating a fragmented ecosystem of different software, hardware, vendors, and processes-one or more for each “silo” that exists within the sponsor/CRO world. Unfortunately, this muddied the waters for sites when it came to performing basic functions such as patient enrollment. These days, we spend more time learning new systems, training, entering data, and troubleshooting than we ever did back then.

	In parallel, there has been more pressure on “fair market value,” and downward price pressure on what sites get paid. Sites are feeling they are doing a lot more for a lot less. As an industry we need to either pay sites more to do the extra work, or make the process simpler for them.

	When you are working to run clinical trials every day in addition to seeing patients in a private practice, it becomes clear by industry picking so-called best of breed point solutions that don’t talk to each other the industry has only made matter worse for their clinical site partners.

	It’s all done with the best intentions, to provide sites with tools that help them do their jobs faster. In reality, speaking as a site myself, I’d rather have one system that does everything well enough to get the job done, than 11 separate systems that each do one (and only one) thing a little bit better. Talking to sites around the world, it’s clear that I am not alone.

	Simply stated, providing disparate systems to sites may make it easier for teams within a sponsor or CRO, but it creates a real headwind for sites that just want to focus on patient recruitment and care. That’s why technology vendors are working to make things simpler. Sites want to use one system for a dozen or more tasks without having to learn, remember, and log into a dozen different systems. Technology vendors can do this by combining investigator experiences with the needs of sponsors and CROs.

	This isn’t the end of our work. If software providers aren’t looking at how our solutions make things easier for sites every single day, we’re not helping anyone.

	At the end of the day, you don’t need to be a principal investigator to think like a site when developing new technology. Go out and talk to study coordinators. Shadow them as they meet with patients. I’ve seen plenty of developers do exactly that. You’ll quickly understand why they might long for the past-even if it means giving up Facebook, Twitter, and Candy Crush.

	Our job as a technology provider, and I think we are on the right track as an industry, is to come together to combine the simplicity of yesterday with the convenience of today. Only then will sites like mine give up yearning for the past (my Terminator 2 Pinball machine of course, is here to stay).

, MD, is Executive Consultant of DrugDev. He can be reached at 

Articles

Hugo Stephenson MD examines his longing for the past with the advances of today's technology in this blog focused around sites.

Stop Expecting Sites to Love Technology Just Because It’s New

Recruitment of patients for participation in clinical trials is a major source of research expense. Estimates of recruitment costs vary by study and disease state; however, data suggests that patient recruitment costs range between less than $10 to more than $1300 per completed subject.

  Patient recruitment delays can compound costs by delaying the time to product approval, often decreasing the period of exclusivity post-launch. Studies, for example, have demonstrated that recruitment difficulties can account for up to 45% of study delays, and these often can last six months or more.

As sponsors search for more time-efficient, less-expensive approaches to conduct community outreach for clinical trials, Internet-based sources have gained more attention. Of particular interest has been the emergence of health-focused, online social networks. In theory, these online social groups, formed around specific medical conditions or health interests, are comprised of people that are highly engaged in their disease, and potentially, motivated to participate in clinical trials.  

While the number of marketing and research groups using online social networks has increased,

  few have published data that explores the potential effectiveness and costs of working with these groups, especially in comparison to outcomes associated with other techniques. In this article, we address this gap in the literature by presenting effectiveness and cost outcomes associated with three different patient outreach approaches. 

The objective of this study was to compare the effectiveness and costs of three direct-to-patient outreach strategies: direct mail outreach through use of a third-party mailing list; email outreach through use of a third-party email list; and email outreach through 

, a Web-based medication monitoring service that includes a social network of members interested in sharing feedback on their medications. 

This study assessed the ability of these three different outreach strategies to identify potential study participants for two clinical trials in rheumatoid arthritis and multiple sclerosis. These medical conditions were chosen because they comprise large numbers of patients suffering chronic illness with substantial potential for morbidity and mortality, and they are often the focus of many costly clinical trials. 

The outreach was conducted in the United States in June, July, and August 2009. Participants from any of the three outreach methods who completed an online survey received $10 in compensation for their time. The survey was blinded and did not contain any information on actual study medication, sponsor, or investigator site. Surveys conducted through 

 are covered under a blanket Institutional Review Board (IRB) assessment from Independent Review Consulting, Inc. IRB approval for the direct mail or email outreach was not seeked because the blinded nature of the questions (i.e., no clinical trial specifics included) met the criteria of 45 CFR 46.10 related to research on educational practices. 

The third-party direct mail and email outreach was conducted by Automated Information Systems, Inc. (Glen Ellyn, IL), a vendor chosen among other candidates for the size of their patient databases in these therapeutic areas; their ability to send both email and direct mail communications; and their ability to prevent duplication between the electronic and paper-based methods. Automated Information System's mailing list is derived from multiple sources, including managed care organizations and patient advocacy groups. Email communications were checked for accuracy using a matching system from Experian, Inc. (Costa Mesa, CA). 

The social network outreach was conducted among members of 

, a free, medication service that monitors the safety of prescription medicines, over-the-counter medicines, and health care supplements for over two million patients. 

 attracts members through online search engines as well as outreach efforts with physicians, pharmacies, and health-related Web sites. 

 members were randomly screened for participation in this study if their profile included an oral rheumatoid arthritis (azathioprine, hydroxychloroquine, leflunomide, methotrexate, or sulfasalazine) or multiple sclerosis (beta interferon or glatiramer acetate) medication. 

The effectiveness of each outreach strategy was measured by the proportion of those contacted who completed the online survey (i.e., the response rate) and the costs associated with generating one individual interested in the study (i.e., a function of response rate, unit costs, and clinical trial interest). The current study focused on outreach and did not follow patients through the investigator site screening process and enrollment. 

The unit cost for third-party mailing was $0.86 and $0.42  for third-party email. Actual marginal costs to contact the 

 members were negligible as the community and communication platform already existed. For the purposes of this analysis, however, the cost associated with 

 outreach was assumed to be equivalent to the third-party email cost of $0.42 per email contact. 

Response rates and demographic characteristics were compiled for each outreach strategy. Comparison between the strategies on gender was performed using Chi-square test, and on age using one-way analysis of variance (ANOVA), with Bonferroni correction to adjust for multiple comparisons. These analyses were conducted using SAS Version 9.1 for Windows from SAS Institute Inc. 

The third-party vendor sent 12,500 letters and 15,000 emails to patients with rheumatoid arthritis and multiple sclerosis (total 25,000 letters and 30,000 emails); these numbers reflected the vendor's minimum requirements. Overall costs associated with these contacts totaled $21,500 for the direct mail ($0.86 per letter) and $12,600 for the email contact ($0.42 per email). 

The response to direct mail outreach was 3.6% (905 of 25,000) as measured by participation in the online survey. Nearly all individuals responded within two weeks to the mail survey. Overall, 78% of individuals responding were interested in participating in the clinical trial, resulting in a cost per interested individual of $30.45. 

The third-party email outreach yielded a lower response rate than the direct mail, 0.5% (157 of 30,000). Most responses for email outreach occurred within 24 hours. The proportion interested in a trial, however, was 80%, very similar to the interest expressed by individuals contacted through direct mail. The resulting cost per interested individual for the direct email outreach was $100.32.  

During the June, July, and August 2009 time frame, 

 surveyed 3657 members treated with a medication for rheumatoid arthritis (n=3057) or multiple sclerosis (n=600). Of these 482 responded, yielding a response rate of 13.2%. Similar to email outreach, most responses to 

 outreach occurred within 24 hours. Of the individuals surveyed, 75% expressed interest in a clinical study, resulting in a cost per interested individual of $4.25 per response (assuming a cost per contact similar to charges for the third-party email outreach). 

The response rates and costs associated with each of the three outreach strategies are shown in Table 1, and demonstrate that the social network-based email strategy was substantially more effective and cheaper than either third-party direct mail or email. Contrary to industry assumptions, third-party email was substantially more expensive than direct, paper-based mailings, primarily driven by the lower response rate. Interestingly, clinical trial interest among patients motivated to respond to the survey was similar across all three outreach methods.  

Table 1. The social-network based recruitment strategy was substantially more effective and cheaper than third-party direct mail or email. 

Some differences were observed on gender and age between the three-outreach methods. 

 had a higher proportion of women respondents (84.9%) compared to other methods, however, women were generally more likely to respond in all groups (76% to 80%, see Table 2). Surprisingly, direct mail appeared to attract slightly younger respondents compared to both email approaches.  

Table 2. Women were more likely to respond to an invitation to participate than their male counterparts, regardless of outreach method. 

Patterns of response rates, proportion of individuals interested in a clinical trial, and costs per interested individual were similar for each of the three strategies when evaluated separately for rheumatoid arthritis and multiple sclerosis, with social network outreach through 

 yielding a substantially greater response rate and lower cost per response (see Table 3). Across all outreach strategies, patients with rheumatoid arthritis were less likely to respond than those with multiple sclerosis: third-party direct mail (1.4% vs. 5.8%), third-party email (0.3% vs. 0.7%), and social network outreach (11.2% vs. 19.5%).  

Table 3. iGuard.org returned the highest percentage of interested candidates among both Rheumatoid Arthritis and Multiple Sclerosis sufferers. 

Once an individual responded, however, interest in clinical trial participation was very similar across all outreach methods: 70% to 73% for rheumatoid arthritis, and 79% to 85% for multiple sclerosis. With respect to costs, the social network-based approach was substantially cheaper than both the third-party direct mail and third-party email outreach for rheumatoid arthritis ($4.82 vs. $86.28 and $195.65 per interested individual, respectively) and multiple sclerosis ($2.69 vs. $18.72 and $66.77 per interested individual, respectively). In these disease specific analyses, third-party email remained the most expensive option driven by the significantly lower response rate. 

Our data suggest that patient outreach using the online social network, 

, is more effective and substantially cheaper than either third-party email or direct mail. This result is driven primarily by a substantially-higher response rate for the social network outreach: over four times higher than third-party mail and over 25 times higher than third-party email. Similar patterns were seen for responses in individual disease states, with the 

 email yielding over 35 times as many responses among rheumatoid arthritis patients as the third-party email, and more than 25 times the responses among multiple sclerosis patients as the third-party email. 

Although the proportion of individuals interested in the study was similar across methods, the outreach strategies differed substantially in their cost per interested individual. In this case, the social network outreach through 

 ($4.25 per respondent) was seven times cheaper than third-party direct mail ($30.45 per respondent) and nearly 25 times cheaper than third-party email ($100.32 per respondent). These results persisted when evaluated by disease state, where the 

 outreach was nearly seven (multiple sclerosis) to 18 times (rheumatoid arthritis) cheaper than the second best option, third-party direct mail, and faster as well (24 hours vs. two weeks to capture the majority of responses). 

These findings are consistent with the existing literature about variation in subject recruitment based on both approach and disease state. Data acquired in association with the Lung Health Study, for example, showed great variation in recruitment rates from various sources, ranging from 1.5% from direct mail to 46% from media sources.

  An unexpected finding of our study was that third-party email performed worse than third-party direct mail. Specifically, the third-party email response rate was 0.5% vs. 3.6% for direct mail, and costs per interested individual totaled $100 for email vs. $30 per respondent for direct mail. 

Importantly, our data extends existing knowledge about the use and relative cost-effectiveness of social networks for patient outreach. Although little has been published to date about the use of social networks in general and online social networks specifically for clinical trial outreach, our data appears to be consistent with the findings of one study conducted in nine community-based organizations. In this study, researchers successfully recruited participants for HIV testing through accessing peer (not online) networks at rates that exceeded other methods by five-fold.

 Furthermore, our findings about relative costs suggest that online social networks may not only increase response rate but may do so faster and at a lower cost than other approaches. 

Our data should be viewed in the context of some limitations. First, only two disease states were studied and it is possible that relative effectiveness and costs of these strategies could vary for different patient groups. The fact that our data showed similar relative response rates and costs when analyzed by specific disease states, however, suggests that the relative benefits of outreach through online social networks may hold in other disease states. 

Another limitation of our data is that we studied relative differences in outreach strategies based on responses to an online survey, not through actual clinical trial enrollment. While it is possible that the relative response and cost differences between strategies erode when actual participation in clinical trials is measured, the 

 experience with clinical trial recruitment in areas such as depression and gastroesophageal reflux suggests that the proportion of patients expressing interest that are ultimately enrolled in the study is somewhat consistent across outreach strategies. 

It is also possible that the quality of the third-party email list may have contributed to our findings. However, as this email list was checked against Experian, Inc., which most third-party vendors use for such validation, it is unclear whether a different result would have be obtained working with another third-party email vendor. There were also differences observed in gender and age among respondents to the three outreach methods. However, it is unclear whether these differences would be relevant in the context of recruitment for clinical trials. 

Finally, it is possible that unique characteristics of the 

 social network account for its relatively high response rates and low costs. As of August 2010, 

 had over 2.3 million members and is much larger than other online medically oriented social networks.

  While this may limit generalizations about the use of medically oriented social networks to recruit patients, the size of 

's membership allows its demographics to more closely approximate those of individuals seeking health information online when compared to smaller social network organizations. The outcomes associated with outreach through nonmedical social networks are unknown. 

The data suggests that patient outreach through an online social network has the potential to be more effective and less expensive than either third-party email or direct mail in the recruitment of subjects for clinical trials. Further studies are needed to quantify the benefit of collaboration with the growing number of social network sites for actual clinical trial enrollment when compared to traditional approaches to patient outreach. 

 66 Witherspoon Street, #262, Princeton, NJ 08542. 

, MD, is Senior Vice President, at Quintiles, Durham, NC.

*To whom all correspondence should be addressed.

1. L. Gren, K. Broski, J. Childs et al., "Recruitment Methods Employed in the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial," 

2. S. Roy, S. Patel, K.H. Sheehan, J. Janavs, D. Sheehan, "Efficacy of Print Advertising for a Bipolar Disorder Study," 

4. D. Borfitz, "Social Networking Sites Have Myriad Trial-Related Uses," Cambridge Healthtech Institute, March 2009, 

http://www.ecliniqua.com/2009/03/02/social-networking-recruitment-sites.html~

6. W.M. Bjornson-Benson, T.B. Stibolt, B.A. Manske et al., Monitoring Recruitment Effectiveness and Cost in a Clinical Trial,

7. L.W. Kimbrough, H.E. Fisher, K.T. Jones et al., "Accessing Social Networks with High Rates of Undiagnosed HIV Infection: The Social Networks Demonstration Project," 

8. M. Allison, "Can Web 2.0 Reboot Clinical Trials?" 

A comparison of the yields and costs of online outreach methods to other recruitment techniques.

Author | Hugo Stephenson