Michael J. Howley

Site perspectives of clinical trial quality are rarely heard, even though they are the foundation of the clinical research enterprise. But sites are not alone in conducting trials. They depend on sponsors and contract research organizations (CROs) to provide services and resources that allow them to execute a trial. The critical question from the site’s perspective then becomes: What are the essential resources and services that sites need from sponsors and CROs in order to conduct high-quality trials? The purpose of this paper is to identify such services and resources that are associated with high-quality clinical trials. 

	This study focuses on three research questions (RQs):

 What are the coproduction activities that sponsors and CROs provide to sites that lead to high-quality trials?

 How do sites evaluate sponsors and CROs on these key drivers of quality? 

How do each of these coproduction activities drive clinical trial quality? 

	To answer these questions, we conducted focus groups to identify all the activities that are important to sites (RQ1). To evaluate RQ2 and RQ3, we loaded all of these activities, as well as a measure of clinical trial quality, into an online survey. We then solicited Association of Clinical Research Professionals (ACRP) members to evaluate a recent trial as to the quality of the study and then how CROs and sponsors were performing on these key drivers of clinical trial quality. Finally, we created a statistical model to give the appropriate weight to each of these quality drivers.

	Trials in which the site had a direct relationship with the sponsor were perceived as being higher quality compared to when the site relied on a CRO. The most critical driver of trial quality from the sites’ perspective was communication-being available for questions, timely responses, and being helpful in resolving problems. While sponsors and CROs are doing reasonable well in this area, improvements in communication will yield the greatest benefits to improving clinical trial quality. Other significant drivers of quality from the sites perspective included the quality of the protocol, budgeting processes, technology, and monitor performance. 

	The results of this study illustrate a pathway by which sites, CROs, and sponsors can improve the quality of clinical studies. These findings can also provide a starting point for achieving an ROI on quality investments. Perhaps, most importantly, the results of this study give clinical trial sites a greater opportunity to provide insights into quality improvement methods for the clinical research enterprise.

	Clinical trial sites are the underappreciated foundation of the medical research enterprise.

 Despite this importance, sites struggle to execute studies effectively and efficiently.

 But sites depend on sponsors and CROs to support them by providing services and resources that allow them to execute the study plan. This lack of support can contribute to trial delays, increased costs, and considerable site turnover.

 As a result, investigative sites are becoming a scarce resource that limits the ability to conduct clinical research.

	The purpose of this paper is to address this situation by identifying the things that sponsors and CROs do that allow sites to execute high-quality clinical trials. Clinical studies depend on multiple stakeholders-often broken into separate organizations-working together effectively to coproduce a high-quality research program. Coproduction refers to the phenomenon by which multiple actors must apply their knowledge, skills, services, and resources to cocreate a complex service.

 While the site may interact with the patient, delivering a high-quality clinical trial depends on sponsors, CROs, and sites working collaboratively.

	But there are potentially thousands of coproduction activities that could impact trial quality. What are the specific drivers that lead to higher quality clinical studies? We distill all of these issues into the three RQs mentioned. 

RQ1: What are the coproduction activities that sponsors and CROs provide to sites that lead to high quality trials?

	To examine RQ1, we conducted two focus groups of 12 to 15 study coordinators each at the 2015 ACRP national conference in Salt Lake City, Utah. We had no difficulties recruiting potential subjects. Clinical research professionals were anxious to participate. Several of the participants told us that they were delighted they could contribute “so our voices could finally be heard.”

	Our approach to these discussions was inductive. The participants all had about 10 years or more of trial experience, so their observations were naturally granular. Each of the sessions was opened by asking, “What are the things (or activities) that sponsors/CROs do that help you execute a high quality study?” This led to a long list of performance activities. The process was iterative as participants clarified and built on each other’s observations. Once we had collated this list of performance activities, we worked with the focus groups to organize all of them into the following eight distinct groupings:

	We tested these groupings with focus group members by challenging, for example, whether budgeting and reimbursement were really different groups.

 Once we confirmed that these were distinct groups and aligned the items within each one, we arranged the participants into subgroups of about three people each to organize the items. At the end of the session, each group presented their refinements and the larger group commented on their work.

	Representative items-framed in survey format-for each category are shown in Table 1 (click to enlarge). We also included a set of questions to assess the quality of the clinical trial from the site’s perspective, which served as the dependent variable, shown at the bottom of Table 1 and derived from the SERVQUAL measure.

 All of these items were edited for clarity, combined with similar items when appropriate, and loaded into the CRO Analytics’ Performer platform in preparation for a survey.

RQ2: How do sites evaluate sponsors and CROs on these key drivers of quality? 

	We solicited responses from the ACRP membership, excluding those members who participated in the focus groups. Respondents logged on to the Performer tool. They were instructed to think about a Phase II or III trial that completed recently and then to evaluate the items with that study in mind. We also asked them about characteristics of the trial and demographic items.

	We received a total of 278 responses from experienced research professionals. The respondents had an average of 10.7 years’ experience. About 94% of respondents characterized themselves as clinical trial coordinators or clinical research managers, with 3% describing themselves as principal investigators and 3% using a variety of other job descriptions (e.g., consultant). The trials involved various therapeutic areas. Cardiology only made up 8% and oncology 3% of the trials. Most of the studies were Phase III (75%); the rest (25%) were Phase II. The average trial consisted of 70 subjects (sd= 495), with a long right tail. Eighty percent of the studies in this sample met their enrollment goals, higher than typically reported.

	How did sites evaluate the overall quality and performance in the coproduction activities? The average overall perceived quality of the clinical trials was 7.1 (sd= 2.17) on a scale of 1 to 10 (1 = low quality; 10 = high quality). This average is high compared to similar studies we have conducted on clinical trial quality. Figure 1 below illustrates the overall quality ratings with the ratings for each of the performance areas. This graph is constructed so that quality is the first column on the left and then performance areas are ordered from least to greatest (left to right). Each of the performance areas demonstrated discriminant validity from the others, meaning that we demonstrated statistically that budgeting was distinct from reimbursement and all the other groups.

	It is not surprising, given our focus group discussions, that budgeting, reimbursement, and monitors were the lower-ranking performance areas and that communication (µ= 7.2) and protocols (µ= 7.7) were rated more highly. Given these averages, we next sought to identify the amount of variation with each of these averages. Are these ratings consistent, or is there wide variation around this average? Greater variation means that there is less consistency or agreement about the quality and performance ratings. We screened for variation by looking at the standard deviations. A high standard deviation would indicate significant variation in the responses around the average. Figure 2 below illustrates the findings on variation and standard deviations. Items with the highest standard deviation scores have the highest amount of variation (or less consensus) around that average (i.e., they are likely to have subgroups of very low or very high scores). Items with the lower standard deviation scores have less variation (greater consensus). Communication (µ= 7.2) has a relatively high rating, but there was also a lot of underlying variation (sd= 2.53) around that rating. This suggests that while many sites are rating communication very high, there is a lack of consensus about this performance metric.

	The findings regarding the protocols were more complex. In examining the means and variation of the ratings of the protocol sub-drivers, we found that some of the important ones-complexity of the protocols and the frequency of amendments-had low ratings with high variation, or a lack of consensus. This is a paradoxical finding: even though the sites rated the overall quality of the protocols highly, they thought they were exceedingly complex, with excessive amendments. We also found that the correlations between overall protocol quality to the complexity (r= .05) and amendments (r= .26) were very low. 

	There is a disconnect between how sites evaluate the quality of a protocol and its complexity and amendments. We interpret these findings as an example of lowered expectations. It may be that sites have simply accepted the complexity of protocols and frequent amendments as a norm, so it does not impact their ratings of the overall quality of the protocol (see Figure 3).

	Finally, we found that sponsor companies outperform CROs in overall quality and many performance areas, as shown in Figure 4 below. Labels that have a single asterisk (*) were found to be statistically significant at p= .05 in a multivariate ANOVA (analysis of variance) and the labels with a double asterisk (**) were significant at p< .001. We see two possible explanations here based on our focus group discussions. It may be that CROs are conducting a wide variety of studies with many different sites, thus an individual site does not feel like it is getting the support or attention from the CRO. It may also be that the CRO can be perceived as a “middle-man” that blocks or delays the site from the getting information or support it needs to execute the study. Further research is needed on this question.

RQ3: How do each of these coproduction activities drive clinical trial quality?

	To this point, we have assumed so far that each of the performance drivers all have an equal impact on quality. To drill down further and weigh the relationships between overall quality and the eight performance areas, we created a multivariate linear regression model. The sites’ perception of the quality of the trial was the dependent variable and the performance areas were the independent variables. We also included a set of 

covariates-variables that might influence the relationship between trial quality and performance-in the model. The covariates were the size of the trial (number of subjects); whether it met its enrollment goals and endpoints; the complexity of the protocol; the appropriateness of protocol amendments; whether the site’s contract was with a sponsor or CRO; and the length of the experience in clinical trials of the respondent.

	The regression model explained a statistically significant (F(15,244)= 48.5, p< .001) amount of variance (R2= .75) in clinical trial quality. The model met all of the regression assumptions and variable were centered to enhance interpretation. The coefficients with their statistical significance tests are shown in Table 2 (click to enlarge).

	 We compare the magnitude of the coefficients graphically in Figure 5 below. In interpreting the coefficients, each of the variables was mean-centered before running the model. Each coefficient gives an estimate of the unique impact of that variable on trial quality at the mean of all the other variables. Also, the variables for total subjects and years’ experience were skewed to the right and, thus, were log-transformed.

	The impact of the performance drivers on clinical trial quality fall into three distinct layers. In the top, most impactful layer, communication (b= .29, p< .001) and protocol quality (b= .24, p< .001) had the single greatest effects on quality in this study. Communication in common usage is usually a very broad term. The results from our focus groups offer drill-down insights. In the focus groups, sites mean very specific things when they speak of communication. When a study team is communicating well, that means that they provide information in a timely fashion; are available for questions as they arise; and are available and helpful in resolving problems. The impact of protocol quality is particularly impressive because we included protocol complexity and amendments as covariates in the model. Protocol quality has an impact on trial quality above and beyond complexity and amendments.

	In the middle tier of performance drivers, budgeting (b= .16, p< .001) and technology (b= .15, p< .001) had statistically significant and positive-although more moderate-impacts on trial quality. Budgeting refers to the process of establishing the resources for the site’s services. Within this category, fairness was the dominant theme. The recurrent theme that we heard in the focus groups was that site coordinators simply wanted to be paid for the requested work. They become very frustrated when the site must absorb the cost of adjusting to protocol amendments, training new monitors, handling serious adverse events (SAEs), or just budgeting extra time for the coordinator to do the work demanded by the protocol. Notice that this budgeting driver is distinct from the actual reimbursement, which was not a significant driver.

	Within the technology areas, sites often struggle with multiple clunky software systems that are not integrated. On average, attendees at the focus groups had to separately log into six or seven technologies for each trial. Sites want seamless, integrated trial technologies and improving technology will have significant and substantial effects on perceived clinical trial quality. Improving technology will have a significant, though moderate, impact on trial quality.

	The performance of the monitors (b= .10, p= .02) had a slightly less substantial, but still significant impact on clinical trial quality. Sites described in the focus groups that they want monitors who can serve as a resource to help them execute trials better and more efficiently. Inexperienced monitors-that the site must train-who don’t understand the protocol and are disruptive to site operations are an ongoing frustration for sites.

	Site initiation (b= .06, p= ns), closeout (b= .05, p= ns), and reimbursement (p= -.05, p= ns) were not significant performance drivers of quality in this study. Even though closeout has a negative coefficient, it is not statistically different from zero. In understanding these results, remember that this model estimates the unique effects of each performance driver, exclusive of all the other performance drivers. So if it seems like initiation (or closeout or reimbursement) should have been significant, remember that we are estimating the isolated effects of initiation on quality, excluding the effects of communication, the protocol, budgeting, technology, and monitors.

	Sites are the foundation of the clinical research enterprise, but they have surprisingly little input in the development or planning of trials. The purpose of this research was to provide sites the ability to give their perspective of how we can improve the quality of clinical trials. The recurrent theme that we find in both the focus groups and statistical analysis is that sites are looking for partners who can help them serve their patients and conduct excellent science. Their common experience is that they often struggle to be treated fairly. 

	The results of this research may not surprise those who work with sites. This data has been anecdotally available for some time. The real contribution of this study is not only in the systematic collection of all of the performance drivers, but, more importantly, in giving weights to each of these performance drivers. In doing so, we are able to think about an ROI on site relations. 

	Suppose you are a sponsor or a CRO that wants (or needs) to improve site relations. The results of this study provide a way for sponsors and CROs to assess where they stand compared to the rest of the industry. Companies can send out a survey and compare their results to the data illustrated in this report. Based on the 

results of their survey, companies can think about the ROI on investing to improve their scores using the results of the regression model. 

	Suppose a company’s survey shows that it rated a 6.5 on a 1-to-10 scale for communication and a 7.0 on overall quality. Congratulations-not a bad score. But if one looks at our data, they will see that our sample rated an average of 7.2 for communication. This means that the organization would have room to improve in this area. Is it worth investing to improve communication? To assess this question, examine the results of the regression. Communication had a coefficient of .29. That means that if a sponsor or CRO invests in communication and is able to raise its communication rating from 6.5 to 7.5, it would improve its quality score from 7.0 to 7.29. What specifically does one look at to improve communication? Examples of the sub-drivers are found in Table 1.

	Now imagine that a company’s survey shows a 5.5 for closeout and it is compared to our findings of 7.4. The rating gap is fairly significant. Should a company, therefore, invest to improve its closeout ratings? The results of our regression model would suggest no. The closeout driver had no significant effect on quality, so the results of this study would suggest an organization’s investment would be wasted. The firm certainly might make some non-monetary investments using guidance from Table 1, but it would not improve quality by investing heavily in this area. 

	Several limitations should be kept in mind in evaluating the results of these studies. First, some might argue that the ACRP membership is not a representative sampling. Someone who belongs to a professional organization like ACRP, attends the national conference, and even volunteers for a focus group is likely to be more engaged in their professional work that the average site coordinator. While this is possible, we would also point out that these are also likely to be key opinion leaders within the profession-the very group that sponsors and CROs need to reach in order to improve clinical trial quality.

	We would also note that the model estimates are general across a variety of therapeutic areas and phases of clinical trials. Although we would argue that these estimates would generalize across clinical studies, estimates may vary around our means within specialty areas.

PA-C, PhD, is Clinical Professor, LeBow College of Business, Drexel University, email: 

 is Executive Director, Association of Clinical Research Professionals (ACRP), email: 

* Acknowledgement: The authors wish to thank Jenna Rouse, Director of Business Development, ACRP, for her assistance in conducting this study.

	1. Califf, Robert M. “Clinical Research Sites - the Underappreciated Component of the Clinical Research System.”

 Journal of the American Medical Association

	2. Rhymaun, Farouk, “Sponsor - Investigator Relationships: A Crisis of Trust.” 

; published September 23, 2014. Available online at 

http://www.appliedclinicaltrialsonline.com/sponsor-investigator-relationships-crisis-trust-life-sciences

	3. Getz, Kenneth A. “Lifting Up a Fragmented Study Conduct Landscape.” 

	4. Scholars recognize coproduction as being pervasive across all service industries. For a fuller explanation, see Zeithaml, Valerie, Mary Jo Bitner, and Dwayne Gremler,

	5. We later statistically confirmed the discriminant validity of these groupings.

	6. Parasuraman, Anantharanthan, Valarie A. Zeithaml, and Leonard L. Berry. “Servqual: A Multiple - Item Scale for Measuring Consumer Perceptions,” 

	7. Parasuraman, Anantharanthan, Leonard L. Berry, and Valarie A. Zeithaml. “Refinement and Reassessment of the SERVQUAL Scale.” 

	8. Getz, Kenneth A, “Enrollment Performance: Weighing the “Facts,” 

http://www.appliedclinicaltrialsonline.com/enrollment-performance-weighing-facts

Articles

A model to identify the services and resources sites need to conduct high-quality clinical trials.


Site Perspectives on Clinical Trial Quality

	In a recent ACT ‘What do you think?’ survey, we asked readers if a clinical trial was a product or a service. Only 53% of respondents identified a clinical trial as a service. Other responses included product (7%), neither (20%), or other (20%) with the most common response in the other category being ‘research.’
	 
	These responses are typical to what I hear at industry discussions and conference. There is a lot of ambivalence within the industry about what the value created by clinical trials – even to the point of knowing whether a clinical trial is a product or service. This ambivalence degrades value creation, productivity, and confusion about how to measure & manage quality.
	 
	A clinical trial is a service, defined as the application of knowledge and skills to benefit another entity. The value created in a clinical trial is the knowledge about a compound that will advance medicine and the commercialization of that compound. The key characteristic of services are that they are intangible and involve delivery processes. Products are tangible goods manufactured typically in a manufacturing setting. It is much easier to measure and manage the manufacturing process of products. Service processes are much more difficult to measure and manage.
	 
	This confusion about whether a clinical trial is a service or product also leads to lower-quality clinical trials and billions of dollars wasted on ineffectual quality improvement investments. You must take a different approach to measuring quality in a service as opposed to a product. Yet, the industry often measure trial quality as if it is a product – leading to mismanagement and low-quality trials. The industry has spent billions of dollars trying to improve the quality of trials in the past 15 years without any improvement to show for their effort. The fundamental reason is that the industry still hasn’t figured out if a clinical trial is a service or a product.

Most people in our poll said it was a service, but others disagreed. Find out more in this follow-up from Michael Howley PA-C, PhD and Associate Clinical Professor, Department of Marketing at the LeBow School of Business at Drexel University.

Is a Clinical Trial a Product or a Service?

Patient centricity is a popular trend with the potential to revolutionize clinical trials. By shifting from a product-focus to a patient-focus, clinical trials will become more relevant, less costly, and more impactful.

Patient centricity has the potential to be “biopharma’s Holy Grail” that will lead to a “renaissance” in clinical research.

	But will patient centricity ever reach its full potential? Skeptics are beginning to speculate that this approach is simply another fad.

 According to this view, patient centricity applies new jargon to standard clinical trial processes that result in high-visibility but low-impact activities.

 They admit that patient centricity has an intuitive appeal, but that it is based on jargon and ethical imperatives rather than empirical data.

	In order to allow patient centricity to reach its full potential and prevent it from becoming a fad, we must begin to scientifically measure its level of application in our clinical trials. Scientific measures will not only help us to understand the patient 

centricity of our trials, but will also provide a scientific basis for improving its role and clinical trials overall. Once we can measure the patient centricity of each trial, we can begin to understand how management interventions affect patient centricity levels, and provide a scientific foundation on which patient centricity can continue to develop and grow. But this vision all depends on a valid and reliable measurement. Preventing patient centricity from becoming a fad all boils down to scientific measurement of its ultimate impact.

	Patient centricity is a complex construct with a multifaceted sampling process. Despite these challenges, however, it is still possible to develop a straightforward patient centricity measurement program. In this paper, we will describe how clinical trial managers can begin the journey to scientific measurement of patient centricity with a three-step process. These three steps are based on the standard processes from the peer-review psychometric literature.

 As a result, we provide a scientific, valid, and reliable approach to measuring patient centricity. 

	Patient centricity in a clinical trial means designing and conducting a study-including the care of patients within the trial-in way that is respectful of and responsive to patient preferences, needs, and values, and ensuring that patient values guide all decisions within the trial.

 We recognize three important implications in this definition. First, patient centricity measurement is a component of clinical trial quality. Clinical

 trial quality refers to the overall excellence or superiority of the trial. We have previously defined quality and discussed at length in a previous article, “The Quality of Clinical Trials.”

 Responding to patients is an important aspect of clinical trial quality.

One can’t understand the quality of a clinical trial without measuring patient centricity, as shown in Figure 1.

	The second implication of this definition of patient centricity is that measurement must also include the backstage activities of planning and designing the trial. To what degree are patients included in decisions about the research question, inclusion criteria, endpoints, marketing materials, recruitment, data collection, and dissemination of the study results? Measuring patient centricity, as a result, requires assessment at two levels: a high-level perspective on the design of the trial and then individual-level evaluations of patient care. At both levels, we use purposive sampling techniques-not random sampling-to assess the right respondent at the right time. All of this makes for a complex sampling process.

	The third implication of this definition is that patient-centricity measurement also includes assessing the patient care aspects within the trial. Even though a clinical study is an experiment, it still involves caring for patients. It is important to include these care processes in patient-centricity measurement even though it is still a clinical trial.

	One of the unique features of patient care is that both the provider and the patient have to collaborate in order to deliver effective care. Each has an important role in both the patient care side and the clinical trial activities. Providers, for their part, must be responsive to patients’ preferences, needs, and values. Patient engagement refers to patients assessing providers as to how well the provider cared for them in a way that was respectful and responsive to their preferences, needs, and values.

Second, patients must be willing and able to do their part in the trial. Patient activation assesses how well subjects performed their role in the trial.

 Patients must have the knowledge, skills, and motivation to not only manage their health condition but also effectively participate in the trial protocol. Patient-centered medical care, then, is a relational construct in which providers engage patients but subjects also accept responsibility for their role in the care process.  

	Given this definition and implications, we offer a three-step process that trial managers can use to measure patient centricity.

	The first step in developing scientific patient-centricity measurement is to gather patient feedback about the trial. The critical success factor in this activity is to collect the right data. Many managers already gather feedback, for example, on patients’ satisfaction in their trials, and feel like they are measuring patient centricity. But patient centricity is not satisfaction-they are different constructs.

 Collecting unrelated data or undisciplined sampling adds inefficiency, is a waste of money, adds unnecessary complexity to the analysis, and generates biased information that obscures managerial judgment.

	Start your patient feedback program by collecting data on patient engagement (i.e., how responsive the providers were to patients’ needs, values, and preferences) and patients’ perceptions of the quality of the trial. There are established, valid, and

 reliable instruments that clinical teams can use to measure both patient engagement and quality. After reviewing the available instruments,

 we felt that the consultation care measure had the highest face validity for measuring patient engagement in clinical trials, described in Table 1.

	It is also important to include patients’ perceptions of the quality of the study. Since the overall service quality will influence the patient-centricity scores, it is important to know the quality of the trial in order to assess the patient centricity in an unbiased way. We recommend SERVQUAL to measure the quality of the clinical trial.

 The SERVQUAL instrument, shown in Table 2, has been in use for more than 25 years.

 The tool has been tested across multiple industries and has consistently been valid and reliable. Use established, well-accepted instruments like 

SERVQUAL to gather your data whenever possible. Why go through all the work of validating a new instrument when a precision instrument like SERVQUAL is available? 

	Patient engagement and service-quality data would typically be collected at the end of the study in order to understand how variations in patient engagement or quality affects the trial outcomes. Our goal in collecting this data, however, is to improve the quality of a current trial, not just subsequent studies. With that objective in mind, it is more useful to measure patient perceptions early in the trial (while still allowing the patient time to form an assessment), and then periodically throughout the trial. For the first assessment, the patient should have several contacts with the site or provider. After that, the intervals will depend on how often the patient is being seen. Trial participants should not be assessed more than quarterly, and stretching out the intervals will reduce the burden on patients. This approach of multiple assessments will create technical problems (i.e., auto-correlation) for the analysts, but these can be addressed in discussions with your statistical analysts.

	Patient-centricity measurement also includes an evaluation of the trial design. There are no established instruments available for assessing trial design. New assessments will need to be validated to provide scientific measurement. Fortunately, the Patient Centered Outcomes Research Institute (PCORI) has already done the groundwork of identifying all of the topics and issues that should

 be addressed within the trial-design assessment.

 In this section, we operationalize those rubrics into an instrument that can then be validated, as shown in Table 3. The rubric includes four components: planning the study, conducting the study, disseminating the study results, and adherence to the PCORI engagement principles. There are two groups that should be sampled for the trial-design assessments: patients and partners. Partners include family members, caregivers, and patient advocacy organizations.

 While it is permissible to get evaluations from individuals who have multiple roles (e.g., they are a patient in a clinical trial as well as an advocate for the cause), the sample should not rely entirely on partners with multiple roles.

	The planning-the-study assessment evaluates the degree to which patient partners are included in the conception and development of the trial. How much of a voice did patient partners have in identifying the topic, developing the research questions, and selecting the comparators to be studied? How much input did patient partners have in defining the characteristics of study participants? Finally, were the patient partners able to offer advice on how to minimize disruption to patients and incorporate study activities into their stream of activities for ongoing care? This data should be collected as the trial planning is finishing and the study is about to begin. 

	The conducting-the-study section involves measuring the degree to which patient partners participated in carrying out the study. Possible roles for the patients include participating in the drafting and revising trial materials or protocols, assisting with subject recruitment, assisting with data collection, or serving as a patient representative on a data safety monitoring board. This assessment should be timed to be administered as patients are completing the trial.

	The disseminating-the-study-results unit evaluates the degree to which patient partners help identify partner organizations for dissemination, planning dissemination efforts, or even participating in the dissemination (e.g., co-authoring an article or co-presenting at a conference). These data can be collected shortly after the initiation of the dissemination phase, since patients will have formed an impression about their involvement early in dissemination.

	Each of these three areas involve a specific stage of the trial. The fourth instrument assesses adherence to Patient-Centered Outcomes Research (PCOR) Engagement Principles (hereafter, “PCOR Principles”). The PCOR Principles include reciprocal relationships, co-learning, partnerships, trust, transparency, and honesty. Reciprocal relationships refer to the degree to which patient and stakeholder partners are considered key personnel and the degree to which they have decision-making authority in some areas. Co-learning refers to the degree to which patient and stakeholder partners understand the research process and the level to which the research team understands the patient and stakeholder perspectives. Partnership is the degree to which the contributions of the patient and stakeholder partners are valued. Trust, transparency, and honesty means that information is shared readily with the patient and stakeholder partners and they are included in decision-making. This open and honest communication extends from within the research team to the community being studied. The PCOR Principles assessment should be completed whenever any of the stage-specific assessments are administered.

	Patients have to do their part in order for a trial to be considered patient-centric. The degree to which patients are able, willing, and motivated to do their part in a clinical trial is called patient activation.

The first part of patient activation is a self-assessment completed by the study participant. The Patient Activation Scale is a valid and reliable instrument to assess patients’ perceptions of their own 

activation levels (see Table 4). The scale breaks out into four different dimensions: believes role is important, confidence and knowledge to take action, taking action, and staying the course under stress.

The “believes-role-is-important” dimension assesses whether the patient recognizes that they have a role. The “confidence-and-knowledge-to-take-action dimension” measures how able the patient is at taking the initial actions in following the protocol and caring for their health condition. The “taking-action” dimension shifts the perspective from potential ability to the patients’ ability to execute the plan. But patients must also be able to persevere with executing their plan, which is measured by the “staying-the-course-under-stress” section. 

	We recommend that patient activation be measured at the start of the trial as a baseline and then periodically across the duration of the study. The initial activation scores can be used to identify patients with low activation who would benefit from an intervention to improve their activation levels.

 How activation changes over the course of the trial will reflect the impact of the clinical site and

	Self-reports of patient activation may be influenced by social desirability bias.

 To offset this issue, we recommend that providers also assess patients using the Physician Assessment of Patient scale, shown in Table 5.

 We have adapted these items not only for the clinical trial context, but also to facilitate comparison with the Patient Activation scale.

	While there are challenges to measuring patient centricity, we have described a straightforward three-step process that managers can use to begin their patient-centricity measurement programs. It is important to realize that most of the challenges lie in the sampling process. Getting the right instrument to the right person is the challenge, but can be expedited with cloud-based administrative technologies.

	As you begin to gather patient-centricity assessments, it is worth considering how your organization will act on the feedback. If patients are 

taking the time to offer feedback, they will expect a response or some action to their input. Will your organization be able or willing to adapt their clinical trial process to what patients want? Recognize that, at a fundamental level, patient centricity is about power. How much will you let patients dictate what happens in a trial? Obviously it is an issue of balancing power, but this is new territory for the clinical trials industry. 

	Since patient centricity is a dimension of quality-a No. 1 implication of any clinical trial-then patient-centricity measurement should be integrated into an ongoing quality measurement program. Patient centricity should be thought of as an integral component of overall quality and not measured in a vacuum. We have described the scientific way to measure clinical trial quality in previous articles.

	It is important that patient centricity not be assessed with operational metrics. While operational metrics can be “easy data,” they lack validity, which means they lack meaning. To illustrate this problem, we have heard it suggested that the amount of time it takes a site to enter data can be used as a patient-centricity indicator. The rationale is that the more the clinician is focused on their patient, the more inclined they will be to enter the data as soon as the visit is completed. But what if the clinician is spending a tremendous amount of time responding to patient needs, preferences, and values? They are likely to be delayed in entering data while responding to these patient needs. This is the type of problem you get into with operational metrics. It becomes difficult to assign a rational meaning, even though the data is readily available.

	Note that many of the items described here are leading indicators, allowing managers to identify and intervene on impending failures. This means that one of the consequences of this measurement program will be to improve the patient centricity of clinical trials.

	It should be clear from this article that measuring patient centricity is not easy. It takes a lot of work in order to measure patient centricity in a way that meets scientific standards. It is worth considering the potential returns from these patient-centricity measurement investments. Since there is no widespread scientific measurement, we can only speculate about a potential ROI on measuring patient centricity. Since other authors have argued elsewhere about the ethical and patient care benefits,

 we will focus on the other potential returns. 

	The most common reason for denying new drug applications (NDAs) include issues with the selection of trial subjects and the lack of meaningful or relevant endpoints.

 As a result, measuring and then improving the patient centricity of the trial design should enhance the ability to get drug approval. Measuring and improving the patient centricity of the patient care aspects of the trial should enhance site functioning, enhance patient retention, and improve patient recruitment. Finally, patient-centric trials will better illustrate the value of the investigational drug to patients and enhance the ability to get reimbursement on the compound if it reaches the market.

	Patient centricity is an important and necessary development that has the potential to improve the quality, relevance, and value of clinical trials. We are concerned about the perceptions that patient centricity is drifting toward being perceived as a fad. To prevent this from happening, it is important that we begin to measure and manage patient centricity in a valid, reliable, and scientific way. It is critical that we get this right. Patient centricity plays a too-important role in today’s drug development model to allow it to become a fad.

	1. K.A. Getz, “The Transformative Promise of Patient Centric R&D,” 

http://clinicaltrials.pharmaceutical-business-review.com/news/the-transformative-promise-of-patient-centric-rd-070515-4571105 

	2. R. Wright, “How to Build Real Patient Pharmaceutical Companies,” 

http://www.lifescienceleader.com/doc/how-to-build-real-patient-centered-pharmaceutical-companies-0001 

	3. S. Rocchio, “Patient-Centric Clinical Research: A Renaissance in the Making, 

http://www.appliedclinicaltrialsonline.com/patient-centric-clinical-research-renaissance-making 

	4. A fad is defined as a desirable trend characterized by initial enthusiasm and energy over a short period of time that quickly fades away.  Adapted from Merriam-Webster Dictionary at 

http://www.merriam-webster.com/dictionary/fad.

	5. T. Kole, “Patient-Centric Clinical Trials: Fad, Trend, or Future?” 

http://www.almacgroup.com/wp-content/uploads/Patient-Centric-Clinical-Trials-Fad-Trend-or-Future.pdf

	6. P. Ivsen, “The Patient Centricity “Fad” and the Token Patient,” eyeforpharma, July 30, 2014, 

http://social.eyeforpharma.com/clinical/patient-centricity-fad-and-token-patient

	7. For the characteristics of business fads, see J. Hartwick, and D. Miller, “Spotting Management Fads.” Harvard Business Review, 80 (10) 26-27 (2002).

	8. For a review, see M. Furr, Scale Construction and Psychometrics for Social and Personality Psychology. SAGE Publications Ltd, 2011.

	9. Institute of Medicine, Crossing the Quality Chasm, (2001) page 40.

	10. M.J. Howley, and P.M. Malamis, “The Quality of Clinical Trials,” 

http://www.appliedclinicaltrialsonline.com/quality-clinical-trials

	11. M.J. Howley, and P.M. Malamis, “High Performing Study Startups,” 

http://www.appliedclinicaltrialsonline.com/metrics-behind-high-performing-study-startups

	12. M.J. Howley, and P.M. Malamis, “Quality Drivers in Clinical Trial Conduct,” 

http://www.appliedclinicaltrialsonline.com/quality-drivers-clinical-trial-conduct-0

	13. S. James, J. J. Hibbard, T. Agres, R. Lott, and S. Dentzer. “Health Policy Brief: Patient Engagement.” 

	14. J.H. Hibbard, et al. “Development of the Patient Activation Measure (PAM): Conceptualizing and Measuring Activation in Patients and Consumers,” 

	15. J.M. Kupfer and E.U. Bond, “Patient Satisfaction and Patient-Centered Care: Necessary But Not Equal.” 

	16. For a review of patient centeredness instruments, see C. Hudson, M. Fortin, J. Haggerty, , M. Lambert, and E.M. Poitras, “Measuring Patients’ Perceptions of Patient-Centered Care: A Systematic Review of Tools for Family Medicine, 

	17. Despite the name, this assessment provides good measurement of the patient engagement construct. There is a lot of confusion in patient-centricity terminology-both in the managerial and academic sectors.

	18. P. Little, H. Everitt, I. Williamson, G. Warner, et. al. “Observational Study of Effect of Patient Centredness and Positive Approach on Outcomes of General Practice Consultations.” 

	19. A. Parasuraman, V.A. Zeithaml, and L.L. Berry, “SERVQUAL: a Multiple-Item Scale for Measuring Consumer Perceptions of Service Quality.” 

	20. A. Parasuraman, L.L. Berry, and V. A. Zeithaml, “Refinement and Reassessment of The SERVQUAL Scale, 

	21. PCORI Engagement Rubric. patient centricityORI (Patient-Centered Outcomes Research Institute),  

. Published February 4, 2014. Updated June 29, 2015. Page 2.

	22. J. Hibbard, J. Greene, V. Overton, “Patients with Lower Activation Associated with Higher Costs; Delivery Systems Should Know Their Patients’ Scores,” 

	23. R.J. Fisher, and J.E. Katz. “Social-desirability Bias and the Validity of Self-Reported Values,” 

	24. D.H. Thom, S.T. Wong, D. Guzman, A. Wu, et. al. “Physician Trust in the Patient: Development and Validation of a New Measure.” 

	25. M.J. Howley and P.M. Malamis  “Clinical Trial Performance Measures You Can Use (and believe)”, 

http://www.outsourcedpharma.com/doc/clinical-trial-performance-measures-you-can-use-and-believe-0001

	26. MJ Howley and P.M. Malamis (2014) Metrics Malpractice, 

 Online, Accessed June 23, 2015. Available at 

http://www.appliedclinicaltrialsonline.com/appliedclinicaltrials/Blogs/How-We-Fall-into-Metrics-Malpractice/ArticleStandard/Article/detail/844223?contextCategoryId=49914

	27. D.M. Berwick, “What ‘Patient-Centered’ Should Mean: Confessions of an Extremist.” 

	28. L.V. Sacks, H.H. Shamsuddin, Y.I. Yasinskaya, K. Bouri, et. al. “Scientific and Regulatory Reasons for Delay and Denial of FDA Approval of Initial Applications for New Drugs, 2000-2012.” 

, PA-C, PhD, is Associate Clinical Professor, LeBow College of Business, Drexel University, email: 

, MBA, is CEO, CRO Analytics, LLC, email: 

This three-step process can help managers accurately assess how patient-centric their trials really are.

Preventing Patient Centricity from Becoming a Fad

The clinical trials industry has demonstrated a commitment to improving the quality of clinical trials. Over the past decade, a variety of quality initiatives have emerged and billions of dollars have been invested to improve the quality and performance of clinical trials. New quality management techniques like risk-based monitoring and Quality by Design (QbD) have also become popular. Despite these efforts and investments, however, there has been little impact on the quality of clinical trials.

	We believe that these quality efforts and investments have had limited impact because we have been using the wrong quality measures. It is common practice, for example, to use manufacturing quality indicators like the number of days to recruit or the number of queries during the trial. But these don’t measure quality – one is a measure of time and the other is a defect rate. These indicators are performance drivers that may be related to quality. Quality is a different construct that must be measured separately. And since a clinical trial is a type of professional service, it is important to measure the service quality of the clinical trial. Once the service quality has been established, the other performance metrics can then be related to clinical trial service quality.

	While measurement may seem like a mundane detail, it is a critically important issue. First of all, regulators “…require sponsors to monitor the conduct and progress of their trials..." that should continue “… on a continuous basis throughout the design, conduct, evaluation, and reporting of the trial.” If you are not using the right measures in your oversight, then you can’t properly and efficiently monitor your trial. Perhaps more importantly, failure to detect impending quality failures delay trials and are costly. Unless the industry adopts valid and reliable measures of quality and critical performance drivers, then they will not be able to improve the quality of clinical trials.

	The purpose of this research is to assess the quality within the conduct of clinical trials and then identify the key drivers of performance within study conduct and relate them to quality. In doing so, we identify the performance metrics that have the greatest impact on clinical trial quality.

	We focus on quality within the conduct of the trial (as opposed to study startup or closeout). To measure the service quality of the conduct of the trial, we used a single-item global indicator. Since it is common practice within the industry to use operational metrics as proxy indicators for quality, we had to adopt an exploratory posture to identify the performance metrics we should assess. First, we sought to create a comprehensive list of performance drivers. We interviewed a series of clinical trials managers over the course of 18 months and asked them to list all of the performance activities that impact the quality of trial conduct. Both sponsors and CROs were included in our sampling. We continued to interview managers until we were no longer identifying new quality drivers. In all, thirty-six interviews were conducted. Respondents identified seven general areas that were important to conduct quality, including: adaptability, adherence, enrollment, functions, monitoring, project manager, and site relations.

	Second, we then performed a quantitative analysis to distill all of these measures down to the essential indicators and calculate the magnitude of the relationship of each indicator to conduct quality. To do this, the items were loaded into an online survey tool. All of the items were edited for clarity and grouping. Sponsor and CRO trial managers were solicited in a purposive sampling to complete the survey, excluding respondents from the qualitative phase. 

	We received 82 usable surveys with 37% from Phase II trials, 46% from Phase III, and 17% from Phase IV. The average number of sites was 74 with an average 558 subjects.  

	We used the statistical program SmartPLS 3 to identify the essential indicators of conduct quality. Missing data was less than 5% of all variables except for data management, protocol amendments, and project management with less than 10% missing data. For these cases, the missing data points were imputed. All of the items had acceptable univariate and multivariate normality. All variables used a 1 to 10 scale, with the exception of the covariates (phase, sites, subjects, and a dummy sampling variable).

	Within the seven important areas identified by managers in the first qualitative phase of our research, adaptability was a formative construct consisting of protocol amendments, change order processes, managing protocol violations, and resolving queries. Only protocol amendments (

 .01) were significant and only these two items were included in the adaptability index. Adherence was a reflective construct assessed by adherence to both the study protocol (

 .001) and the medical management/safety plan (

 .001).  Enrollment was constructed as a formative indicator and was assessed by evaluating the performance of the clinical study team on enrolling patients that met the criteria and keeping you up-to-date on the enrollment process as well as timeliness in first site, last site, first patient, and last patient. Only adhering to the timeline for enrolling the last patient (

 .001) was significantly related to enrollment performance and so was used as the enrollment indicator. Performance on the functions was structured as a reflective indicator and included project management (

 .001) and were all scaled into the latent functions construct. Performance of the project manager, site relations and routine monitoring visits were all measured as a single global indicator.

 All of the latent constructs had reliabilities > .89, discriminated from each other, and established acceptable validity.

	Once the measurement model was established, we examined the magnitude of the relationships between the performance drivers and conduct quality in a regression equation model using SAS 9.3. All variables were mean-centered prior to estimation, so the coefficient describes the relationship of the predictor on conduct quality at the average of all the other factors.

 .001) and explained a substantial amount of the variance (

 = .80) of conduct quality. The means, standard deviations, and correlations of the variables used in the statistical analysis are shown in Table 1.

	The relationship between the performance activities and conduct quality are illustrated in Figure 1. The magnitude of the coefficient describes the strength of the relationship between the driver and conduct quality. The coefficient for adaptability (

 .04) is interpreted, for example, that a 1 unit increase (on a 1 to 10 scale) in the adaptability of the study team improves conduct quality by .20 at the average levels of adherence, enrollment, functions, monitoring, project manager, and site relations. Given the scaling, you could also describe this as ‘a 10% increase in study team adaptability improves quality by 2%.’

	There was a negative but non-significant relationship between adherence (

 .31) and conduct quality. While the coefficient estimate is negative, the non-significant coefficient means that we could not identify a relationship between adherence (to the medical management, safety plan, and study protocol) and conduct quality.

 .02) as positively and significantly related to conduct quality. Improving enrollment by 10% increased conduct quality by 2% at the average levels of all the other conduct quality drivers. The various functions (

 .22) involved in a trial did not improve quality. The project manager (

 .001) had the greatest impact on project quality. Improving project manager performance by 10% increased conduct quality by 3½%.

 .002) had the second greatest impact on conduct quality. The routine monitoring visits (

 .06), however, had a negative and marginally significant impact on project quality. There was not a significant (

 .51) interaction between site relationships and the routine monitoring visits. Regardless of the status of site relationships, the routine monitoring visits degraded conduct quality.

Adopting scientific quality measurement that recognizes that clinical trials are a service will reap benefits.

Michael J. Howley

Articles

Site Perspectives on Clinical Trial Quality

Is a Clinical Trial a Product or a Service?

Preventing Patient Centricity from Becoming a Fad

Quality Drivers In Clinical Trial Conduct

Quality Drivers In Clinical Trial Conduct

Properly Assessing Quality in Clinical Trials

The Quality of Clinical Trials

The Metrics Behind High Performing Study Startups

How We Fall into Metrics Malpractice

The Next Generation of Clinical Trial Performance Measurement