Uncovering Enrollment Underperformance via Business Analytics

The Emergence of Business Insights
Uncovering challenges associated with clinical trial enrollment has been an ongoing issue ever since the inception of clinical research. More recently, nonetheless, many business insight services claim that they could resolve the issue of identifying enrollment underperformance through offering advanced charting, visualizations and statistical analysis.

Unfortunately clinical operations personnel have been pressured into interpreting these insights and turning them into actionable business solutions.  This article will provide an anonymized example from a Phase III trial around identifying enrollment underperformance through insight interpretation & analysis and statistical regression models.

Enrollment Performance Assessment
Whenever implementing data analysis, it is important to first define research objectives; establishing clearly defined research objectives not only save study teams time, but they also allow study teams to implement meaningful business and clinical trial performance strategies.  The objectives of this performance assessment were to (a) identify underperforming sites and (b) uncover strategies that outperforming sites were implementing to maximize the success of enrollment activities across all sites.

The first step was to perform a regression analysis on two factors involving enrollment performance, which included subjects screened VS randomized.  In other words, we wanted to identify whether screening subjects had any statistical effect on randomizing subjects.  Table 1 demonstrates the results of the regression analysis.

With a T-Stat figure of 33.224 and a T-Critical figure of 1.969, we can determine that T-Stat > T-Critical and we can reject the null hypothesis (H0).  This means that there is a statistically significant relationship between screening and randomizing subjects; each subject screened is equivalent to a 0.463 subject randomization rate.  In other words, a site will need to screen approximately 2 subjects in order to randomize 1 subject, and since this figure is statistically significant, this rate applies towards all sites in a normally distributed scenario with 95% confidence.

Correspondingly, we leveraged customized clinical trial business data and insights to construct the model illustrated in Figure 1.

Figure 1 demonstrates the concentration of subjects screened VS randomized by study site in the form of a plot.  Darker/bolder circles represent higher site concentrations, whereas fainter circles represent lower concentrations.  For example, the first vertical line of circles on the bottom left hand side of the graph suggests that many sites (especially on the bottom) screened less than 10 subjects and did not randomize any subjects. 

The orange line/area on Figure 1 represents the statistically significant randomization range/slope of 0.463 subjects randomized per 1 subject screened (normalized characteristics). Circles below this line represent outperformance, whereas circles above this line represent underperformance. Outperforming sites are outlined in green ellipses, whereas underperforming sites are represented by red ellipses.  To illustrate, you will notice that there is one circle that lies within the green region (indicated by a blue arrow), descriptively, 8 screened versus 8 randomized rate, which is a 1:1 ratio instead of a 1 : 0.463 ratio. 

Turning Underperformance Identification into Effective Business Solutions
Since we have identified underperforming and outperforming clinical trial sites, it is important to understand factors that affect site performance other than screening patients. What factors are causing sites in the outperforming (green) region to perform well? Which factors are causing sites in the underperforming (red) region to perform poorly?  Is outperformance associated with medical chart reviews? Familiarity with the protocol’s inclusion/exclusion criteria?  Is underperformance a result of a lack of training? Site/CRO staff turnover? Is site budget a factor?  Simply ignoring the notion that no other factors have any influence on site enrollment performance to outperform/underperform is a dangerous assumption to make.  Henceforth, it is advised that the study team implement additional research initiatives in order to unveil potentially unseen factors.

An effective method involves in-depth focus group and individual interview research with study sites & CRAs, and correspondingly, surveying study sites in order to quantify and verify factors affecting outperformance/underperformance. Subsequently, the study team could leverage research findings in order to influence underperforming sites to perform more optimally, and average sites to outperform (ultimately try to push the slope of the orange line downwards, and the screened/randomization ratio higher).

Expert site communication services and continual analytical performance assessments are essential operational activities in order to determine the effectiveness of the initiatives.  This customized assessment and interpretation analysis is merely one of many that could be conducted in order to identify study site trial activity and enrollment underperformance.

Summary
The emergence of business insights has allowed clinical operations personnel to access valuable visualizations in order to assist with performance identification. However, without statistical, business, financial and market research expertise, clinical operations personnel can easily misinterpret these visualizations and execute suboptimal business decisions.

You can contact Moe Alsumidaie at [email protected] if you have any questions or are interested in accessing these tools.