Jeffrey Helton

Clinical trials represent an opportunity for sponsors and investigators to further medical knowledge and bring new interventions and discoveries to market. Regulatory standards and shorter trial deadlines present challenges for trials, but equally as important is the financial risk to the organizations involved. In today's market where a contract between sponsors and service providers is either based on a fixed fee or per enrollment, it is critical to get the study budget right.     

PHOTOGRAPHY: COMSTOCK IMAGES, GETTY IMAGES ILLUSTRATION: PAUL A. BELCI			

It is well known that research organizations operate with paper-thin margins, and many studies operate with operating losses "because of unexpected expenses."

  These unexpected expenses can result from recruiting additional patients later by opening new sites late in the process, only after required enrollment fails to meet project requirements. This can result in time delays, which will add expenses and extend the trial longer than originally anticipated. 

Delays are quite common, with well over half of all clinical trials in the United States missing completion deadlines.

  Variability between budget projections and study realities can make or break the financial outcome of a project in the participating organization. 

Development managers and investigators often overestimate the amount of subjects in any given time period, and underestimate costs. This difference between a projected value and an actual one represents the variability, or change. For project managers, variability and risk represent a roadblock to hitting the trial's budgeted financial performance in the long-run, which also complicates contract negotiations. 

So, how do project budgets include a reasonable allowance for variability in the accrual and retention of study subjects and the unexpected costs that can come along in the normal course of a study while not discouraging funding sources? One way is to build simulation models that can cope with uncertainty by generating thousands of scenarios, which helps give a more realistic picture of expected trial outcomes.  

The variety of costs involved in a clinical trial must be closely considered to ensure that adequate allowances are made for the usual costs of clinical research and to scrutinize the deviations in those things that drive research project costs (e.g., regulatory, quality, protocol development). Since clinical trials are a dynamic endeavor with multiple moving parts that directly impact payments and prices, managers have a lot of variables to consider when establishing a budget. 

It is the dynamic nature of so many expense components that creates the risk of understating project budget expenses and overstating potential margins from a contract proposal. Perhaps the most critical assumption underpinning the trial budget is that of the accrual and retention of trial subjects. Missed patient enrollment assumptions can cascade into missed projections for recruitment, additional supplies, and testing associated with replacing subjects in a study, or support of additional periods of overhead as a project goes beyond expected time frames.  

Table 1. The key assumptions of the simulation model should be clearly articulated.			

One of the most common ways to build a clinical trial budget is to take a historical average of expenses from cost accounting systems, or to develop a "rule of thumb." Historical averages are just that, averages, which vary above or below some value. Variability in the numbers making up an average can be risky. The average of 45 and 55 is 50, just as it is for 30 and 70. However, if a budget projection using an average value of 50 uses a historical basis of 30 and 70, the organization may be in for an unpleasant surprise if the actual result nears either of those two extremes. Knowing the spread and historical frequency should be taken into account in a comprehensive budget projection. 

Additionally, there are a number of different factors that can influence the amounts expended on a trial that can render even the best estimate arrived at in this deterministic fashion inaccurate. Clinical project managers need to understand the variety of costs in a proposed study, and the factors or drivers that influence study costs.  

Table 2. Project managers must closely monitor key assumptions, such as subject recruiting, and modify budgets accordingly.			

Using probability to create a range of estimates from high to low, planners could build realistic cost variability into estimates and reduce the degree of risk of a projection error. One way to do this is with Monte Carlo simulation models. 

Monte Carlo simulation uses a random number generator to assign different values to the variables that make up a projection. The conditions that make up a budget projection can be repeated multiple times, using different randomly generated input values to create a range of possible outcomes. Considering the spread of results from those multiple iterations of a budget model that's been compiled with variations of inputs gives managers a better idea of a future projection that takes into account the multiple scenarios of variation that can happen with multiple budget components. 

While this may appear to be a task suited for an organization with high-powered computing resources and specialized software, in this case, looks are deceiving. This sort of projection can actually be handled with a common personal computer spreadsheet and a low-cost, add-in software package.  

Figure 1. Computer simulation results offer budget managers mutliple financial scenarios and likely outcomes in only minutes.			

Following is an example of how Monte Carlo simulation can be applied to a clinical trial budget projection using a spreadsheet and an add-in package, in this case, the Oracle Crystal Ball software, although many others are on the market today, such as Palisade's @Risk or Frontline Systems. 

In this example we will use somewhat simplistic assumptions from the perspective of a service provider or CRO to show the concept, considering only basic expense, patient enrollment, study cycle, and site assumptions. Since CROs and service providers need to ensure that the sponsor reimburses at a level that maintains margins over costs, this example will highlight how average costs based on historical estimates can be used to build a budget. Finance managers, on the other hand, would focus more on volumes, timing of payments, and cost controls. But this example demonstrates conceptually how simulation models can be utilized. 

The simulation model requires three types of assumptions: input variables, decision points, and forecasts. Decision points are the variables the budget planner is trying to determine—in this case, setting price for a contract negotiation. Forecasts are the outputs of the model, or a projection of the outcomes. Here we're using study operating margin. 

Using the historical averages from Table 1, the budget manager can determine that total costs per patient are $1000, and that total fixed costs are $1.1 million. Therefore, minimum volume of patients is 1110 (FC/expected price less variable costs). Based on this estimate, the 1200 patients expected in the study create a margin of over 4%—revenue is $2000 x 1200 for $2.4 million, and expenses are $1.1 million fixed + $1.2 million in variable. 

Now, let's assume that this projection could vary in several respects. Specifically, the number of patients enrolled and number of sites. These are typically estimated based on educated guesses about how many patients will enroll and when, both of which can significantly impact bottom-line projections. Since forecasts are usually wrong, either positively or negatively, we need to incorporate a range of estimates. 

We will assume these figures have normal distributions, where there are an equally likely number of estimates on both sides of the average estimate. Table 1 shows the assumption was 1200, but your project manager has suggested this could be off around ±10%, either way, or around 240 persons in total. One standard deviation represents a 20% change from the mean, a projection using just that mean has about a one in three chance of being off by more than 20%. 

Of course, if you didn't know a range of estimate but you felt that the choices were limited to two or three estimates, a triangular or binomial distribution can also be used. For example, with the second assumption, let's assume that the number of sites needed to complete the trial is 50, but could either be 45 minimum, 48 most likely or 50 max. Monte Carlo simulation allows for this triangular range of estimates and distribution patterns. One other element that can have variability built in to it for this projection is the pricing charge or revenue per patient, which are the costs to participate for a sponsor or other participant. For this example, assume the range was bound between $1500 and $2100. 

Once we have defined these variables and estimated a distribution or variance for the key assumptions, we run a simulation involving several thousand iterations, which takes less than two seconds. Based on the simulation, the new average of overall study margin is projected to fall within a range of <$200,000> and +$400,000, with an average of $98,820, which represents a 3.6% operating margin (see Figure 1). 

Budget managers could interpret this in two ways. The first is that the $98,820 average is not much different than the $100,000 originally budgeted. But on closer reflection, the better way to interpret this is that there are a lot of potential outcomes, each of which can be expected given a slight change in the assumptions. For example, as shown in Table 2 in the fourth scenario, if the number of patients enrolled falls to just 1096 (versus 1200) the study margin will experience a loss of $3751. In other words, an 8.6% decrease in patients results in a >100% change in study margin. 

This highlights the importance of understanding each scenario, and with dozens of variables all changing simultaneously, this is only possible through simulation modeling. 

Clinical trials are a dynamic undertaking, demanding the most of trial managers to maintain an organized project that delivers clinically valid results in a financially viable setting. In this article we demonstrate an example from the service provider side, where margins are highly influenced by small changes in the key assumptions, and developing sophisticated budgets involves dynamic analyses. 

Monte Carlo simulation models offer one potential solution for the trial manager to cope with this uncertainty. A budget manager that simply observed the budget estimate originally prepared in Table 1, would have run a high risk of financial loss since the static, deterministic budget model used did not take into account the wide variability of underlying assumptions. Using a Monte Carlo simulation, managers can see the risks in the assumptions that support a budget and adjust their actions and expectations accordingly. While no estimate is guaranteed to be perfect, having the ability to model the variation that comes in the dynamic world of the clinical trial puts managers at a higher level of sophistication in the understanding of their business and at a greater chance of achieving the financial goals of the trial. 

1. J. Lacera, "Here's How to Improve Study Budgeting and Policies," 

2.  J. Rowe, M. Elling, J. Hazlewood, R. Zakhary, "A Cure for Clinical Trials," 

3.  J. Adelgass, "Identifying the Money Pits in Conducting Clinical Trials," 

,* PhD, CMA, is associate director at the Fleming Center for Healthcare Management, the University of Texas Health Science Center, 1200 Pressler, RAS E-341, Houston, TX 77030, email: 

, MS, CFE, is research assistant at the Fleming Center for Healthcare Management, University of Texas School of Public Health. 

, MBA, MPA, is director of research finance at US Oncology Research in Houston, TX.

*To whom all correspondence should be addressed.

Articles

With a PC and some software you can generate realistic projections that save time and money.

Jeffrey Helton

Articles

Financial Simulation Modeling