Data Monitoring Committees in Practice

Tips on using DMCs to improve trial efficiency and safety
Mar 01, 2004

Subject safety in clinical trials has received much recent media attention, most publicly because of the death of a young man during a gene therapy study at the University of Pennsylvania.1 Some large and important clinical trials have been halted before their planned conclusions for safety and other concerns (for example, the Women's Health Initiative).2

Data monitoring committees (DMCs, sometimes called data and safety monitoring boards or DSMBs) have played an important role in providing oversight to critical clinical trials. A DMC is an independent group of experts who monitor unblinded safety and efficacy data while a trial is ongoing. They can recommend making changes to the conduct of the trial, including recommending to terminate the trial early for safety concerns, for overwhelming treatment efficacy, or for demonstrated futility in being able to show a benefit of treatment. In November 2001, the U.S. Food and Drug Administration (FDA) published draft guidance on DMCs3 explaining their need and how they should be integrated with clinical investigations. An extremely useful text by Ellenberg, Fleming, and DeMets covers many of the topics in this article.4

When is a DMC called for? Not all clinical trials need a DMC. The draft guidance published by the FDA3 provides some insight in deciding if a DMC is needed for a trial:

  • DMCs have generally been established for large, randomized multisite studies that evaluate interventions intended to prolong life or reduce risk of a major adverse health outcome such as a cardiovascular event or recurrence of cancer. Because monitoring of accumulating results is almost always essential in such trials, DMCs should be established for controlled trials with mortality or major morbidity as a primary or secondary endpoint. They may also be helpful in settings where trial participants may be at elevated risk of such outcomes even if the study intervention addresses lesser outcomes such as relief of symptoms.

Figure 1. When a DMC is needed. (Adapted from Ellenberg et al., [4], p. 160.)
DMCs can also prove useful in clinical trials where mortality or major morbidity is not the primary endpoint. Figure 1 provides a diagram to help determine when a DMC is appropriate.4

The DMC is in the unusual position of being able to review unblinded study data while the trial is ongoing. Using prespecified monitoring guidelines, the DMC might, based on the data it sees, recommend stopping a study early. In general, there are three scenarios under which a DMC might recommend stopping a trial:

  1. The data sufficiently demonstrates efficacy at such a high level of significance that it is very unlikely that if the trial were to continue the results would change
  2. There is a safety concern important enough to warrant changing or stopping the study
  3. There is sufficient data to indicate that even if the trial were to continue to its conclusion, it is extremely unlikely that the new treatment would show a statistically significant benefit (so-called futility curtailment or stopping for lack of efficacy).

Stopping a trial for futility can save subjects from unnecessary risk and can represent a substantial savings of the sponsor's money and time that would otherwise have been spent completing a trial highly unlikely to produce a positive result.

A DMC will always monitor for subject safety; however, the monitoring boundaries for overwhelming benefit or demonstrated futility must be specified to the DMC in advance, and are at the ultimate discretion of the sponsor with input from the DMC as well as the FDA or other regulatory agencies. Choosing appropriate monitoring boundaries is an important strategic decision for the sponsor. Furthermore, regulatory agencies should be informed about the use and responsibilities of the DMC and of the monitoring boundaries to be used.

As the DMC monitors for safety, it takes into account all available information. This might include, for example, the results from toxicology studies, or results from other clinical trials using the same class of drug. The DMC will need to carefully assess the risk-benefit balance. Because there are no set rules or guidelines for making decisions and recommendations regarding safety, this is an area where the expertise and judgment of the DMC members is critical.

The timing of DMC meetings is typically based on the number of patients with endpoint data; calendar time may also be a factor. DMC meetings might be convened based on some threshold, for example occurrence of more than one Serious Adverse Event, or more than two for a cancer trial since some level of side effects is tolerable. Again, the specifics of the disease and the treatment need to be weighed in setting up such guidelines.

The role of the DMC is advisory to the sponsor and study leadership. The sponsor (and possibly the study leadership) has ultimate responsibility for making decisions regarding the trial. As a trial is organized, it is important to consider to whom the DMC will report their recommendations. In many cases, a steering committee led by the sponsor is responsible for the conduct of the trial. Because the DMC may recommend major changes in the study, someone on the steering committee needs to have the authority to accept and implement the DMC recommendations.

Interim monitoring boundaries Interim monitoring boundaries provide guidance to the DMC by defining the threshold beyond which the DMC might recommend stopping or altering a trial. They must be designed and agreed upon prior to the first "look" at the data. It is a nontrivial statistical task to design these boundaries, given that the DMC will review the data multiple times over the course of the study.

Figure 2. Sample boundary.
With repeated looks at the data, the observed p-values at each look cannot be compared using traditional measures of statistical significance (say, p < 0.05 or 0.01). The more often one looks at the data, the more likely one is to see a significant p-value totally by chance. Therefore, a higher level of statistical significance (smaller p-value) is needed earlier in the trial. An example of a possible boundary is shown in Figure 2. The "Normalized Z-Score" is a measure of the benefit or lack of benefit of the new treatment compared to the old (either placebo or standard) treatment. A larger Z-score indicates greater benefit, and a smaller (or negative) Z-score indicates lack of benefit or even harm. Thus, a Z-score of zero indicates neither benefit nor harm from the new therapy. If the observed Z-score crosses the top boundary of the "funnel" shown in Figure 2, then the DMC could recommend termination for overwhelming benefit. If the observed Z-score crosses the bottom boundary of the "funnel," then the DMC could recommend termination for demonstrated futility or lack of benefit.

This example presumes that the maximum enrollment would be 1,200 participants and that the DMC will meet after every 200 participants to review data.

These boundaries can be modified to fit the desires of the sponsor, DMC, and regulatory agencies. The shape of the boundary determines how conservative the DMC will be in making its recommendations. For example, a sponsor may be pursuing multiple indications for a single agent. One indication will, potentially, have a larger market than another. The sponsor may design the boundaries for the less marketable indication so that it is more likely that the study would be stopped for lack of benefit (to cut losses sooner). Alternatively, the sponsor may want to collect as much data as possible. In this case, the sponsor would only want to stop the trial in the case of demonstrated harm to subjects, but would otherwise continue even in the presence of results that are highly positive or that have little chance of a final positive result.

Figure 3. Sample boundary with observed interim results.
The particular sample boundary shown in Figure 2 is an Emerson-Fleming symmetric boundary.5 An interesting feature of this particular boundary is that at the halfway point of 600 participants, a Z-score of zero or lower (no observed difference or a negative trend) would trigger a DMC recommendation to halt for demonstrated futility. If the trial went all the way to the end (n = 1200), the critical Z-score for statistical significance would be 2.02. This Z-score corresponds to a one-sided p-value of 0.022, very close to a traditional one-sided p-value of 0.025. This slightly higher hurdle at the end of the trial is indicative of the relatively small statistical price paid for the advantage of potentially stopping the trial with fewer participants.

The statistics allow considerable flexibility in scheduling when the DMC will look at the results. For example, if the first scheduled analysis was after 200 subjects had been enrolled, but there was a safety concern that prompted the next analysis at 300 instead of 400 subjects, the boundaries could be adjusted to take this into account.

Figure 3 depicts a hypothetical trial with hypothetical results. At the first look (n = 200) there is basically no difference in the two treatments. Results look promising at the next two looks (n = 400 and 600), and at the fourth look (n = 800) the results have crossed the threshold for overwhelming benefit. Assuming there were no other issues that the DMC was concerned about (e.g., side effects that warranted additional monitoring), the DMC would issue a recommendation to terminate the trial early for benefit.

lorem ipsum