A Signal Isn’t Resolved Until It Stays Closed—RBQM Lessons

In a risk-based quality management approach, signals are structured prompts to check whether patient safety, data integrity, or trial conduct may be at risk. This aligns with ICH E6(R3) and current regulatory expectations for proportionate, lifecycle oversight focused on what is critical to quality. Practically, this requires signal processes that are both efficient—enabling timely, defensible decisions—and robust, so closures don’t need to be repeatedly revisited.

Using CluePoints data from 2022 to today across 881 studies, we assessed two practical measures of signal performance: how long it typically takes study teams to investigate and reach a first closure decision, and how often a signal is reopened later—reflecting how durable that first decision is over time. In this analysis, a signal represents a single-issue type that may apply at patient, site, country, or study level. Signal statuses (Open/Closed) are set by study teams, and reopened means the study team determined the same issue warrants new investigation or action.

We compared two complementary signal approaches. Statistical data monitoring (SDM) signals arise from broad, data-driven review of trial data, including unexpected patterns—not limited to predefined risks. Key risk indicators (KRI) signals track risks selected by study teams and may include both standard and study-specific KRIs. Investigation time is broadly similar across both signal types. SDM signals reach a first closure decision in a median of 67 days (interquartile range: 31–131; N=20,545), compared with 61 days for KRI signals (interquartile range: 31–119; N=56,398). A slightly longer timeline for SDM was expected: SDM signals can surface unanticipated issues that require deeper root-cause exploration, whereas KRIs focus on anticipated risks with typically clearer drivers. The main difference is durability—only 1.1% of SDM signals are reopened versus 11.2% of KRI signals (about ten times higher) (Table 1). Reopens are not inherently negative—they can reflect good oversight as issues evolve. The higher reopen rate for KRIs also fits their operating cadence: KRIs are typically reviewed monthly, while SDM reviews occur every 3-6 months, so KRIs have more opportunities for trend drift to reappear.

Practically, aim for evidence-based closure using a short, repeatable checklist:

• Validate in context: rule out data timing/completeness effects, planned milestones, or expected study/site characteristics.
• Pinpoint the driver: data/process/equipment artifact, site performance issue, or expectation/design issue.
• Act proportionately (if needed): assign owner + due date, and define objective evidence of completion.
• Close with clear criteria:
  • Non-actionable: documented rationale and no meaningful Critical-to-Quality impact.
  • Controlled: root caused identified and confirm improvement over ≥2 review cycles (or a defined observation window).
• Set reopen triggers: recurrence beyond expected variability, persistence past the stability window, new data changing the conclusion, or clustering with related signals.

Overall, these findings support a simple takeaway: the opportunity is not primarily to close signals faster, but to close them more consistently and defensibly. In practice, durability improves when teams use thresholds as decision support, apply explicit evidence-based closure criteria, and add stability expectations for frequently reviewed, trend-sensitive signals such as KRIs. This reduces unnecessary reopen cycles while ensuring evolving risks are appropriately re-assessed.

Sylviane de Viron, Data and Knowledge Manager; and Melissa Thomas, PhD, RBQM Lead; both with CluePoints