WCG Metrics Champion Consortium QTL Working Group

of our guidance for quality tolerance limit (QTL) use in clinical research, we summarized the WCG Metrics Champion Consortium (MCC), now part of the 

, QTL Working Group discussions that explored the relationship between QTL and key risk indicators (KRIs). In 

 of the series, we delved into the process of defining QTLs. In this final section, we will focus on methods for the early detection of risk. We then offer a summary plus an appendix featuring a worked example.

QTLs are designed for risk control (see section 5.0.4 of ICH E6 (R2)).

 A key element of effective risk control is the earliest possible detection of risk to enable proactive interventions that prevent or minimize the impact of issues that matter. Unfortunately, as described in Part 1 of the series, the more sensitive a measurement is to risk early in the trial, the more likely it will provide false signals. As deviations from the QTLs (signals) need to be reported in the clinical study report (CSR), this is an effective disincentive to use QTLs that will provide an early signal. TransCelerate Biopharma

 proposes using a “secondary limit” to overcome this challenge. The secondary limit is typically 50-75% of the QTL threshold. It is an “early warning” system to alert the study team that the threshold might be exceeded in due course. It provides an earlier signal of risk that does not need to be reported in the CSR.

Figures 1a and 1b below provide examples of using a secondary limit (shown in yellow) along with the QTL threshold (shown in red) with two QTL parameters proposed by TransCelerate Biopharma.

 The secondary limit is set at 75% of the QTL threshold. Rather than waiting for the QTL threshold to be exceeded at week 21, there is an early warning that the secondary limit is exceeded at week 18. Three weeks’ additional warning may be enough time to take action and avoid the QTL threshold being breached.

Figures 2a and 2b below show the same data but with the secondary limit at 50% of the QTL threshold rather than 75%. Here the secondary limit is exceeded at 17 weeks, or 10 weeks, depending on the QTL parameter being used. Many organizations encounter a challenge when using this approach to determine the appropriate secondary limit. The 50% secondary limit provides an earlier signal, but a decision is needed as to whether the signal is noise or worthy of investigation. If the rate continues, will the QTL threshold itself be exceeded before study end? In these examples, we show data to study end, so we know that the QTL threshold is exceeded. However, earlier in the study, week 13, for example, how would you determine whether there is a systemic issue or whether exceeding the secondary limit is a signal worthy of further investigation?

As we review the figures above, clearly, the parameters are both tending to increase over time. For the number of withdrawals, this is no surprise, as it is a cumulative measure. But why is the % withdrawals parameter tending to increase over time? This is because the 

 for a participant to withdraw increases the longer they are on the trial. A participant who has been on the trial for seven weeks has had seven weeks to potentially withdraw, whereas a participant who has just been enrolled has had very little opportunity to withdraw.

Figures 1b and 2b, which depict results for a parameter normalized by the number of enrolled participants

 have values closer to the QTL threshold (and the secondary limit) much earlier than the simple count of withdrawals in Figures 1a and 2a. Other normalization factors can be used to take account of situations where the 

 increases over time. They can provide an alternative approach to the use of the secondary limit.

Metric definition and impact on early signal detection

The way a metric (whether a KRI or a QTL) is defined can profoundly affect how early a signal is detected. Some organizations employ complex statistical models, e.g., Bayesian statistics or funnel plots, to allow for early data variability and the ability to extrapolate impact from early views. A simpler but arguably less precise approach is to change the denominator for calculating a metric. This approach can help provide a much earlier signal of risk, which can lead to further detailed analysis as required. There is more information plus examples of defining parameters to provide early signals in a WCG white paper “Measure the Right Things at the Right Time: Design KRIs and KPIs that Provide Timely Insights.” Key points from the paper:

The normalization of a metric (the denominator) can impact whether a metric is a leading or lagging one. Selecting the right normalization factor can mean a potential issue is detected weeks or even months earlier.

If the opportunity of an item being measured (e.g., participant withdrawal) increases the longer the available time, then the leading metric should take this time element into account

Normalization of metrics used as QTLs and KRIs

 propose two possible QTL parameters to monitor different risks to the critical quality factor “withdrawal criteria and trial participation retention.” (Table 1)

The data for these are readily available, and teams are likely to establish quality tolerance limits based on the study design and statistical analysis plan. It is critical to the quality of the study that the QTL threshold for these parameters is not exceeded by study end. As demonstrated above, using the QTL threshold with these parameters does not provide an early signal that the QTL might be exceeded by study end. From an operational perspective, though, during the study, it would be preferable to have an early signal that if the trend continues, the QTL parameter is likely to be exceeded by study end.

participants with withdrawal of informed consent

 is more likely to occur the longer participants are in the study. Early in the study, there are likely to be few participants who have withdrawn informed consent. However, as participants are in the study longer, they have more opportunity to withdraw informed consent. Similarly, this logic can be applied to the QTL parameter 

 A leading parameter can be defined with a normalization factor that takes this time element into account. A normalization factor such as participant-week or participant visits helps adjust for the increased opportunity for withdrawal of informed consent or loss to follow-up over time. (Table 2)

 Critical-to-quality factors and parameters

Modifying the normalization factor can mean dispensing with the secondary limit. A potential issue is detected weeks, or even months, earlier—allowing investigation and mitigation actions to be implemented sooner. This is demonstrated with an example in the appendix following the end of this article.

The Consortium QTL Working Group proposes an alternative to using a QTL parameter with a secondary limit. It is an approach that helps overcome the challenge posed by the balance between obtaining a signal of an emerging issue in time to act and the CSR reporting requirements. The alternative is to use a leading parameter such as that described in the previous section 

 to the QTL. The leading parameter can be considered an operational KRI used as a companion to the QTL. This way, the easily measured QTL parameter and threshold can be used to provide a signal where the quality objectives are not, or are unlikely to be, met. However, the companion KRIs can provide an earlier signal that, if the study continues as it has, the quality objectives will not be met.

As stated in the previous section, the leading parameter companion KRI should be defined to give an early signal of whether, if the current rate continues, the QTL threshold is likely to be exceeded by study end. The approach of using companion KRIs can be expanded further. Companion KRIs could be defined at other levels such as country and site. Using this approach allows the QTL parameter and threshold to be kept in the easy-to-interpret form, e.g., the number of withdrawals, and the operational companion KRIs providing an early signal. Investigation and action might be needed for the QTL threshold not to be exceeded. The approach is shown diagrammatically in Figure 3 below.

In addition, the companion KRIs would be available to help inform root cause analysis efforts when there is a concerning trend at the study level.

The use of QTL companion KRIs in this way provides more time to address conditions that may result in exceeding QTL thresholds, which need to be reported in the CSR if considered an important deviation. Exceeding KRI limits should be documented in the trial master file (TMF) but do not need to be included in the CSR. Thus, the barrier to their usage as well as any actions taken or not taken is lower, and they reduce the likelihood of experiencing significant QTL deviations. Companion KRIs could be used operationally to help ensure the QTL threshold is not exceeded at study end. This is, undoubtedly, the purpose of including QTLs in ICH E6 (R2).

The Consortium QTL Working Group discussed many of the key challenges with the definition, implementation and use of QTLs in clinical trials. Building on TransCelerate Biopharma definitions, participants concluded that QTLs should be considered a special subset of study-level KRIs, focused on the most critical risks. The group agreed on areas where QTLs are typically used and 

 and developed an approach to overcome the challenge of balancing the need to define a QTL to provide an early signal with the potentially increased reporting burden in the CSR. The approach is to use additional KRIs alongside the QTL and for these KRIs to be defined to provide an early signal of an emerging issue.

By careful use of normalization factors, it is possible to design companion KRIs that can focus on the rate of change and can be used to project whether, if the rate continues, the QTL is likely to be breached by study end. The use of companion KRIs to the QTL helps realize the benefits of QTLs—to help maintain the control of the trial for critical parameters and restrict the reporting of deviations in the CSR to those that are important.

 of this series for the whole story. If you would like more information on the Consortium, please visit our 

, MBA (corresponding author), Senior Advisor, Metrics & Performance Management, WCG, Founder and former Executive Director, Metrics Champion Consortium

, Medidata (previously associated with PPD), 

Contributions by the co-authors and contributors are solely their own and are not intended to express the views of their organizations.

https://www.ich.org/page/efficacy-guidelines

Quality Tolerance Limits: Framework for Successful Implementation in Clinical Development. Bhagat, R et al. Therapeutic Innovation & Regulatory Science 2020. 

https://doi.org/10.1007/s43441-020-00209-0

Example: Demonstrating the Impact of Different Normalization Factors

Note: this worked example is based on experience of consortium members but is not actual study data.

Let’s consider two QTL parameters used in a worked example to see when you might have a signal (flag) that the study has a higher-than-expected number of withdrawals.

If 11 or more participants withdraw from the study, by study end, it is unlikely the study will have the statistical power needed to draw a conclusion.

Table 3 below provides the QTL parameters proposed by TransCelerate Biopharma2 and the QTL thresholds. It also provides a leading parameter example and the associated threshold.

Figures 4a, b, and c below show the parameter values through week 13. The thresholds are shown as a red line and the secondary limit for the QTL parameters (here set at 75%) as the yellow line.

At week 13, two participants have withdrawn. This is below the QTL threshold and the secondary limit for both QTL parameters in Figures 4a and 4b. There is no risk signal at week 13 using these QTL parameters. If you were monitoring QTL parameter 1 or 2 only, then at week 13 you would likely not be concerned as to whether the QTL threshold would be exceeded by study end. But because there is more opportunity (chance) for participants to withdraw the longer they are on the study; these QTL parameters may be giving false reassurance.

The leading parameter shown in Figure 4c has a clear risk signal—the threshold has been exceeded since week 10. Exceeding the threshold for this leading parameter means that 

, then by study end, the number of withdrawn participants is likely to exceed the acceptable number, i.e., the QTL parameters will exceed their QTL thresholds if the current rate of withdrawal continues. Note that the leading parameter is designed to provide an early signal of risk, and no secondary limit is required.

Figures 5a-c below show the parameter values as they progress through week 24.

In Figures 5a and 5b, we see that the secondary limits and QTL thresholds were exceeded during this time period. Figure 5a shows that 13 participants withdrew from the study, exceeding the 11-participant QTL threshold stated in Table 3, which might lead to a loss of statistical power for the study. Notice that Figure 5a shows the parameter values rising rapidly beginning around week 13—illustrating that the number of participant withdrawals increased the longer participants were in the study.

The leading parameter—shown in Figure 5c—has values exceeding its threshold from week 10. It provides an early signal that if the rate of withdrawal continues, then the QTL threshold is likely to be exceeded by study end. By taking early action and reducing the withdrawal rate to remain below the leading parameter threshold (if possible), it is less likely that the QTL threshold will be exceeded by study end. The additional benefit is that no arbitrary secondary limit is required for the leading parameter. By week 15, the leading parameter is leveling off at around 11 withdrawals per 1000 participant-weeks—exceeding the threshold limit of 9.2. This illustrates an additional benefit of using this leading parameter. It is easier to estimate the number of participant withdrawals by study end (assuming the rate continues unchanged). As there are 1200 participant-weeks planned, we can multiply 11 by 1200/1000 to estimate 13.2 participant withdrawals by study end.

Thus, using the leading parameter makes it possible to detect an early signal (week 10) of a possible issue. And by week 14, to estimate the potential impact if there are no changes to the trend. This contrasts with the detection of a possible issue at week 18 with the two QTL parameters shown in Figures 5a and 5b when using the secondary limit at 75% of the QTL threshold. It is leading parameters such as these that are proposed as companion KRIs.

Articles

Series Part 3—Methods for early detection of risk and summary.

Defining Quality Tolerance Limits and Key Risk Indicators that Detect Risks in a Timely Manner: Reflections from Early Adopters on Emerging Best Practices (Part 3)

, QTL Working Group discussions that explored the relationship between QTL and key risk indicators (KRIs). In this second part of the series, we will delve into the process of defining QTLs.

TransCelerate Biopharma has published how and when to define QTLs

 and the link to critical to quality (CtQ) factors described in ICH E8 R1.

 That approach is endorsed here with some additional detail.

The recommended steps for developing QTLs are shown in Figure 1 below. The process should begin in the study design phase when robust statistical considerations are formulated. These considerations are foundational when defining QTL parameters and limits. ICH E8 R1 sets expectations that CtQ factors be established. ICH E6 R2

 then focuses on identifying the associated critical data and processes in the study. Risks associated with these critical data and processes should be minimized where possible as part of the study design. It may be attainable, at this stage, to begin identifying possible QTL parameters to monitor the remaining critical risks related to the CtQ factors.

As the operational design of the study is developed, the CtQ factors, critical processes and data—plus any proposed QTL parameters—should be reviewed and revised. At this stage, there may be additional parameters identified that can assist with providing an early signal that the QTL threshold might be deviated from—termed “companion KRIs” by the QTL Working Group and described in more detail in 

Defining QTL thresholds can be challenging. A recent paper

 provides some historical data and approaches to this challenge. Where QTL expectations are defined in the statistical analysis plan, it may be possible to derive a threshold from this information, e.g., the number of participant withdrawals that likely lead to the statistical power being too low for the study.

The selection of meaningful QTLs should be specific to each trial, but there are common areas that usually warrant careful consideration to apply a QTL:

Missing or non-evaluable data for primary and secondary endpoints, e.g., data not collected at the visit, data collected outside of visit window, participant discontinuation, etc.

Other data that could impact the number of participants eligible for key endpoint analyses, such as protocol deviations, e.g., a high rate of prohibited concomitant medications, early treatment or study discontinuation

Table 1 below provides areas where QTLs are typically considered, along with the rationale and examples.

 there are additional examples in Table 2 of Ref 1

Table 2 below highlights typical areas where, for most trials, teams should carefully consider alternatives before applying a QTL. It may be difficult to obtain data to monitor these risks effectively via QTLs. Teams should consider whether there is a more effective way of monitoring risks in these areas.

 highlighting methods for early detection of risk. If you would like more information on the Consortium, please visit our 

Bhagat, R et al.Quality Tolerance Limits: Framework for Successful Implementation in Clinical Development. Therapeutic Innovation & Regulatory Science. 2020. 

https://www.ich.org/page/efficacy-guidelines#8-1

https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf

Makowski M et al. Historical Benchmarks for Quality Tolerance Limits Parameters in Clinical Trials. Therapeutic Innovation & Regulatory Science. 2021. https://link.springer.com/article/10.1007/s43441-021-00335-3

CTTI Quality by Design Project – Critical to Quality Factors Principles Document. Version 19 May 2015. 

https://ctti-clinicaltrials.org/topics/quality/quality-by-design/critical-to-quality-ctq-factors-principles-document/

TransCelerate Interpretation of Clinical Guidances & Regulations: Quality Tolerance Limits Initiative Frequently Asked Questions. 

https://www.transceleratebiopharmainc.com/wp-content/uploads/2020/09/TransCelerate_Interpretations-of-Clinical-Guidances-and-Regulations_QualityToleranceLimits-FAQs_September-2020.pdf

Series Part 2—The process of defining QTLs.

Defining Quality Tolerance Limits and Key Risk Indicators that Detect Risks in a Timely Manner: Reflections from Early Adopters on Emerging Best Practices (Part 2)

Today’s regulatory updates are driving innovation to revolutionize quality improvement in clinical research. While quality tolerance limits (QTLs) are standard in some industries, the International Conference on Harmonization’s ICH E6 (R2)

 represents the first time QTLs have been introduced for application in good clinical practice (GCP). Although early adopters have collaborated to understand and apply QTL-related practices, most clinical studies employing QTLs have not yet passed through regulatory inspection—leaving much room for discovery of and reflection on emerging best practices.

 along the journey to help stakeholders get ahead of the curve for QTL adoption. New insights will help our industry focus not on doing 

 to ensure impactful quality—without overburdening sites or sponsors. These quality improvements represent significant opportunities to reframe and rethink how we bring drugs to market, ultimately offering new hope to patients awaiting medical breakthroughs.

, an alliance of industry organizations through which sponsors, CROs and vendors collaborate in a progressive, pre-competitive environment. Our shared objective is to enable innovation, elevate quality and bring key stakeholders in the clinical trial process into greater alignment.

In late 2020, consortium members decided to develop guidance on the definition of QTL—regarding metrics, implementation and use—by channeling experiences from across the industry. A working group of 20 individuals (representing 16 organizations) met virtually during 2021 to discuss the challenges and develop a leading practice approach to be shared with the industry. The group adopted and built upon definitions of key terms used by TransCelerate Biopharma Inc. This series summarizes the areas of agreement that the working group considers most beneficial to the industry.

Traditionally, quality in clinical trials has been achieved by inspection—checking outputs for errors. Examples include checking data quality before database lock, auditing investigational sites and source document verification (SDV) by site monitors. While other industries have successfully adopted Six Sigma and lean process improvement methodologies—initially developed in the manufacturing industry during the second half of the 20th century—clinical research has been slow to adopt them.

These process improvement methodologies are based on two fundamental premises: designing quality into the protocol and processes from the start (a.k.a. quality by design (QbD)) and monitoring trial processes (rather than their outputs) to maintain them in a consistent state (in control). When a process is determined to be outside the normal operating parameters (out of control), the process is stopped; investigation takes place, and adjustments are made to put the process back on track. By focusing on the process, the intention is that an early signal can be detected—signaling a risk to the outputs. By responding to that signal, any output that does not meet requirements can be minimized and, hopefully, eliminated.

It is preferable not to produce faulty output at all, rather than produce it and then discard or rework it later. Detecting an early signal is critical to this approach. Process improvement efforts often initially focus on understanding the detailed cause and effect in processes to help establish the process measurements most likely to provide a signal that can be detected in time to act. This process focus has spread across many industries—particularly through the adoption of the ISO 9001 standard.

ICH E6 (R2) was adopted for clinical trials in 2016. Section 5.0 (Quality Management) owes much to the process focus described above. The “predefined QTLs” referred to are measurements that can give an early signal of risk to the overall quality—impacting human subject protection or reliability of trial results. The earlier a signal of risk can be detected, the more likely an investigation can lead to actions that minimize or even eliminate these quality impacts.

Consortium members have regularly discussed the challenges of implementing QTLs. One significant challenge is a limited understanding of “cause and effect” within clinical trial processes— in part because every trial is, by definition, new and different. ICH E6 (R2) requires that “important deviations from the predefined quality tolerance limits” are summarized in the clinical study report (CSR). There is, in effect, a balance of judgment taking place (see Figure 1 below). Ideally, an emerging issue is detected early to give enough time to act—to understand the issue and reduce its impact on the trial output. A parameter designed to provide this early signal will lead to signals due to non-systemic issues, i.e., noise.

For example, monitoring the proportion of trial participants who have withdrawn could lead to such a signal if one participant withdraws when only three are enrolled. There may be no evidence that the single withdrawal might impact other participants and, therefore, the trial output. In addition, newly defined parameters designed to detect emerging issues early might be imperfect, leading to further “false” signals.

If a QTL parameter is defined to provide an early signal of an emerging issue, such signals will each have to be assessed for whether they need to be reported in the CSR. Sponsors do not want to have to report deviations that are false signals because of a poorly defined measurement or because they are an artifact of limited data early in the study. As a result, the regulation has the effect of encouraging sponsors to define QTLs that are unlikely to be breached, which undermines the intended purpose of providing early detection of risk.

 Balance between providing an early signal with a QTL and the reporting requirements

Another challenge is that the relationship between key risk indicators (KRIs) and QTLs is inconsistent across the industry. Many sponsors have adopted an approach of monitoring investigational site risk using KRIs due to the emergence of risk-based quality management methodologies. Deviations from KRIs do not have to be reported in the CSR, so KRIs can be designed to provide early signals of quality risk without the burden of including a summary of deviations in the final study report. However, the implementation and use of QTLs is often considered a separate exercise. This issue is compounded by the common misconception that any measurement of study-level risk must be a QTL rather than a KRI.

Consortium members participating in the QTL Working Group met over several months in 2021 to discuss these challenges and propose possible solutions. They agreed to accept the definitions of key terms from TransCelerate Biopharma

 and to build on that approach. The following proposals have been made:

KRIs can be defined and assessed at the study level and other levels (e.g., country, site, participant). QTLs can be considered the most critical study-level KRIs. This concept is described under 

Quality tolerance limits vs. key risk indicators

Determining the right QTL parameter and threshold for given risks is often difficult. Further guidance can help organizations with QTL definition, including areas where QTLs are typically used and 

The earliest possible detection of an emerging risk is important to managing study risks and reducing the likelihood of them becoming 

. The QTL Working Group proposes the use of companion KRIs to a QTL. These companion KRIs can provide an early signal of an emerging issue but do not lead to the burden of additional reporting in the CSR. The definition of KRIs can significantly impact the timeliness of risk detection. This concept is described in 

Let’s explore the relationship between QTLs and KRIs as applied to clinical research. KRIs are “metrics used to monitor identified risk exposures over time.”

 They have a metric definition and a threshold (or possibly two). When the value of the KRI goes beyond the threshold, there is a signal of an emerging risk or issue that may need investigating. They are most often used to detect site-level risks, and they trigger follow-up action steps such as a remote site contact or an on-site monitoring visit to investigate the risks and address any issues. Numerous site-level KRIs may be used in a study, with the total number typically ranging from 10–20. By detecting possible issues at a site early, actions can reduce the potential impact on data integrity or human subject protection.

While sites represent a primary focus of KRI-driven risk detection, the potential sources of risk on a given study may also include the geographic location (e.g., country) or individual participants. Many of the same KRIs used to monitor site risks may also help detect risks at these other levels. Similarly, KRIs can be defined and used at the study level. In this case, the data across all sites are aggregated, and the KRI is compared over time against a discrete risk threshold. Study-level KRIs are thus designed to detect risks and issues that are systemic in nature and not isolated to one or only a handful of sites in the study.

For example, the cycle time from participant visit to data entered into EDC (data entry timeliness) is a commonly used KRI. It detects sites taking significantly longer than expected to enter data, which may pose a risk to data quality. Additionally, it identifies an operational risk that the centralized data monitoring KRI dashboards are not based on near-real-time data. A study-level KRI for data entry timeliness may be useful to monitor the overall trend in the timeliness of data entry across all sites in the study. This could be compared against a standard “expected” time. Suppose the average timeliness across all sites (at study level) is higher than the standard. In that case, there is a risk for the study, which may warrant study-wide actions (e.g., awareness communication to all sites) vs. a site-by-site intervention.

QTLs as a special subset of study-level KRIs

As with KRIs, QTLs are a combination of the parameter (measurement) and threshold level (limit) used to help monitor risks to quality during the conduct of a study. QTLs can therefore be considered a subset of the study-level KRIs, designated to monitor the most important risks associated with critical-to-quality factors

 in the study. In other words, they are study-level KRIs that focus on study risks that have the greatest potential impact on data integrity or human subject protection. TransCelerate Biopharma recommends the use of 3-5 QTLs for a study. KRIs are not referenced in GCP—ICH E6 (R2), but QTLs are:

“Predefined quality tolerance limits should be established, taking into consideration the medical and statistical characteristics of the variables as well as the statistical design of the trial, to identify systematic issues that can impact subject safety or reliability of trial results. Detection of deviations from the predefined quality tolerance limits should trigger evaluation to determine if action is needed.”

Perhaps this concept has led to the perspective that QTLs are entirely different from KRIs. Table 1 below helps to highlight the differences and similarities between QTLs and KRIs.

According to ICH E6 (R2), QTLs should be “predefined,” which is taken to mean that the parameter and threshold should be established before study start or first participant in (FPI). The QTLs are indicators of whether the expected quality in the trial is being achieved, and, as with hypothesis testing, the expectation is set before gathering data. The aim is to avoid bias from data in the current trial when setting the QTL parameter and threshold. However, this does not mean that a QTL must remain unchanged throughout the course of a trial. Despite best efforts to establish QTL definitions and thresholds in advance, a study team may become aware of information not previously recognized or known that triggers some adjustments. It is important to clearly document the reason and justification for the updates in these cases.

There is an additional requirement in ICH E6 (R2) relating to QTLs:

“The sponsor should describe the quality management approach implemented in the trial and summarize important deviations from the predefined quality tolerance limits and remedial actions taken in the clinical study report (ICH E3, Section 9.6 Data Quality Assurance).”

Compared to KRIs, documentation requirements are significantly different for QTLs, referenced in ICH E6 (R2). The CSR should include an explanation of the selection of QTL parameters and QTL thresholds, a description of actions taken due to important deviations from the thresholds and a justification of any changes to the thresholds.

QTLs are a special subset of KRIs that focus on critical study risks.

Figure 2 below shows the relationship between QTLs and KRIs. The parameters used to monitor QTLs and KRIs are metrics. QTLs are a special subset of KRIs that focus on critical study risks.

There has been much discussion on interpreting the term “important deviation” in ICH E6 (R2). Most deviations from the QTLs are likely to be important, as they are the special subset of KRIs that focus on critical study risks. However, early in the study, where there are few participants, a QTL might be deviated from simply due to limited data. For example, one participant terminating early out of two enrolled may be simply a special cause for that one participant rather than a systemic study-wide issue. These deviations may not be considered “important” as long as the QTL is not exceeded at the end of the study.

 in this series which will highlight the process of defining QTLs. If you would like more information on the Consortium, please visit our 

https://www.iso.org/iso-9001-quality-management.html

Quality Tolerance Limits: Framework for Successful Implementation in Clinical Development. Bhagat, R et al. Therapeutic Innovation & Regulatory Science. 2020. 

https://www.transceleratebiopharmainc.com/rbminteractiveguide/the-next-frontier-of-rbm/glossary-key-terms-defined/

Series Part 1—Introduction and the relationship between QTL and KRI.

Defining Quality Tolerance Limits and Key Risk Indicators that Detect Risks in a Timely Manner: Reflections from Early Adopters on Emerging Best Practices (Part 1)