Introduction

In the past decade, clinical development regulations have moved with the pharmaceutical industry to modernize clinical development and embrace the paradigms of risk-based quality management. A major revision to Good Clinical Practice (GCP) Guidelines occurred in November 2016 with the publication of the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E6(R2) guidelines [1], later adopted by various Health Authorities (HA). Section 5.0 Quality Management was added to the guidelines with the introduction of a risk-based approach to quality management at the protocol and system levels. The guidelines included the requirement to establish Quality Tolerance Limits (QTLs) to guide clinical trial quality proactively by controlling for risks and allowing for corrective actions to be taken during the conduct of the trial to avoid later quality issues. The introduction of QTLs challenged sponsors with interpreting and operationalizing the guidance.

The QTL process described in this framework includes three stages: Define, Monitor, and Report. The Define stage, occurring after availability of a draft protocol and before enrollment of the first participant, includes defining the parameters and thresholds for QTLs. This stage also includes development of a QTL monitoring plan to define timeframe and frequency of reviews and data sources for monitoring. During the Monitor stage, while the clinical trial has participants in the clinic, the framework calls for periodic reporting according to the monitoring approach. Any QTL deviations from the predefined threshold(s) would be investigated and corrected as needed. After the trial ends, a summary report of QTL deviations and associated preventive and/or corrective actions would be generated. Under the framework, the highlights or important QTL deviations and associated actions would be included in the clinical study report (CSR). Implementing QTLs according to this framework is consistent with ICH E6 and industry best practice for measuring and monitoring clinical trial quality.

ICH Guideline Reference

ICH E6(R2) indicates that QTLs be established to “identify systematic issues that can impact subject safety or reliability of trial results” and that important deviations from the predefined QTLs and associated remedial actions taken are reported in the CSR [1].

Development of the TransCelerate QTL Framework

Following the release of ICH E6(R2), TransCelerate’s Risk-Based Monitoring initiative produced a position paper exploring this new concept and providing implementation considerations for establishing QTLs and risk reporting in the CSR [2].

In response to learnings from implementation of QTLs since the position paper was written, TransCelerate’s Interpretations of Guidances & Regulations (IGR) initiative identified a team of subject matter experts (SMEs) from 11 of its member companies to revisit the ICH E6(R2) guideline with respect to QTLs. This document is the result of discussions sharing best implementation practices for effective and efficient implementation of QTLs.

How QTLs Fit in a Quality Management System

Quality by Design (QbD) principles underpin a risk-based Quality Management System (QMS). QTLs are part of that risk-based approach. They are an added control for risks to factors that are critical to quality (i.e., CtQ factors). In clinical trials, CtQ factors are those with the potential to impact participant protection and/or the reliability of trial results. These may include the following:

  • Primary objective

  • Safety objectives

  • Patient eligibility

  • Investigational product exposure

CtQ factors related to critical data and processes are also described in ICH E8(R1) General Considerations for Clinical Studies (draft version) [3]. Section 3.2 of the draft guidance states that these quality factors are considered to be critical because, if their integrity were to be undermined by errors of design or conduct, the reliability or ethics of decision-making would also be undermined.

Therefore, at the time of protocol design, CtQ consideration is foundational to ensuring that trials are designed with quality built in. In parallel, sponsors should define appropriate risk management strategies to protect trial participants and the reliability of trial results (i.e., Integrated Quality Risk Management Plans enabling risk-based monitoring strategy). This includes the use of controls like QTLs and Key Risk Indicators (KRIs).

As noted in Fig. 1, QbD is present throughout a trial, starting with the protocol design and the identification of CtQ factors, through to the monitoring of data during the conduct of the trial, and concluding with analysis of the impact of important deviations in the CSR.

Fig. 1
figure 1

Risk-Based Quality Management Components. CSR Clinical Study Report; QTL Quality Tolerance Limit

Full size image

QTL Definition and Purpose

A QTL is a level, threshold, or value associated with a parameter which is critical to quality. QTLs are set for risks identified at the trial level. A deviation from a threshold during the conduct of the trial may indicate a systematic issue that could impact participants’ safety or reliability of trial results. QTLs should be defined with the protocol and no later than the first participant’s first visit (FPFV).

Performance should be monitored against the predefined QTLs and assessed on a regular basis throughout the course of a trial. Additionally, any trend indicating that a QTL deviation may occur could trigger an evaluation to assess if action is needed to avoid a potential deviation from the predefined QTL.

Based on the systematic nature of the issues discovered through the use of QTLs, some of the remediations would benefit future studies by identifying risks and ways to control them proactively. This can be addressed in the risk review part of the risk management process [4].

Early action thresholds (i.e., secondary limits) could be specified for QTL parameters to provide study teams with early opportunities to mitigate risks to participant safety or reliability of trial results and avoid a QTL deviation. This added limit allows study teams to intervene before an important deviation from a QTL is observed, if desired.

Number of QTLs

In keeping with a risk-based approach, the number of QTLs should be commensurate with level of risk associated with the protocol. QTLs should be carefully selected and, ideally, aligned with CtQ factors. Too many QTLs will dilute the importance of each QTL and the amount of time available to spend on controlling factors that contribute to each one.

Relationship Between QTLs and KRIs

QTLs and KRIs help control risks identified early in the clinical development process. Both are defined and measured to manage factors that are critical to quality during the conduct of the trial. In some cases, QTLs and KRIs may share the same parameter (e.g., proportion of participants with protocol deviations on eligibility criteria or proportion of participants with premature discontinuation). KRIs and QTLs differ in that KRIs are typically measured at the site level to inform site monitoring activities, while QTLs are a higher-level indication of overall quality in a trial. An example of a potential relationship between a CtQ factor, a QTL, and a KRI is shown in Table 1.

Table 1 Example CtQ Factor and Associated QTL and KRI.
Full size table

Additionally, a QTL deviation at the trial level, for example participants with withdrawal of informed consent at the trial level, is not necessarily coincident with a KRI deviation at the site or country level.

On the contrary, if the QTL deviation occurs, it is anticipated that an equivalent KRI deviation will occur for multiple sites or countries. Either scenario will require an evaluation of the issue and mitigation activities at the appropriate level.

Finally, in a risk-based approach, some KRIs may not be suitable as QTL parameters. The most important factors across the trial warrant QTLs. Other supporting quality indicators are better suited for KRIs. For example, metrics related to compliance (e.g., Site Trial Master File and Investigator Site Form completeness metrics) require oversight to ensure the integrity of the trial, but may not be as significant to human subject protection or reliability of trial results as QTLs.

QTL Process

QTLs act as controls for risk and are part of the Clinical Trial Quality Risk Management process. This QTL Framework includes a process for defining, monitoring, and reporting QTLs, which corresponds to the set-up, conduct, and closeout phases of a clinical trial (Fig. 2). A separate detailed process map and considerations for each stage follow (Figs. 3, 4, and 5).

Fig. 2
figure 2

Quality Tolerance Limit (QTL) Framework—Process Overview.

Full size image
Fig. 3
figure 3

Define QTLs Process Map.

Full size image
Fig. 4
figure 4

Monitor QTLs Process Map.

Full size image
Fig. 5
figure 5

Report on QTL Process Map.

Full size image

Defining, monitoring, and reporting of QTLs is typically a cross-functional process involving some or all of the following groups: Clinical Development, Clinical Operations, Biostatistics, Medical Monitoring, Medical Writing, Data Management, Clinical Supplies, Pharmacovigilance, Clinical Development Quality Assurance, and Centralized Monitoring. Different companies employ different models for resourcing QTL development. Whether using a facilitated model in which a quality professional or study management is leading the process or a functional model in which members of the study team are leading QTL activities, clinical and statistical functional experts should be engaged throughout the process.

If the program uses outsourcing, the sponsor may choose to involve the contract research organization (CRO), or other involved organizations, in management of QTLs. If management of QTLs is outsourced, the sponsor maintains accountability for the process. In addition, QTL-related activities must be documented in order to demonstrate compliance.

Define Process Stage

The Define stage of the QTL process starts after a draft protocol is available. The following steps are included in the Define stage (Fig. 3):

  1. 1.

    Perform Risk Assessment—identify CtQ factors and associated risks to critical processes and data, which could impact patient safety and reliability of trial results.

  2. 2.

    Define QTLs—identify a limited set of QTLs which focus on factors that are the most critical to quality. The following items should be considered:

    • Parameter and description

    • Thresholds for the parameter

    • Justification for the parameter and threshold

    • Action plan in case of deviation from predefined threshold

    • Algorithm to calculate the threshold

    • Source of data from which to calculate periodic results

  3. 3.

    Develop QTL Monitoring Approach—develop a QTL monitoring plan that includes the following items:

    • Timeframe to start and end QTL monitoring

    • Frequency of QTL review

    • Definition of data sources for QTL monitoring

    • Criteria for determining deviation importance

    • Action to be taken if important deviations are observed

    • Programming and report setup

It is regarded as a best practice to document the details of the QTLs and the QTL monitoring approach in the QTL monitoring plan. This QTL monitoring plan is separate from the study monitoring plan and other trial-level monitoring activities. Note that while it is generally best practice to have QTLs defined prior to the FPFV, time should be allotted for programming, where required.

Monitor Process Stage

The Monitor stage of the QTL process starts after the first participant has been enrolled. The following steps are included in the Monitor stage (Fig. 4):

  1. 4.

    Setup QTL Report—complete programming, configuration, and data mapping activities needed to start QTL reporting for the study.

  2. 5.

    Run Report and Assess Result(s)—assess results for the presence of any deviations.

  3. 5a.

    Investigate Deviation(s)—the following activities could be included when investigating a QTL deviation from the predefined threshold:

    • For important deviations that potentially affect participant safety or reliability of trial results, issue management processes may include root cause analysis and determination of corrective or preventive actions.

    • For unimportant deviations, incorporate operational changes (within the protocol) or revise the QTL definition or threshold.

Report Process Stage

The report stage of the QTL process starts after study closure. The following steps are included in the report stage (Fig. 5):

  1. 6.

    Generate QTL Summary Report—the QTL Summary Report is a list of deviations from the defined QTLs and actions taken to address the deviations.

  2. 7.

    Review QTL Deviations—review all QTL deviations from the trial.

  3. 8.

    Report in CSR—summarize the quality management approach and important QTL deviations and actions taken in response in the CSR.

General Considerations

When implementing QTLs, there are additional important factors to consider as these may influence the overall process.

Historical Data for Setting Thresholds

Historical data from literature searches or internal company data can be used to help establish QTLs. Sponsors should identify and remove the systematic error and/or bias from historical data used to determine QTLs. Examples of ways to remove systematic errors and bias include the following:

  • Excluding significantly outlying studies from the historical data set

  • Using the median instead of the mean to define the expected value and minimize the influence of outliers

  • Broadening the set of studies in the historical data set if studies of the same type are not available

  • Reviewing past quality issues (events) related to QTL deviations

  • Using the most recent studies to better represent current practices if the number of historical studies is large

  • Considering CRO historical knowledge and/or data

Bias may also arise from the selection of the historical datasets. To avoid bias, the sponsor might consider the following factors when selecting historical data:

  • Therapeutic and disease area

  • Phase of development

  • Patient population (e.g., stage of disease, age, gender, etc.)

  • Whether significant process changes have taken place since the historical data was collected

  • Assumptions made in prior studies that are not appropriate for the current trial

  • Definitions or exclusions of data in an external trial

Finally, in addition to historical data and clinical knowledge, the study team could also leverage other trial-level information from the protocol or the Statistical Analysis Plan (SAP) to predefine QTLs, especially when historical data is not available for a particular type of trial.

Carefully setting the predefined QTL thresholds improves the probability of detection of systemic issues through QTLs.

Additional Considerations for Defining QTLs

In addition, QTLs fall under the Quality Management section in ICH E6(R2) as a control. This section of ICH E6 notes that the methods used to assure and control the quality of the trial should be proportionate to the risks inherent in the trial and the importance of the information collected. The following information can be considered to assess the level of risk and resulting applicability of QTLs as a control:

  1. 1.

    Trial-level risk management plan (including controls)

  2. 2.

    Number of participants

  3. 3.

    Number of sites

  4. 4.

    Trial Duration—adequate duration of the trial is a consideration to implementing the QTL process and implementing any remedial actions as a part of the QTL process

  5. 5.

    Recruitment rate

  6. 6.

    Trial Design (e.g., dose escalating cohorts because of the small number of participants in each cohort)

  7. 7.

    Trial population

The ICH E6(R2) guideline applies to clinical trial data intended for submission to regulatory authorities. The applicability of QTLs to early phase studies then depends on the clinical development plan for a molecule and whether or not the data is intended for submission. Decisions on the applicability of QTLs and/or the number of QTLs to implement is the responsibility of the sponsor, commonly informed by the study clinician and statistician. At the end of the trial, the quality management approach and any important QTL deviations will need to be documented in the CSR.

Use of a QTL Library

QTLs are based on the medical and statistical characteristics of a trial; thus, they are inherently trial-specific. However, a QTL library, or set of previously used QTLs and their defining characteristics, can be established as a starting point for QTL definition. While QTLs selected from a library may need to be adapted to trial-specific characteristics, building from common definitions enables later comparison across studies and may streamline the definition process. A QTL library may be built based on therapeutic area or other common clinical trial characteristics. For example, in many oncology studies, parameters focused on Response Evaluation Criteria in Solid Tumors (RECIST) data quality or completeness may be closely related to the interpretability of trial results and applicable to a broad set of studies.

Table 2 is a sample QTL Library with parameters and thresholds for consideration in developing a QTL program. QTL parameters included in the example library are generic and may be applied to studies independent of therapeutic areas.

Table 2 QTL Library Example.
Full size table

The list of parameters included in Table 2 is not intended to be exhaustive. Depending on the trial design, therapeutic area, and indication, parameters from the list may be identified by the study team as applicable to address areas of the highest risk to reliability of trial results and patient safety. Trial-specific QTLs may be added as deemed appropriate by the cross-functional study team.

Conclusion

Clinical development continues to emphasize risk-based approaches to clinical trial quality. ICH E6(R2) establishes the use of QTLs as a method of risk control to identify systematic issues potentially impacting participant safety or reliability of trial results.

This QTL Framework has been developed to aid clinical development professionals in the implementation of QTLs as part of a broader Quality Risk Management System. The approaches described to manage risk across trial design, conduct, and reporting should benefit sponsors, their vendors, and particularly trial participants in ensuring clinical trials adhere to the principles of GCP and more effectively bring new therapies to those who need them.