Introduction

The United States Food and Drug Administration’s (FDA’s) drug approval standard requires substantial evidenceFootnote 1of effectiveness from adequate and well-controlled investigationsFootnote 2 including clinical investigations that incorporate, among other factors, a valid comparison to a control, to “distinguish the effect of a drug from other influences [1], such as spontaneous change in the course of the disease, placebo effect, or a biased observation” [1]. The FDA, consistent with regulations (21 CFR 314.126) and ICH E10 guidance, generally recognizes internallyFootnote 3 controlled [1] study designs (placebo, active treatment, dose comparison, no-treatment) where “the control group and test groups are chosen from the same population and treated concurrently” [1]. However, FDA does recognize that in studies for diseases with high and predictable mortality or progressive morbidity, and in particular for certain rare diseases, when it is not feasible or would not be considered ethical to use an “internal control”, reliance on “external controls”Footnote 4,Footnote 5 may be acceptable [1, 2]. When a trial is externally controlled, the results of treatment with the test drug may be compared with experience derived from the adequately documented natural history of the disease or condition, a registry, published literature, or patient medical records [3, 4]. Patients may also serve as their own controls [1] (by comparison to their status before therapy).Footnote 6

In this article, we briefly review guidance documents discussing the use of external controls and provide examples of approvals where external controls were deemed satisfactory to meet FDA standards for approval. We highlight some methodological and statistical considerations and advocate for a change in guidance to promote the continued use of external controls, including retrospective natural history, in drug development and approval.

Definitions and Categories of External Controls

The ICH E10 guidance defines an externally controlled trial as “one in which the control group consists of patients who are not of part of the randomized study as the group receiving the investigational agent i.e., there is no concurrently randomized control group” [1].

External controls can be categorized by the time the subject data were collected into [1, 3, 4]:

  • Concurrent External Controls: The control group is based on subject level data collected at the same time as the treatment arm but in another setting [1]. An example is data from a concurrent prospective natural historyFootnote 7 study as the control arm for an open-label treatment study.

  • Non-concurrent External Controls (Historical Control): The control group is based on data collected at a time different (e.g., historical) from the treatment arm. Such “historical controls” can be derived from several different types of sources including:

    • Retrospectively Collected Natural History: Subject level data collected retrospectively from a natural history study. Such data may be extracted from sources such as existing medical records (for example patient charts [3, 4]), or from a previously conducted registry.Footnote 8

    • Published Data: Data only available in the published literature. Such published data may have been derived from individual cases, however, it is distinguished from retrospectively collected natural history data based on the lack of access to subject level data and the lack of detailed information on data collection methodology.

    • Previous Clinical Study: Subject level data from an arm of a previously completed clinical study [3] in the same indication and/or patient population.

    • Baseline-Controlled Study: Historical control derived from a patient’s baseline (“patient baseline control or baseline-controlled study”) [1]. The data could be collected over a period of time prior to initiation of treatment, and patients’ status on therapy is compared with status before therapy.

Of note, the term “real-world-evidence” (RWE)Footnote 9 has recently been used to describe data sourced from natural history studies, chart reviews, registries and other settings and used as a comparison arm for a single-arm study [5,6,7].

Summary of Current Guidance Discussing Use of External Controls

Several guidance documents discuss the use of external controls as a comparator in clinical trials [1,2,3,4, 8,9,10,11,12,13,14] (see Table 1), particularly for rare diseases. The ICH E10 guideline on “Choice of control group and related issues in clinical trials [1] provides a comprehensive discussion on such controls, stating “The choice of the control group should be considered in the context of available standard therapies, the adequacy of the evidence to support the chosen design, and ethical considerations.” While the guideline emphasizes that in most situations, an internal concurrent control is necessary to minimize bias and obtain robust statistical analyses, it also highlights that this may not always be feasible. In addition, it is broadly recognized by developers and researchers that initiation of prospective natural history studies for use as a source of external controls may not be feasible, especially in rare diseases, thus alternative approaches, such as the use of retrospective natural history data, have frequently been leveraged to support product development and approval. ICH E10 envisages this flexibility by acknowledging the acceptability of external controls from a group of patients treated at an earlier time (“historical”). Additionally, the FDA guidance “Rare diseases: common issues in drug development” [4] emphasizes that product development should not be delayed due to the lack of prospective natural history data. While FDA highlights the use of prospectively collected natural history data as the preferred approach, the guidance specifically, states that “initiation of prospective natural history studies should not delay interventional testing otherwise ready to commence for a serious disease with unmet medical need” [4]. This point abides by a pragmatic approach to product development and underscores the importance of ensuring development can proceed to expedite patient access to treatment.

Table 1 FDA and ICH regulatory guidance discussing the use of external controls

FDA further articulates the importance of flexibility in trial design in the recently published FDA draft guidance on “Demonstrating substantial evidence of effectiveness for human drug and biological products” [2]. The draft guidance indicates that FDA may rely on study designs that produce less certainty (such as externally controlled studies) in some circumstances such as “life-threatening and severely debilitating diseases with an unmet medical need,Footnote 10 for certain rare diseases, or potentially even for a more common disease where the availability of existing treatments makes certain design choices infeasible or unethical” [2]. The guidance also notes that a single trial with compelling results compared to either an external or concurrent control, could further be supported by data from separate sources (e.g., a natural history study, case report forms, or registries) as confirmatory evidence.

Collectively, these guidance documents [1,2,3,4, 8,9,10,11,12,13,14] reaffirm that use of external controls is acceptable under certain circumstances (see Fig. 1 for details) and reinforce the need for flexibility both in guidance as well as in application during product development and FDA decision making.

Fig. 1
figure 1

The use of external control design is most persuasive under the following circumstances

Methods

We searched FDA regulatory approvals between 2000 and 2019Footnote 11 for drug and biologic products where pivotal studies employed external controls. We included original marketing applications and supplemental applications for new indications specifically mentioning use of natural history data or historical controls to support a pivotal study. Applications in which natural history data were used in other ways, such as to guide endpoint development or to interpret nonclinical studies, were excluded.

Since the use of external controls appears to be well accepted in the field of oncology [15, 16], we focused our assessment on non-oncology product approvals, concentrating on the FDA divisions responsible for reviewing the following therapeutic areas: gastroenterology and inborn errors of metabolism; neurology; metabolism and endocrinology; reproduction, bone diseases, and urology; and non-malignant hematology. Anti-infectives, vaccines and immunoglobulins were excluded as their development pathways are dictated by guidelines unique to the therapeutic area.

We examined the characteristics of such applications with respect to rare disease status, seriousness of the disease, degree of unmet medical need, and objectivity of the primary endpoint and categorized the source of the external control data.

Results

Based on our search criteria, we identified forty-five productsFootnote 12 (see Table 2) for which pivotal trials were supported by external controls. Nearly half (49%) of the cases identified were for non-malignant hematological products (Fig. 2) with gastroenterology and inborn errors of metabolism products comprising the second largest category (22%), followed by metabolism and endocrinology products (13%), neurology products (9%), and reproduction, bone diseases and urology products (7%), illustrating that experience with the use of external controls is variable across the Divisions at FDA.

Fig. 2
figure 2

FDA review divisions responsible for the 45 product approvals relying on external controls (2000–2019, select therapeutic areas)

Table 2 Characteristics of US FDA approvals based on external controls 2000–2019 (selected indications; n = 45)—presented in reverse order of approval

Overall Trends

The majority (80%; see Table 3) of the approvals relying on external controls were for a rare disease where regulatory flexibility was applied due to the size of the population and/or to the unmet medical need (i.e., no or inadequate available therapy). This is consistent with other reports of FDA flexibility with respect to the quantum of evidence relied upon for the approval of orphan products [15, 17]. Overall, for the therapeutic categories evaluated, approximately one in three (33%) first-time approvals of products for rare diseases relied on external controls over the 20-year period.

Table 3 Characteristics of products approved based upon use of external controls (2000–2019)

Types of External Controls Used to Support Approval

Of the 45 approvals evaluated, historical controls derived retrospectively from natural history data were the most common source (44%) of external control, including retrospective reviews of patient medical records (see Fig. 3). While prospectively gathered natural history data sources are preferred based on FDA guidance, none of the external controls included in the regulatory approvals assessed in this review were prospective. Other data sources of external control were less common (baseline control: 33%; published data: 11%; data from a previous clinical study: 11%). A hybrid approach, where external control data were added to a concurrent randomized control arm (placebo and/or active), was used for at least three products (velaglucerase alfa, corticotropin, centruroides anti-venom) developed to treat conditions for which there were no available therapies. Two of these were approved for the treatment of rare pediatric conditions.

Fig. 3
figure 3

Categories of external controls to support product approval by the US FDA (2000–2019, select therapeutic areas)

Objective Versus Subjective Endpoints

The vast majority (87%) of cases identified utilized an objective measure as a primary endpoint. Survival at pre-specified endpoints (sodium phenylacetate and sodium benzoate combination [Ammonul], onasemnogene abeparvovec [Zolgensma]), and urinary free-cortisol concentration (pasireotide diaspartate [Signifor]) are some examples of the objective endpoints used. For the few cases where the endpoint was subjective, the benefit was so large it was unlikely to be due to chance alone. For example, in the case of burosumab (Crysvita, X-linked hypophosphatemia [XLH] in adult and pediatric patients ≥ 1 year, a rare disease), the studies in support of the pediatric indication used data from a retrospective natural history study conducted in 52 children who were on conventional therapies (phosphate/calcitriol) as external controls and a subjective clinician-reported outcome (reduction in total Rickets Severity Scale [RSS] scored by a radiologist) as the endpoint. The large effect size for reduction in RSS (50–59% versus 12% in the historical controlFootnote 13) supported the pediatric approval. Factors that strengthened this case were the assessment of radiographs (from both the retrospective natural history study and children treated with Crysvita) by the same blinded radiologist and the use of three propensity score analyses to mitigate several imbalances in the demographics (i.e., sex and baseline rickets scores) between the treatment and historical control groups. FDA’s statistical review noted that while the comparisons were imperfect they were still supportive of the conclusion that Crysvita is more effective than conventional therapies at correcting rickets in pediatric XLH. Ultimately, FDA determined that the unmet medical need and the totality of data, including improvements in secondary and pharmacodynamic endpoints, supported approval in the pediatric indication.

Examples of Approvals Based on Different Types of External Controls

A relatively recent example of utilizing retrospective natural history data is FDA’s approval of Zolgensma (a gene replacement therapy) for infantile-onset spinal muscular atrophy (SMA) due to biallelic mutations, a rare disease with high unmet need. SMA is a serious, life-threatening disease where untreated patients will either die or require permanent ventilation by 24 months of age. Given the rare nature of the disease, data from 23 patients were successfully used as an external control. In this case, the natural history of SMA was predictable, the efficacy of Zolgensma was objectively measured, there was a large treatment effect (90% alive without ventilation versus 25% based on natural history), and there was evidence of a temporal association with the intervention.Footnote 14

Another approval that provides interesting insights into the use of retrospectively collected natural history data is that of defibrotide sodium (Defitelio) approved for the treatment of adult and pediatric patients with hepatic veno-occlusive disease (VOD) after hematopoietic stem-cell transplantation (HSCT), a rare disease with an 80% mortality rate and no available treatment options. The primary endpoint in the Defitelio pivotal study (survival at day 100) was compared to historical control data selected by independent retrospective review of patient records. Supportive data came from a dose-finding study, a compassionate use study, and a registry study. The major review issues pertained to the selection of the historical control group. The patients included in the historical control group were selected by a blinded, independent medical review committee who screened subjects undergoing HSCT. The committee used narratives, inclusion/exclusion case report forms, and partially redacted medical charts to select patients to be included in the control group. Although data collection for the treatment and historical control groups spanned vastly different timeframes (2 years and 12 years, respectively), the inclusion and exclusion criteria were pre-specified and were similar for both groups. The number of subjects in the historical control group was reduced (in two rounds) from 6867 to 123 and finally to 32 patients who had developed VOD and received standard of care. The last round was conducted after an interim efficacy analysis raised some concerns about bias because the survival rate in the larger historical control group initially selected was substantially higher than the rate generally reported in the literature. To adjust for the confounding effect of the potential prognostic factors, propensity score adjusted analyses were performed using four pre-specified covariates (all baseline prognostic factors of survival). Nonetheless, the day 100 survival rates of treated patients (38 to 45%) were higher than the historical control group (25%), the supportive care arm of the registry (31%), and published literature (< 20%). While FDA’s review included comments regarding the small size of the chosen historical control group and the risk of Type I error given the unplanned interim analyses, FDA ultimately approved Defitelio based on the totality and consistency of the data, particularly the consistency of the survival results in the pivotal study and supportive studies.

An example of a case using baseline control data for regulatory decision making is deferiprone (Ferriprox), an oral therapy for transfusional iron overload due to thalassemia syndrome (a rare disease). Deferoxamine, the only available therapyFootnote 15 at the time of Ferriprox’s new drug application (NDA) review, was not tolerated by all patients, leaving an unmet medical need. Initially, the sponsor received a complete response letter mainly due to uncertainty regarding the clinical meaningfulness of the change in a novel surrogate endpointFootnote 16 in a single pivotal study versus deferoxamine. Ultimately, an independent committee selected a subset of patients (in whom previous chelation therapy was inadequate) from the sponsor’s previously conducted clinical studies, to be included in a prospectively planned study. This study compared the selected patients’ pre- and post-Ferriprox treatment results and showed that treatment with Ferriprox significantly decreased serum ferritin in about 50% of refractory patients. The statistical reviewer noted “this study has several serious limitations including lack of randomization, lack of control group, high rate of missing data and ignoring the variation between studies by simply pooling, all of which can introduce biases to the primary outcome.” Nevertheless, FDA considered the use of a prospectively planned statistical analysis plan and the selection of patients by the independent committee allowed an adequate selection of patients for the trial, minimized the possibility of bias, and allowed for an adequate assessment of drug effect. The review documents noted “This trial can be considered an adequate and well-controlled trial under the CFR and ICH E10 guidance for regulatory purposes.” Ferriprox was approved under the accelerated approval regulations.

An unusual use of a historical control that leveraged data from previously conducted clinical studies, was the addition of a new indication (monotherapy in patients with partial seizures) for lamotrigine extended release tablets (Lamictal XR) which was reviewed at an advisory committee meeting.Footnote 17 The supplemental NDA was based on a single study in which 223 patients who received one of two-dose levels were compared to a historical control group based on a retrospective analysis of control arms from eight studies previously conducted for other anti-epileptic products [18]. The sponsor considered use of placebo or pseudo-placebo controls unethical given the significant control data already available from previously conducted studies. At the advisory meeting, FDA presented a systematic evaluation of the key statistical issues based on the Pocock criteria [19], which were applicable to this situation, as the historical control data were specifically derived from the control arm from prior studies with similar designs and methods. This evaluation included the timeframe for assessment of seizure frequency and severity, how exit rate was calculated, medications at baseline, and regional differences between study and historical controls. The advisory committee agreed (14 yes/0 no) that the proposed historical control approach was acceptable in this specific circumstance. FDA’s presentations and discussions at the advisory committee demonstrate the importance of proactively assessing the comparability of an external control to the treatment group across multiple parameters and ensuring that endpoint evaluations and statistical methods address potential biases as thoroughly as possible. This precedent for use of historical controls from previously conducted clinical studies was later applied to other antiepileptic drugs, including lacosamide (Vimpat).

Finally, the recombinant antihemophilic factor Novoeight is an example where historical control data from nine publications were used as external controls to support its approval for prophylactic treatment of Hemophilia A, a rare disease. In this case, annualized bleeding rate (ABR) in patients treated prophylactically with Novoeight was compared with the ABR observed in historical controls treated with on-demand regimens. The historical ABR was calculated using data weighted by the number of patients in each of the nine published studies. Calculated mean ABR was 22 bleeds per patient per year for historical controls treated with on-demand regimens compared to 6.9 bleeds per patient per year in subjects treated with prophylactic Novoeight, a 68% reduction in bleeding rate for subjects treated with Novoeight prophylaxis as compared to on-demand therapy historical controls. This was considered acceptable for the approval of Novoeight for routine prophylaxis treatment.

Statistical and Methodological Considerations When Using External Controls

It is outside the scope of this article to provide a comprehensive review of methodological and statistical topics pertaining to the use of external controls, but some important considerations are highlighted in this section.

A key challenge of using external controls is that differences in prognostic variables (such as demographics, diagnostic criteria, disease stage, baseline status, and concomitant therapies) between the treated and external control groups could lead to biases particularly in the absence of randomization. One way of addressing bias is through proper selection of the external control group. Pocock proposed six criteria for a historical control group to be acceptable [19] (Fig. 4), sometimes cited by FDA reviewers, as in the previously mentioned Lamictal XR example. However, Pocock specifically intended these criteria (deemed stringent by Lim et al. [20]) for specialized methods for combining a historical control group from a previous trial with a randomized concurrent control group. Indeed, Pocock’s use of the term “historical control” differs from his contemporaries [21, 22], and from current usage in reference to non-concurrent external controls in general (a historical control per ICH E10 guidance is any “well-documented population of patients observed at an earlier time”) [1]. Thus, while the Pocock criteria may not all be applicable in a given situation, those that are should be applied to the extent possible in the selection of a historical control group and to the ensuing comparative statistical analyses [23].

Fig. 4
figure 4

Pocock’s key criteria for accepting historical control data

Three statistical methods are often used to adjust for baseline imbalances between the treated and external control groups: matching, covariate adjustment, and stratification. Propensity scoresFootnote 18 can be used as part of all three methods; one can match or stratify based on propensity score, or one could use a propensity score as a covariate [24, 25]. Propensity scores are a widely used and important method, but the method has its detractors. For example, Elze et al. [24] argue that propensity scores are not necessarily preferable to covariate adjustment, and King and Nielsen [26] raise fundamental questions about whether propensity scores succeed in addressing imbalance, inefficiency, model dependence, and bias.

Some variables may not fit into standard analysis approaches for covariates but may be addressed using other methods. An example is the contemporaneousness of external control versus trial data, which can be explored by examining the external control data for trends in outcome variables versus calendar date of assessment.

Bias can also be addressed by transparency about critical aspects of the analyses such as exclusion of subjects and handling of missing data. Missing data in particular represents a critical and widely studied issue. In brief, the principles of analyses to address missing data in nonrandomized trials “include the need to design and conduct trials to minimize the amount of missing data, the need to use principled missing data adjustments based on scientifically plausible assumptions, the need to conduct sensitivity analyses for potential deviations from the primary assumed mechanisms of missing data, and the need to collect covariate information that is predictive of missingness and the study outcomes” [27].

Another important set of issues pertains to longitudinal data. Longitudinal analysis approaches fully utilize the available data but present several challenges. For example, it may be difficult to align timepoints in the clinical trial with those of the external control because of the different assessment schedules or irregular assessment timing in the external data. Additionally, missing data may be a greater concern in a natural history data source than in a well-conducted clinical trial, given its observational nature as well as the potential for patients to enter and exit the database at various times, ages, disease states, etc. Cross-sectional analysis approaches, on the other hand, avoid complex methods for longitudinal missing data handling, but utilize less of the available data.

With all the considerations described above, an overarching principle is to apply alternative reasonable analysis approaches; consistency among the results lends confidence to the conclusions. When possible, consistency among results from different outcome measures, or multiple sources of external control data (sometimes preferable to attempting to combine the external control data sources), is valuable.

In the settings in which external controls are justified, it is critical to make optimal use of the available data, including retrospective natural history data, however imperfect. Statistical analysis should be carried out balancing practical matters with sound methodology.

Discussion

Despite cautionary guidance from regulators, the use of external controls, including retrospective natural history data, to support FDA decision making is neither new nor particularly unusual, especially for orphan drugs. This review identified 45 cases in select therapeutic areas over the past 20 years where external controls were used in the pivotal trials supporting product approval. Nearly half (44%) of the 45 cases evaluated used controls sourced from retrospective natural history data; about one-third (33%) used controls sourced from patients’ baseline data; and the remainder used controls sourced from published data or previous clinical studies. Of note, none of the 45 cases where external controls were used were sourced from prospectively collected natural history data, perhaps not surprising knowing the difficulties of performing a meaningful prospective natural history study in a realistic time frame. This is a critical point given that the regulatory precedent is contrary to the FDA guidance which identifies prospective natural history as the gold standard, while discouraging use of retrospective natural history [3, 4]. While prospective natural history studies may be ideal, such an approach is often impractical, would lead to significant delays in the availability of life-saving therapies, and could ultimately stifle the development of products for rare diseases.

Recently, FDA has communicated that it is less swayed by the size of natural history studies than by the rigor of data collection and clarity on the course of the disease [28], and has recommended the use of longitudinal rather than cross-sectional data sources, as they yield more comprehensive information about disease onset and progression over time [3]. However, in a rare disease setting it is more likely that only cross-sectional data are available. Moreover, even when longitudinal data are available, longitudinal and cross-sectional statistical approaches both have potential advantages and should be used based on the objective of the statistical analysis.

External controls have been most frequently leveraged in situations where the conduct of prospective randomized, controlled studies was not feasible; examples include products approved for rare, life-threatening or severely debilitating conditions, in some cases slowly progressing, with no/inadequate available therapy. Good data quality and appropriate statistical analyses (including appropriate sensitivity analyses) are important factors in reducing bias when comparing new treatments to external controls. Furthermore, the use of common data standards such as CDISC (Clinical Data Interchange Standards Consortium) and OMOP (Observational Medical Outcomes Partnership) might facilitate standardization of data collection in observational studies and further minimize bias. Nevertheless, for small rare disease clinical trials, such comparisons cannot be perfect, and the ultimate decision should rely on the totality of evidence, recognizing that some unresolved questions may remain.

Conclusion

Overall, the sponsors of the products identified appear to have achieved regulatory support to leverage external controls, including retrospective natural history, leading to successful product approval. Whilst acknowledging the limitations of the use of the external control, FDA invariably considered the nature and rarity of the condition, the unmet medical need, and ultimately the totality of evidence, including positive secondary or pharmacodynamic endpoints or positive data from supportive studies. Given this regulatory history, it is important to update FDA guidance to highlight the acceptability of retrospective natural history, to acknowledge the statistical challenges and provide recommendations for managing bias when using various types of external controls, and to facilitate information sharing. Such guidance would be a welcome addition to support clinical development and product approvals, especially in rare diseases.