Skip to main content

Reducing Patient Burden in Clinical Trials Through the Use of Historical Controls: Appropriate Selection of Historical Data to Minimize Risk of Bias

Abstract

Historical data have been used to augment or replace control arms in some rare disease and pediatric clinical trials. With greater availability of historical data and new methodology such as dynamic borrowing, the inclusion of historical data in clinical trials is an increasingly appealing approach for larger disease areas as well, as this can result in increased power and precision and can minimize the burden on patients in clinical trials. However, sponsors must assess whether the potential biases incurred with this approach outweigh the benefits and discuss this trade-off with the regulatory agencies. This paper discusses important points for the appropriate selection of historical controls for inclusion in the analysis of primary and/or key secondary endpoint(s) in clinical trials. The general steps are as follows: (1) Assess whether a trial is a suitable candidate for this approach. (2) If it is, then carefully identify appropriate historical trials to minimize selection bias. (3) Refine the historical control set if appropriate, for example, by selecting subsets of studies or patients. Identification of trial settings that are amenable to historical borrowing and selection of appropriate historical data using the principles discussed in this paper has the potential to lead to more efficient estimation and decision making. Ultimately, this efficiency gain results in lower patient burden and gets effective drugs to patients more quickly.

This is a preview of subscription content, access via your institution.

Figure 1.
Figure 2.

References

  1. A Medical Research Council Investigation. Streptomycin treatment of pulmonary tuberculosis. Br Med J. 1948;1948(2):769–82.

    Google Scholar 

  2. Santayana G. Reason in common sense. In: New York Charles Scribner’s Sons, editor. The life of reason, vol. 1. London: Constable and Co. Ltd.; 1905. p. 284.

    Google Scholar 

  3. Lim J, Walley R, Yuan J, et al. Minimizing patient burden through the use of historical subject-level data in innovative confirmatory clinical trials: review of methods and opportunities. Ther Innov Regul Sci. 2018;52(5):546–59. https://doi.org/10.1177/2168479018778282.

    Article  PubMed  Google Scholar 

  4. Viele K, Berry S, Neuenschwander B, et al. Use of historical control data for assessing treatment effects in clinical trials. Pharm Stat. 2014;13:41–54. https://doi.org/10.1002/pst.1589.

    Article  PubMed  Google Scholar 

  5. Food and Drug Administration Center for Devices and Radiological Health (FDA CDRH). Guidance for the use of bayesian statistics in medical device clinical trials. Rockville, MD. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials-pdf-version. 2010.

  6. Food and Drug Administration Draft Guidance (Center for Drug Evaluation and Research [CDER]/Center for Biologics Evaluation and Research [CBER]). Adaptive designs for clinical trials of drugs and biologics. Rockville, MD. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/adaptive-design-clinical-trials-drugs-and-biologics. 2018.

  7. TransCelerate PSoC meeting with FDA (11 May 2017) and EMA (22 Aug 2017): unofficial minutes.

  8. Viele K, Mundy LM, Noble RB, Li G, Broglio K, Wetherington JD. Phase 3 adaptive trial design options in treatment of complicated urinary tract infection. Pharm Stat. 2018;17:811–22. https://doi.org/10.1002/pst.1892.

    Article  PubMed  Google Scholar 

  9. Chen MH, Ibrahim JG. The relationship between the power priors and hierarchical models. Bayesian Anal. 2006;1:551–74.

    Article  Google Scholar 

  10. Schmidli H, Steiger S, Roychoudhury S, O’Hagan A, Spiegelhalter D, Neuenschwander B. Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics. 2014;70:1023–32. https://doi.org/10.1111/biom.12242.

    Article  PubMed  Google Scholar 

  11. Han B, Zhan J, John Zhong Z, Liu D, Lindborg S. Covariate-adjusted borrowing of historical control data in randomized clinical trials. Pharm. Stat. 2017;16:296–308. https://doi.org/10.1002/pst.1815.

    Article  PubMed  Google Scholar 

  12. Stuart EA, Rubin DB. Matching with multiple vontrol groups with adjustment for group differences. J Educ Behav Stat. 2008;33(3):279–306.

    Article  Google Scholar 

  13. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.

    Article  Google Scholar 

  14. Weinreb RN, Kaufman PL. The glaucoma research community and FDA look to the future: a report from the NEI/FDA CDER glaucoma clinical trial design and endpoints symposium. Invest Ophthalmol Vis Sci. 2009;50(4):1497–505.

    Article  Google Scholar 

  15. Iovieno N, Papakostas GI. Does the presence of an open-label antidepressant treatment period influence study outcome in clinical trials examining augmentation/combination strategies in treatment partial responders/nonresponders with major depressive disorder? J Clin Psychiatry. 2012;73(5):676–83. https://doi.org/10.4088/JCP.11r06978.

    CAS  Article  PubMed  Google Scholar 

  16. Moher D, Liberati A, Tetzlaff J, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. https://doi.org/10.7326/0003-4819-151-4-200908180-00135.

    Article  Google Scholar 

  17. Russo M. How to review a meta-analysis. Gastroenterol Hepatol (NY). 2007;3(8):637–42.

    Google Scholar 

  18. Rothstein H, Sutton AJ, Borenstein M. Publication bias in meta-analysis: prevention, assessment and adjustments. Hoboken: Wiley; 2005.

    Book  Google Scholar 

  19. Pocock S. The combination of randomized and historical controls in clinical trials. J Chronic Dis. 1976;29(3):175–88.

    CAS  Article  Google Scholar 

  20. Transcelerate BioPharma Inc. Placebo and standard of care data sharing. https://transceleratebiopharmainc.com/initiatives/placebo-standard-of-care. 2017.

  21. ProjectDataSphereLLC. https://www.projectdatasphere.org/projectdatasphere/html/home/. 2013.

  22. Critical Path Institute. https://c-path.org/. 2005.

  23. Vivli Center for Global Clinical Research Data. https://vivli.org/about/www-vivli-org-data-sharing-platform/. 2013.

  24. ClinicalStudyDataRequest. https://clinicalstudydatarequest.com/Default.aspx. 2014.

  25. The Yale University Open Data Access (YODA) Project. https://yoda.yale.edu/. 2014.

  26. FinnGen. https://www.finngen.fi/en/FinnGenresearchprojectisanexpeditiontothefrontierofgenomicsandmedicine. 2017.

  27. Gamalo MA, Tiwari RC, LaVange LM. Bayesian approach to the design and analysis of non-inferiority trials for anti-infective products. Pharm Stat. 2014;13:25–40. https://doi.org/10.1002/pst.1588.

    Article  PubMed  Google Scholar 

  28. Weber K, Hemmings R, Koch A. How to use prior knowledge and still give new data a chance? Pharm Stat. 2018;17:329–41. https://doi.org/10.1002/pst.1862.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Pocock S. The justification for randomized controlled trials. In: Pocock SJ, editor. Clinical trials: a practical approach, vol. 1. New York: Wiley; 2013. p. 50–65.

    Chapter  Google Scholar 

  30. Austin P. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar Behav Res. 2011;46(3):399–424. https://doi.org/10.1080/00273171.2011.568786.

    Article  Google Scholar 

  31. ICHE9(R1) Statistical principles for clinical trials: addendum: estimands and sensitivity analysis in clinical trials; international council for harmonisation; draft guidance for industry. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-ich-e9-r1-addendum-estimands-sensitivity-analysis-clinical-trials-guideline-statistical_en.pdf. 2017.

  32. Perucca E. Marketed new antiepileptic drugs: are they better than old-generation agents? Ther Drug Monit. 2002;24(1):74–80.

    Article  Google Scholar 

  33. LaRoche SM. A new look at the second-generation antiepileptic drugs: a decade of experience. Neurologist. 2007;13(3):133–9.

    Article  Google Scholar 

  34. Turner RM, Spiegelhalter DJ, Smith GC, Thompson SG. Bias modelling in evidence synthesis. J R Stat Soc Ser A. 2009;72(1):21–47.

    Article  Google Scholar 

  35. Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  Google Scholar 

Download references

Acknowledgements

All authors were involved in the writing and editing of the article. The authors gratefully acknowledge the support of TransCelerate BioPharma Inc. The authors also gratefully acknowledge the support of the following people who contributed to this manuscript: Saba Tahseen, PA Consulting; Ryan Feld, Accenture; Mark Donovan, BMS; Christina Fahmy, Kilpatrick Townsend; Ruchira Glaser, GSK; Takahiro Hasegawa, Shionogi; Rob Hemmings, Consilium, Salmonson and Hemmings; Alistair Lindsay, GSK; Mary Ann Lukas, GSK; Paul Nitschmann, Amgen; Frances Pu, Renaissance Writing Services.

Funding

No funding sources were provided to the authors.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jessica Lim.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest. However, all authors are employees and/or stockholders of the companies with which they are affiliated.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Glossary

Appendix 1: Glossary

Terms Definition/Explanation
Concurrent control Control (e.g., placebo or standard of care) arm data from the clinical trial of interest
Historical control Control arm data from a clinical trial run prior to the trial of interest
External control Similar to historical control—could be from a clinical trial running concurrently to the trial of interest
Bias Discrepancy between the true (unknown) parameter and expected value
Drift The discrepancy between the true (unknown) current control parameter and the observed historical data
Window of benefit In the context of this paper, when incorporating historical controls, the Window of Benefit is defined as the range of differences between the true current control parameter and the observed historical data where improved operating characteristics are observed. Outside of this range, where the differences are larger, operating characteristics are poorer.
Cherry-picking In this context, choosing historical data in a biased manner
Partial replacement Replacing a portion of a planned control arm with historical controls to achieve the target sample size
Augmentation Adding historical controls to a planned control arm to increase the power from the originally planned sample size
Weight/Weighting The weight/weighting determines the amount of historical information incorporated into an analysis
Downweighting A general term representing statistical techniques used to reduce the influence of historical data on the outcome of an analysis according to the level of agreement or discordance between the historical and concurrent data
Bayesian/frequentist framework Guiding principles of Bayesian or Frequentist statistics
Operating characteristics How a clinical trial will perform under various assumptions. For example, the probability of obtaining a false-positive or false-negative result.
Type I error Assuming there is no true difference between two treatments, the probability of concluding a difference (false positive)
Power Assuming there is a true difference between two treatments, the probability of concluding a difference (true positive)
Point estimate A single value used as an estimate of a parameter
Variance A measure of the spread of the data
Precision The inverse of the variance—larger values for precision correspond to smaller variability
Patient-level data Historical data that are available for individual patients that participated in the study (in contrast to data available only at summary-level)
Estimand The target of estimation to address the scientific question of interest posed by the trial objective—for more details see ICH E9(R1)
Intercurrent Events that occur after treatment initiation and either preclude observation of the outcome or affect its interpretation

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lim, J., Wang, L., Best, N. et al. Reducing Patient Burden in Clinical Trials Through the Use of Historical Controls: Appropriate Selection of Historical Data to Minimize Risk of Bias. Ther Innov Regul Sci 54, 850–860 (2020). https://doi.org/10.1007/s43441-019-00014-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43441-019-00014-4

Keywords

  • TransCelerate
  • Historical controls
  • Clinical trials
  • Selection bias