Abstract
Randomization eliminates selection bias, and attenuates imbalance among study arms with respect to prognostic factors, both known and unknown. Thus, information arising from randomized clinical trials (RCTs) is typically considered the gold standard for comparing therapeutic interventions in confirmatory studies. However, RCTs are limited in contexts wherein patients who are willing to accept a random treatment assignment represent only a subset of the patient population. By contrast, observational studies (OSs) often enroll patient cohorts that better reflect the broader patient population. However, OSs often suffer from selection bias, and may yield invalid treatment comparisons even after adjusting for known confounders. Therefore, combining information acquired from OSs with data from RCTs in research synthesis is often criticized due to the limitations of OSs. In this article, we combine randomized and non-randomized substudy data from FIRST, a recent HIV/AIDS drug trial. We develop hierarchical Bayesian approaches devised to combine data from all sources simultaneously while explicitly accounting for potential discrepancies in the sources’ designs. Specifically, we describe a two-step approach combining propensity score matching and Bayesian hierarchical modeling to integrate information from non-randomized studies with data from RCTs, to an extent that depends on the estimated commensurability of the data sources. We investigate our procedure’s operating characteristics via simulation. Our findings have implications for HIV/AIDS research, as well as elucidate the extent to which well-designed non-randomized studies can complement RCTs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agresti, A., Min, Y.: Effects and noneffects of paired identical observations in comparing proportions with binary matched pairs data. Stat. Med. 23, 65–75 (2004)
Austin, P.: A critical appraisal of propensity score matching in the medical literature between 1996 and 2003. Stat. Med. 27, 2037–2049 (2008)
Austin, P.: An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46, 399–424 (2011)
Austin, P.: A comparison of 12 algorithms for matching on the propensity score. Stat. Med. 33, 1057–1069 (2014)
Austin, P., Mamdani, M.: A comparison of propensity score methods: a case study estimating the effectiveness of post-ami statin use. Stat. Med. 25, 2084–2106 (2006)
Austin, P., Grootendorst, P., Anderson, G.: A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26, 734–753 (2007)
Avorn, J., Aaron, S.: The 21st Century Cures Act—will it take us back in time? N. Engl. J. Med. 372, 2473–2475 (2015)
Baron, R., Kenny, D.: The moderator–mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J. Pers. Soc. Psychol. 51, 1173–1182 (1986)
Concato, J., Shah, N., Horwitz, R.: Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342, 1887–1892 (2000)
D’Agostino, R.: Tutorial in biostatistics: propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat. Med. 17, 2265–2281 (1998)
Evidence-Based Medicine Working Group: Evidence-based medicine. A new approach to teaching the practice of medicine. J. Am. Med. Assoc. 268, 2420–2425 (1992)
Greenland, S., Robins, J., Pearl, J.: Confounding and collapsibility in causal inference. Stat. Sci. 14, 29–46 (1999)
Ho, D., Imal, K., King, G., Stuart, E.: Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit. Anal. 15, 199–236 (2007)
Hobbs, B., Carlin, B., Mandrekar, S., Sargent, D.: Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials. Biometrics 67, 1047–1056 (2011)
Hobbs, B., Sargent, D., Carlin, B.: Commensurate priors for incorporating historical information in clinical trials using general and generalized linear models. Bayesian Anal. 7, 639–674 (2012)
Hobbs, B., Carlin, B., Sargent, D.: Adaptive adjustment of the randomization ratio using historical control data. Clin. Trials 10, 430–440 (2013)
Ibrahim, J., Chen, M.: Power prior distributions for regression models. Stat. Sci. 15, 46–60 (2000)
Imbens, G.: Nonparametric estimation of average treatment effects under exogeneity: a review. Rev. Econ. Stat. 86, 4–29 (2004)
Ioannidis, J.: Contradicted and initially stronger effects in highly cited clinical research. J. Am. Med. Assoc. 294, 218–228 (2005)
Kurth, T., Walker, A., Glynn, R., Chan, K., Gaziano, J., Berger, K., Robins, J.: Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. Am. J. Epidemiol. 163, 262–270 (2006)
Leth, F.V., Andrews, S., Grinsztejn, B., Wilkins, E., Lazanas, M., Lange, J., Montaner, J.: The effect of baseline CD4 cell count and HIV-1 viral load on the efficacy and safety of nevirapine or efavirenz-based first-line HAART. AIDS 19, 463–471 (2005)
Ligthelm, R., Borzi, V., Gumprecht, J., Kawamori, R., Wenying, Y., Valensi, P.: Importance of observational studies in clinical practice. Clin. Ther. 29, 1284–1292 (2007)
Lunceford, J., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23, 2937–2960 (2004)
Lunn, D., Spiegelhalter, D.J., Thomas, A., Best, N.: The BUGS project: evolution, critique and future directions. Stat. Med. 28, 3049–3067 (2009)
MacArthur, R., Chen, L., Mayers, D., Besch, C., Novak, R., van den Berg-Wolf, M., Yurik, T., Peng, G., Schmetter, B., Brizz, B., Abrams, D.: The rationale and design of the CPCRA (Terry Beirn Community Programs for Clinical Research on AIDS) 058 FIRST (Flexible Initial Retrovirus Suppressive Therapies) trial. Control. Clin. Trials 22, 176–190 (2001)
MacLehose, R., Reeves, B., Harvey, I., Sheldon, T., Russell, I., Black, A.: A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol. Assess. 4, 1–154 (2000)
McCulloch, C.E., Searle, S.R.: Generalized, Linear, and Mixed Models. Wiley, New York (2001)
Mitchell, T., Beauchamp, J.: Bayesian variable selection in linear regression. J. Am. Stat. Assoc. 83, 1023–1032 (1988)
Murray, T., Hobbs, B., Carlin, B.: Combining nonexchangeable functional or survival data sources in oncology using generalized mixture commensurate priors. Ann. Appl. Stat. 9, 1549–1570 (2015)
Newhouse, J., McClellan, M.: Econometrics in outcomes research: the use of instrumental variables. Annu. Rev. Public Health 19, 17–34 (1998)
Pearl, J.: Causal inference in statistics: an overview. Stat. Surv. 3, 96–146 (2009)
Pocock, S.: The combination of randomized and historical controls in clinical trials. J. Chronic Dis. 29, 175–188 (1976)
Rosenbaum, P., Rubin, D.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)
Rosenbaum, P., Rubin, D.: Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)
Rosenbaum, P., Rubin, D.: Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am. Stat. 39, 33–38 (1985)
Rubin, D.: For objective causal inference, design trumps analysis. Ann. Appl. Stat. 2, 808–840 (2008)
Stern, J., Robinson, P., Love, J., Lanes, S., Imperiale, M., Mayers, D.: A comprehensive hepatic safety analysis of nevirapine in different populations of HIV infected patients. J. Acquir. Immune Defic. Syndr. 34, S21–S33 (2003)
Stuart, E.: Matching methods for causal inference: a review and a look forward. Stat. Sci. 25, 1–21 (2010)
Upton, F., DeGette, D., Pitts, J., Pallone, F., Green, G.: 21st Century Cures Act (2015)
U.S. Preventive Services Task Force: Guide to Clinical Preventive Services: Report of the U.S. Preventive Services Task Force. Williams & Wilkins, Baltimore (1996)
van den Berg-Wolf, M., Hullsiek, K., Peng, G., Kozal, M., Novak, R., Chen, L., Crane, L., MacArthur, R.: Virologic, immunologic, clinical, safety, and resistance outcomes from a long-term comparison of efavirenz-based versus nevirapine-based antiretroviral regimens as initial therapy in HIV-1-infected persons. HIV Clin. Trials 9, 324–336 (2008)
Acknowledgments
The authors are grateful to Dr. Thomas Murray for helpful discussions regarding the two-spike prior distribution and other aspects of commensurate prior modeling, and to Drs. James Neaton and Kathy Huppler-Hullsiek for sharing the FIRST data and providing valuable insights regarding its proper interpretation.
Funding
The work of the first and last authors was supported in part by a grant from the Amgen Research Grant Program. The work of the second and last authors was supported in part by National Cancer Institute Grant 1-R01-CA157458-01A1. Finally, the work of the second author was supported in part by National Cancer Institute M.D. Anderson Cancer Center Support Grant P30-CA016672.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare they have no conflicts of interest.
Ethical approval
All analyses of human participants performed by the author for this work involved only secondary data analysis of pre-existing, deidentified datasets, preserving the subjects’ confidentiality. As such, the work is of the kind typically considered exempt from IRB approval in the United States.
Rights and permissions
About this article
Cite this article
Zhao, H., Hobbs, B.P., Ma, H. et al. Combining non-randomized and randomized data in clinical trials using commensurate priors. Health Serv Outcomes Res Method 16, 154–171 (2016). https://doi.org/10.1007/s10742-016-0155-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-016-0155-7