International Journal of Clinical Pharmacy

, Volume 38, Issue 3, pp 714–723 | Cite as

Methods to control for unmeasured confounding in pharmacoepidemiology: an overview

  • Md. Jamal UddinEmail author
  • Rolf H. H. Groenwold
  • Mohammed Sanni Ali
  • Anthonius de Boer
  • Kit C. B. Roes
  • Muhammad A. B. Chowdhury
  • Olaf H. Klungel
Review Article


Background Unmeasured confounding is one of the principal problems in pharmacoepidemiologic studies. Several methods have been proposed to detect or control for unmeasured confounding either at the study design phase or the data analysis phase. Aim of the Review To provide an overview of commonly used methods to detect or control for unmeasured confounding and to provide recommendations for proper application in pharmacoepidemiology. Methods/Results Methods to control for unmeasured confounding in the design phase of a study are case only designs (e.g., case-crossover, case-time control, self-controlled case series) and the prior event rate ratio adjustment method. Methods that can be applied in the data analysis phase include, negative control method, perturbation variable method, instrumental variable methods, sensitivity analysis, and ecological analysis. A separate group of methods are those in which additional information on confounders is collected from a substudy. The latter group includes external adjustment, propensity score calibration, two-stage sampling, and multiple imputation. Conclusion As the performance and application of the methods to handle unmeasured confounding may differ across studies and across databases, we stress the importance of using both statistical evidence and substantial clinical knowledge for interpretation of the study results.


Observational studies Pharmacoepidemiology Residual confounding Review Statistical methods Unmeasured confounding Unobserved confounding 




Conflicts of interest

There are no financial, personal, political, academic or other relations that could lead to a conflict of interest.


  1. 1.
    Grobbee DE, Hoes AW. Clinical epidemiology: principles, methods, and applications for clinical research. Jones & Bartlett Learning, ISBN: 1449674321; 2009.Google Scholar
  2. 2.
    Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet. 2004;363(9422):1728–31.CrossRefPubMedGoogle Scholar
  3. 3.
    Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, et al. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC good practice task force report. Value Health. 2014;17(2):143–56.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Feinstein AR. Current problems and future challenges in randomized clinical trials. Circulation. 1984;70(5):767–74.CrossRefPubMedGoogle Scholar
  5. 5.
    Black N. Why we need observational studies to evaluate the effectiveness of health care. Br Med J. 1996;312(7040):1215–8.CrossRefGoogle Scholar
  6. 6.
    Brookhart MA, Stürmer T, Glynn RJ, Rassen J, Schneeweiss S. Confounding control in healthcare database research: challenges and potential approaches. Med Care. 2010;48(6 SUPPL.):S114–20.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Baiocchi M, Cheng J, Small DS. Instrumental variable methods for causal inference. Stat Med. 2014;33(13):2297–340.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    McMahon AD. Approaches to combat with confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol Drug Saf. 2003;12(7):551–8.CrossRefPubMedGoogle Scholar
  9. 9.
    Klungel OH, Martens EP, Psaty BM, Grobbee DE, Sullivan SD, Stricker BHC, et al. Methods to assess intended effects of drug treatment in observational studies are reviewed. J Clin Epidemiol. 2004;57(12):1223–31.CrossRefPubMedGoogle Scholar
  10. 10.
    Bosco JLF, Silliman RA, Thwin SS, Geiger AM, Buist DSM, Prout MN, et al. A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. J Clin Epidemiol. 2010;63(1):64–74.CrossRefPubMedGoogle Scholar
  11. 11.
    Groenwold RHH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol. 2009;62(1):22–8.CrossRefPubMedGoogle Scholar
  12. 12.
    Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, editors. Developing a protocol for observational comparative effectiveness research: a user’s guide. AHRQ Publication No. 12(13)-EHC099. Rockville, MD: Agency for Healthcare Research and Quality; January 2013.
  13. 13.
    VanderWeele TJ, Shpitser I. On the definition of a confounder. Ann Stat. 2013;41(1):196–220.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Alemayehu D, Alvir JM, Jones B, Willke RJ. Statistical issues with the analysis of nonrandomized studies in comparative effectiveness research. J Manag Care Pharm. 2011;17(9 Suppl A):S22–6.PubMedGoogle Scholar
  15. 15.
    Schneeweiss S, Seeger JD, Smith SR. Methods for developing and analyzing clinically rich data for patient-centered outcomes research: an overview. Pharmacoepidemiol Drug Saf. 2012;21(Suppl. 2):1–5.CrossRefGoogle Scholar
  16. 16.
    Groenwold RHH, Hoes A, Nichol KL, Hak E. Quantifying the potential role of unmeasured confounders: the example of influenza vaccination. Int J Epidemiol. 2008;37(6):1422–9.CrossRefPubMedGoogle Scholar
  17. 17.
    Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.CrossRefPubMedGoogle Scholar
  18. 18.
    Brookhart MA, Wang PS, Solomon DH, Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology. 2006;17(3):268–75.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009;63(5):691–7.CrossRefPubMedGoogle Scholar
  20. 20.
    Bradbury BD, Gilbertson DT, Brookhart MA, Kilpatrick RD. Confounding and control of confounding in nonexperimental studies of medications in patients with CKD. Adv Chronic Kidney Dis. 2012;19(1):19–26.CrossRefPubMedGoogle Scholar
  21. 21.
    Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and confounding in critical care health services research involving observational data. J Crit Care. 2006;21(1):1–7.CrossRefPubMedGoogle Scholar
  22. 22.
    Stürmer T, Glynn RJ, Rothman KJ, Avorn J, Schneeweiss S. Adjustments for unmeasured confounders in pharmacoepidemiologic database studies using external information. Med Care. 2007;45(10 SUPPL. 2):S158–65.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Maclure M. The case-crossover design: a method for studying transient effects on the risk of acute events. Am J Epidemiol. 1991;133(2):144–53.PubMedGoogle Scholar
  24. 24.
    Nordmann S, Biard L, Ravaud P, Esposito-Farèse M, Tubach F. Case-only designs in pharmacoepidemiology: a systematic review. PLoS One. 2012;7(11):e49444.CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Schneeweiss S, Stürmer T, Maclure M. Case-crossover and case-time-control designs as alternatives in pharmacoepidemiologic research. Pharmacoepidemiol Drug Saf. 1997;6(SUPPL. 3):S51–9.CrossRefPubMedGoogle Scholar
  26. 26.
    Delaney JA, Suissa S. The case-crossover study design in pharmacoepidemiology. Stat Methods Med Res. 2009;18(1):53–65.CrossRefPubMedGoogle Scholar
  27. 27.
    Aberra FN, Brensinger CM, Bilker WB, Lichtenstein GR, Lewis JD. Antibiotic use and the risk of flare of inflammatory bowel disease. Clin Gastroenterol Hepatol. 2005;3(5):459–65.CrossRefPubMedGoogle Scholar
  28. 28.
    Ray WA, Fought RL, Decker MD. Psychoactive drugs and the risk of injurious motor vehicle crashes in elderly drivers. Am J Epidemiol. 1992;136(7):873–83.CrossRefPubMedGoogle Scholar
  29. 29.
    Hebert C, Delaney J, Hemmelgarn B, Lévesque LE, Suissa S. Benzodiazepines and elderly drivers: a comparison of pharmacoepidemiological study designs. Pharmacoepidemiol Drug Saf. 2007;16(8):845–9.CrossRefPubMedGoogle Scholar
  30. 30.
    Suissa S. The case-time-control design. Epidemiology. 1995;6(3):248–53.CrossRefPubMedGoogle Scholar
  31. 31.
    Wang SV, Coull BA, Schwartz J, Mittleman MA, Wellenius GA. Potential for bias in case-crossover studies with shared exposures analyzed using SAS. Am J Epidemiol. 2011;174(1):118–24.CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Hernandez-Diaz S, Hernan MA, Meyer K, Werler MM, Mitchell AA. Case-crossover and case-time-control designs in birth defects epidemiology. Am J Epidemiol. 2003;158(4):385–91.CrossRefPubMedGoogle Scholar
  33. 33.
    Schneider MF, Gange SJ, Margolick JB, Detels R, Chmiel JS, Rinaldo C, et al. Application of case-crossover and case–time–control study designs in analyses of time-varying predictors of T-cell homeostasis failure. Ann Epidemiol. 2005;15(2):137–44.CrossRefPubMedGoogle Scholar
  34. 34.
    Farrington CP. Relative incidence estimation from case series for vaccine safety evaluation. Biometrics. 1995;51(1):228–35.CrossRefPubMedGoogle Scholar
  35. 35.
    Whitaker HJ, Hocine MN, Farrington CP. The methodology of self-controlled case series studies. Stat Methods Med Res. 2009;18(1):7–26.CrossRefPubMedGoogle Scholar
  36. 36.
    Whitaker HJ, Farrington CP, Spiessens B, Musonda P. Tutorial in biostatistics: the self-controlled case series method. Stat Med. 2006;25(10):1768–97.CrossRefPubMedGoogle Scholar
  37. 37.
    The Self Controlled Case Series Method (internet). The Open University, UK (updated 2010; cited 2015 Dec 4).
  38. 38.
    Yu M, Xie D, Wang X, Weiner MG, Tannen RL. Prior event rate ratio adjustment: numerical studies of a statistical method to address unrecognized confounding in observational studies. Pharmacoepidemiol Drug Saf. 2012;21(Suppl. 2):60–8.CrossRefPubMedGoogle Scholar
  39. 39.
    Tannen RL, Weiner MG, Xie D. Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings. BMJ. 2009;338(7691):395–9.Google Scholar
  40. 40.
    Tannen R, Xie D, Wang X, Yu M, Weiner MG. A new “Comparative Effectiveness” assessment strategy using the THIN database: comparison of the cardiac complications of pioglitazone and rosiglitazone. Pharmacoepidemiol Drug Saf. 2013;22(1):86–97.CrossRefPubMedGoogle Scholar
  41. 41.
    Tannen RL, Weiner MG, Xie D. Replicated studies of two randomized trials of angiotensin-converting enzyme inhibitors: further empiric validation of the ‘prior event rate ratio’ to adjust for unmeasured confounding by indication. Pharmacoepidemiol Drug Saf. 2008;17(7):671–85.CrossRefPubMedGoogle Scholar
  42. 42.
    Weiner MG, Xie D, Tannen RL. Replication of the Scandinavian Simvastatin Survival Study using a primary care medical record database prompted exploration of a new method to address unmeasured confounding. Pharmacoepidemiol Drug Saf. 2008;17(7):661–70.CrossRefPubMedGoogle Scholar
  43. 43.
    Uddin MJ, Groenwold RHH, Van Staa TP, De Boer A, Belitser SV, Hoes AW, et al. Performance of prior event rate ratio adjustment method in pharmacoepidemiology: a simulation study. Pharmacoepidemiol Drug Saf. 2015;24(5):468–77.CrossRefPubMedGoogle Scholar
  44. 44.
    Lipsitch M, Tchetgen E Tchetgen, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–8.CrossRefPubMedPubMedCentralGoogle Scholar
  45. 45.
    Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat Med. 2014;33(2):209–18.CrossRefPubMedGoogle Scholar
  46. 46.
    Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35(2):337–44.CrossRefPubMedGoogle Scholar
  47. 47.
    Flanders WD, Klein M, Darrow LA, Strickland MJ, Sarnat SE, Sarnat JA, et al. A method for detection of residual confounding in time-series and other observational studies. Epidemiology. 2011;22(1):59–67.CrossRefPubMedPubMedCentralGoogle Scholar
  48. 48.
    Tchetgen E Tchetgen. The control outcome calibration approach for causal inference with unobserved confounding. Am J Epidemiol. 2014;179(5):633–40.CrossRefGoogle Scholar
  49. 49.
    Groenwold RHH. Falsification end points for observational studies. JAMA. 2013;309(17):1769–70.CrossRefPubMedGoogle Scholar
  50. 50.
    Rassen JA, Brookhart MA, Glynn RJ, Mittleman MA, Schneeweiss S. Instrumental variables I: instrumental variables exploit natural variation in nonexperimental data to estimate causal relationships. J Clin Epidemiol. 2009;62(12):1226–32.CrossRefPubMedPubMedCentralGoogle Scholar
  51. 51.
    Chen Y, Briesacher BA. Use of instrumental variable in prescription drug research with observational data: a systematic review. J Clin Epidemiol. 2011;64(6):687–700.CrossRefPubMedGoogle Scholar
  52. 52.
    Davies NM, Smith GD, Windmeijer F, Martin RM. Issues in the reporting and conduct of instrumental variable studies: a systematic review. Epidemiology. 2013;24(3):363–9.CrossRefPubMedGoogle Scholar
  53. 53.
    Martens EP, Pestman WR, De Boer A, Belitser SV, Klungel OH. Instrumental variables: application and limitations. Epidemiology. 2006;17(3):260–7.CrossRefPubMedGoogle Scholar
  54. 54.
    Hernán MA, Robins JM. Instruments for causal inference: an epidemiologist’s dream? Epidemiology. 2006;17(4):360–72.CrossRefPubMedGoogle Scholar
  55. 55.
    Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000;29(4):722–9.CrossRefPubMedGoogle Scholar
  56. 56.
    Schneeweiss S, Setoguchi S, Brookhart A, Dormuth C, Wang PS. Risk of death associated with the use of conventional versus atypical antipsychotic drugs among elderly patients. Can Med Assoc J. 2007;176(5):627.CrossRefGoogle Scholar
  57. 57.
    Uddin MJ, Groenwold RHH, de Boer A, Belitser S, Roes KCB, Hoes AW, et al. Performance of instrumental variable methods in cohort and nested case–control studies: a simulation study. Pharmacoepidemiol Drug Saf. 2014;23:165–77.CrossRefPubMedGoogle Scholar
  58. 58.
    Brookhart MA, Rassen JA, Schneeweiss S. Instrumental variable methods in comparative safety and effectiveness research. Pharmacoepidemiol Drug Saf. 2010;19(6):537–44.CrossRefPubMedPubMedCentralGoogle Scholar
  59. 59.
    Instrumental Variables (internet). Econometrics Academy (updated 2015; cited 2015 Dec 4).
  60. 60.
    Lin NX, Logan S, Henley WE. Bias and sensitivity analysis when estimating treatment effects from the cox model with omitted covariates. Biometrics. 2013;69(4):850–60.CrossRefPubMedPubMedCentralGoogle Scholar
  61. 61.
    Groenwold RH, Nelson DB, Nichol KL, Hoes AW, Hak E. Sensitivity analyses to estimate the potential impact of unmeasured confounding in causal research. Int J Epidemiol. 2010;39(1):107–17.CrossRefPubMedGoogle Scholar
  62. 62.
    Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25(6):1107–16.CrossRefPubMedGoogle Scholar
  63. 63.
    Lee WC. Detecting and correcting the bias of unmeasured factors using perturbation analysis: a data-mining approach. BMC Med Res Methodol. 2014;14(1):18.CrossRefPubMedPubMedCentralGoogle Scholar
  64. 64.
    Vanderweele TJ, Arah OA. Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology. 2011;22(1):42–52.CrossRefPubMedPubMedCentralGoogle Scholar
  65. 65.
    Wen SW, Kramer MS. Uses of ecologic studies in the assessment of intended treatment effects. J Clin Epidemiol. 1999;52(1):7–12.CrossRefPubMedGoogle Scholar
  66. 66.
    Morgenstern H. Ecologic studies in epidemiology: concepts, principles, and methods. Annu Rev Public Health. 1995;16:61–81.CrossRefPubMedGoogle Scholar
  67. 67.
    Groome PA, Mackillop WJ, Naylor CD. Uses of ecologic studies in the assessment of intended treatment effects multiple letters. J Clin Epidemiol. 1999;52(9):903–4.CrossRefPubMedGoogle Scholar
  68. 68.
    Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin Pharmacol Ther. 2007;82(2):143–56.CrossRefPubMedGoogle Scholar
  69. 69.
    Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.CrossRefGoogle Scholar
  70. 70.
    Ali MS, Groenwold RH, Belitser SV, Pestman WR, Hoes AW, Roes KC, et al. Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: a systematic review. J Clin Epidemiol. 2015;68(2):112–21.CrossRefPubMedGoogle Scholar
  71. 71.
    Stürmer T, Schneeweiss S, Avorn J, Glynn RJ. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am J Epidemiol. 2005;162(3):279–89.CrossRefPubMedPubMedCentralGoogle Scholar
  72. 72.
    Sturmer T, Schneeweiss S, Rothman KJ, Avorn J, Glynn RJ. Performance of propensity score calibration—a simulation study. Am J Epidemiol. 2007;165(10):1110–8.CrossRefPubMedPubMedCentralGoogle Scholar
  73. 73.
    Lunt M, Glynn RJ, Rothman KJ, Avorn J, Stürmer T. Propensity score calibration in the absence of surrogacy. Am J Epidemiol. 2012;175(12):1294–302.CrossRefPubMedPubMedCentralGoogle Scholar
  74. 74.
    Collet JP, Schaubel D, Hanley J, Sharpe C, Boivin JF. Controlling confounding when studying large pharmacoepidemiologic databases: a case study of the two-stage sampling design. Epidemiology. 1998;9(3):309–15.CrossRefPubMedGoogle Scholar
  75. 75.
    Hanley JA, Dendukuri N. Efficient sampling approaches to address confounding in database studies. Stat Methods Med Res. 2009;18(1):81–105.CrossRefPubMedGoogle Scholar
  76. 76.
    Stürmer T, Schneeweiss S, Rothman KJ, Avorn J, Glynn RJ. Comparison of performance of propensity score calibration (PSC) and multiple imputation (MI) to control for unmeasured confounding using an internal validation study. Pharmacoepidemiol Drug Saf. 2006;15:S39–40 (abstract).CrossRefGoogle Scholar
  77. 77.
    Faries D, Peng X, Pawaskar M, Price K, Stamey JD, Seaman JW. Evaluating the impact of unmeasured confounding with internal validation data: an example cost evaluation in type 2 diabetes. Value Health. 2013;16(2):259–66.CrossRefPubMedGoogle Scholar
  78. 78.
    Yao X, Lix L. A flexible method to apply multiple imputation using SAS/IML® studio. SAS Global Forum 2013; Paper 283-2013.Google Scholar
  79. 79.
    Soley-Bori M. Dealing with missing data: key assumptions and methods for applied analysis 2013.Google Scholar
  80. 80.
    Horton NJ, Lipsitz SR. Multiple imputation in practice: comparison of software packages for regression models with missing variables. Am Stat. 2001;55(3):244–54.CrossRefGoogle Scholar
  81. 81.
    Miettinen OS. The need for randomization in the study of intended effects. Stat Med. 1983;2(2):267–71.CrossRefPubMedGoogle Scholar
  82. 82.
    Joseph K, Mehrabadi A, Lisonkova S. Confounding by indication and related concepts. Curr Epidemiol Rep. 2014;1(1):1–8.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing 2016

Authors and Affiliations

  • Md. Jamal Uddin
    • 1
    • 2
    Email author
  • Rolf H. H. Groenwold
    • 3
  • Mohammed Sanni Ali
    • 3
  • Anthonius de Boer
    • 2
  • Kit C. B. Roes
    • 3
  • Muhammad A. B. Chowdhury
    • 4
  • Olaf H. Klungel
    • 2
  1. 1.Department of Statistics (Biostatistics and Epidemiology)Shahjalal University of Science and TechnologySylhetBangladesh
  2. 2.Division of Pharmacoepidemiology and Clinical PharmacologyUniversity of UtrechtUtrechtThe Netherlands
  3. 3.Julius Center for Health Sciences and Primary CareUniversity Medical Center UtrechtUtrechtThe Netherlands
  4. 4.Department of Biostatistics, Robert Stempel College of Public Health and Social WorkFlorida International UniversityMiamiUSA

Personalised recommendations