Skip to main content
SpringerLink
Log in

Methods to control for unmeasured confounding in pharmacoepidemiology: an overview

  • Review Article
  • Published:
International Journal of Clinical Pharmacy Aims and scope Submit manuscript

Abstract

Background Unmeasured confounding is one of the principal problems in pharmacoepidemiologic studies. Several methods have been proposed to detect or control for unmeasured confounding either at the study design phase or the data analysis phase. Aim of the Review To provide an overview of commonly used methods to detect or control for unmeasured confounding and to provide recommendations for proper application in pharmacoepidemiology. Methods/Results Methods to control for unmeasured confounding in the design phase of a study are case only designs (e.g., case-crossover, case-time control, self-controlled case series) and the prior event rate ratio adjustment method. Methods that can be applied in the data analysis phase include, negative control method, perturbation variable method, instrumental variable methods, sensitivity analysis, and ecological analysis. A separate group of methods are those in which additional information on confounders is collected from a substudy. The latter group includes external adjustment, propensity score calibration, two-stage sampling, and multiple imputation. Conclusion As the performance and application of the methods to handle unmeasured confounding may differ across studies and across databases, we stress the importance of using both statistical evidence and substantial clinical knowledge for interpretation of the study results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Grobbee DE, Hoes AW. Clinical epidemiology: principles, methods, and applications for clinical research. Jones & Bartlett Learning, ISBN: 1449674321; 2009.

  2. Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet. 2004;363(9422):1728–31.

    Article  PubMed  Google Scholar 

  3. Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, et al. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC good practice task force report. Value Health. 2014;17(2):143–56.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Feinstein AR. Current problems and future challenges in randomized clinical trials. Circulation. 1984;70(5):767–74.

    Article  CAS  PubMed  Google Scholar 

  5. Black N. Why we need observational studies to evaluate the effectiveness of health care. Br Med J. 1996;312(7040):1215–8.

    Article  CAS  Google Scholar 

  6. Brookhart MA, Stürmer T, Glynn RJ, Rassen J, Schneeweiss S. Confounding control in healthcare database research: challenges and potential approaches. Med Care. 2010;48(6 SUPPL.):S114–20.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Baiocchi M, Cheng J, Small DS. Instrumental variable methods for causal inference. Stat Med. 2014;33(13):2297–340.

    Article  PubMed  PubMed Central  Google Scholar 

  8. McMahon AD. Approaches to combat with confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol Drug Saf. 2003;12(7):551–8.

    Article  PubMed  Google Scholar 

  9. Klungel OH, Martens EP, Psaty BM, Grobbee DE, Sullivan SD, Stricker BHC, et al. Methods to assess intended effects of drug treatment in observational studies are reviewed. J Clin Epidemiol. 2004;57(12):1223–31.

    Article  PubMed  Google Scholar 

  10. Bosco JLF, Silliman RA, Thwin SS, Geiger AM, Buist DSM, Prout MN, et al. A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. J Clin Epidemiol. 2010;63(1):64–74.

    Article  PubMed  Google Scholar 

  11. Groenwold RHH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol. 2009;62(1):22–8.

    Article  CAS  PubMed  Google Scholar 

  12. Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, editors. Developing a protocol for observational comparative effectiveness research: a user’s guide. AHRQ Publication No. 12(13)-EHC099. Rockville, MD: Agency for Healthcare Research and Quality; January 2013. www.effectivehealthcare.ahrq.gov/Methods-OCER.cfm.

  13. VanderWeele TJ, Shpitser I. On the definition of a confounder. Ann Stat. 2013;41(1):196–220.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Alemayehu D, Alvir JM, Jones B, Willke RJ. Statistical issues with the analysis of nonrandomized studies in comparative effectiveness research. J Manag Care Pharm. 2011;17(9 Suppl A):S22–6.

    PubMed  Google Scholar 

  15. Schneeweiss S, Seeger JD, Smith SR. Methods for developing and analyzing clinically rich data for patient-centered outcomes research: an overview. Pharmacoepidemiol Drug Saf. 2012;21(Suppl. 2):1–5.

    Article  Google Scholar 

  16. Groenwold RHH, Hoes A, Nichol KL, Hak E. Quantifying the potential role of unmeasured confounders: the example of influenza vaccination. Int J Epidemiol. 2008;37(6):1422–9.

    Article  CAS  PubMed  Google Scholar 

  17. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.

    Article  PubMed  Google Scholar 

  18. Brookhart MA, Wang PS, Solomon DH, Schneeweiss S. Evaluating short-term drug effects using a physician-specific prescribing preference as an instrumental variable. Epidemiology. 2006;17(3):268–75.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lu CY. Observational studies: a review of study designs, challenges and strategies to reduce confounding. Int J Clin Pract. 2009;63(5):691–7.

    Article  CAS  PubMed  Google Scholar 

  20. Bradbury BD, Gilbertson DT, Brookhart MA, Kilpatrick RD. Confounding and control of confounding in nonexperimental studies of medications in patients with CKD. Adv Chronic Kidney Dis. 2012;19(1):19–26.

    Article  PubMed  Google Scholar 

  21. Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and confounding in critical care health services research involving observational data. J Crit Care. 2006;21(1):1–7.

    Article  PubMed  Google Scholar 

  22. Stürmer T, Glynn RJ, Rothman KJ, Avorn J, Schneeweiss S. Adjustments for unmeasured confounders in pharmacoepidemiologic database studies using external information. Med Care. 2007;45(10 SUPPL. 2):S158–65.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Maclure M. The case-crossover design: a method for studying transient effects on the risk of acute events. Am J Epidemiol. 1991;133(2):144–53.

    CAS  PubMed  Google Scholar 

  24. Nordmann S, Biard L, Ravaud P, Esposito-Farèse M, Tubach F. Case-only designs in pharmacoepidemiology: a systematic review. PLoS One. 2012;7(11):e49444.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Schneeweiss S, Stürmer T, Maclure M. Case-crossover and case-time-control designs as alternatives in pharmacoepidemiologic research. Pharmacoepidemiol Drug Saf. 1997;6(SUPPL. 3):S51–9.

    Article  PubMed  Google Scholar 

  26. Delaney JA, Suissa S. The case-crossover study design in pharmacoepidemiology. Stat Methods Med Res. 2009;18(1):53–65.

    Article  PubMed  Google Scholar 

  27. Aberra FN, Brensinger CM, Bilker WB, Lichtenstein GR, Lewis JD. Antibiotic use and the risk of flare of inflammatory bowel disease. Clin Gastroenterol Hepatol. 2005;3(5):459–65.

    Article  PubMed  Google Scholar 

  28. Ray WA, Fought RL, Decker MD. Psychoactive drugs and the risk of injurious motor vehicle crashes in elderly drivers. Am J Epidemiol. 1992;136(7):873–83.

    Article  CAS  PubMed  Google Scholar 

  29. Hebert C, Delaney J, Hemmelgarn B, Lévesque LE, Suissa S. Benzodiazepines and elderly drivers: a comparison of pharmacoepidemiological study designs. Pharmacoepidemiol Drug Saf. 2007;16(8):845–9.

    Article  PubMed  Google Scholar 

  30. Suissa S. The case-time-control design. Epidemiology. 1995;6(3):248–53.

    Article  CAS  PubMed  Google Scholar 

  31. Wang SV, Coull BA, Schwartz J, Mittleman MA, Wellenius GA. Potential for bias in case-crossover studies with shared exposures analyzed using SAS. Am J Epidemiol. 2011;174(1):118–24.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Hernandez-Diaz S, Hernan MA, Meyer K, Werler MM, Mitchell AA. Case-crossover and case-time-control designs in birth defects epidemiology. Am J Epidemiol. 2003;158(4):385–91.

    Article  PubMed  Google Scholar 

  33. Schneider MF, Gange SJ, Margolick JB, Detels R, Chmiel JS, Rinaldo C, et al. Application of case-crossover and case–time–control study designs in analyses of time-varying predictors of T-cell homeostasis failure. Ann Epidemiol. 2005;15(2):137–44.

    Article  PubMed  Google Scholar 

  34. Farrington CP. Relative incidence estimation from case series for vaccine safety evaluation. Biometrics. 1995;51(1):228–35.

    Article  CAS  PubMed  Google Scholar 

  35. Whitaker HJ, Hocine MN, Farrington CP. The methodology of self-controlled case series studies. Stat Methods Med Res. 2009;18(1):7–26.

    Article  PubMed  Google Scholar 

  36. Whitaker HJ, Farrington CP, Spiessens B, Musonda P. Tutorial in biostatistics: the self-controlled case series method. Stat Med. 2006;25(10):1768–97.

    Article  PubMed  Google Scholar 

  37. The Self Controlled Case Series Method (internet). The Open University, UK (updated 2010; cited 2015 Dec 4). http://statistics.open.ac.uk/sccs/index.htm.

  38. Yu M, Xie D, Wang X, Weiner MG, Tannen RL. Prior event rate ratio adjustment: numerical studies of a statistical method to address unrecognized confounding in observational studies. Pharmacoepidemiol Drug Saf. 2012;21(Suppl. 2):60–8.

    Article  PubMed  Google Scholar 

  39. Tannen RL, Weiner MG, Xie D. Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings. BMJ. 2009;338(7691):395–9.

    Google Scholar 

  40. Tannen R, Xie D, Wang X, Yu M, Weiner MG. A new “Comparative Effectiveness” assessment strategy using the THIN database: comparison of the cardiac complications of pioglitazone and rosiglitazone. Pharmacoepidemiol Drug Saf. 2013;22(1):86–97.

    Article  CAS  PubMed  Google Scholar 

  41. Tannen RL, Weiner MG, Xie D. Replicated studies of two randomized trials of angiotensin-converting enzyme inhibitors: further empiric validation of the ‘prior event rate ratio’ to adjust for unmeasured confounding by indication. Pharmacoepidemiol Drug Saf. 2008;17(7):671–85.

    Article  CAS  PubMed  Google Scholar 

  42. Weiner MG, Xie D, Tannen RL. Replication of the Scandinavian Simvastatin Survival Study using a primary care medical record database prompted exploration of a new method to address unmeasured confounding. Pharmacoepidemiol Drug Saf. 2008;17(7):661–70.

    Article  PubMed  Google Scholar 

  43. Uddin MJ, Groenwold RHH, Van Staa TP, De Boer A, Belitser SV, Hoes AW, et al. Performance of prior event rate ratio adjustment method in pharmacoepidemiology: a simulation study. Pharmacoepidemiol Drug Saf. 2015;24(5):468–77.

    Article  PubMed  Google Scholar 

  44. Lipsitch M, Tchetgen E Tchetgen, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–8.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Schuemie MJ, Ryan PB, Dumouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat Med. 2014;33(2):209–18.

    Article  PubMed  Google Scholar 

  46. Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35(2):337–44.

    Article  PubMed  Google Scholar 

  47. Flanders WD, Klein M, Darrow LA, Strickland MJ, Sarnat SE, Sarnat JA, et al. A method for detection of residual confounding in time-series and other observational studies. Epidemiology. 2011;22(1):59–67.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Tchetgen E Tchetgen. The control outcome calibration approach for causal inference with unobserved confounding. Am J Epidemiol. 2014;179(5):633–40.

    Article  Google Scholar 

  49. Groenwold RHH. Falsification end points for observational studies. JAMA. 2013;309(17):1769–70.

    Article  CAS  PubMed  Google Scholar 

  50. Rassen JA, Brookhart MA, Glynn RJ, Mittleman MA, Schneeweiss S. Instrumental variables I: instrumental variables exploit natural variation in nonexperimental data to estimate causal relationships. J Clin Epidemiol. 2009;62(12):1226–32.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Chen Y, Briesacher BA. Use of instrumental variable in prescription drug research with observational data: a systematic review. J Clin Epidemiol. 2011;64(6):687–700.

    Article  PubMed  Google Scholar 

  52. Davies NM, Smith GD, Windmeijer F, Martin RM. Issues in the reporting and conduct of instrumental variable studies: a systematic review. Epidemiology. 2013;24(3):363–9.

    Article  PubMed  Google Scholar 

  53. Martens EP, Pestman WR, De Boer A, Belitser SV, Klungel OH. Instrumental variables: application and limitations. Epidemiology. 2006;17(3):260–7.

    Article  PubMed  Google Scholar 

  54. Hernán MA, Robins JM. Instruments for causal inference: an epidemiologist’s dream? Epidemiology. 2006;17(4):360–72.

    Article  PubMed  Google Scholar 

  55. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000;29(4):722–9.

    Article  CAS  PubMed  Google Scholar 

  56. Schneeweiss S, Setoguchi S, Brookhart A, Dormuth C, Wang PS. Risk of death associated with the use of conventional versus atypical antipsychotic drugs among elderly patients. Can Med Assoc J. 2007;176(5):627.

    Article  Google Scholar 

  57. Uddin MJ, Groenwold RHH, de Boer A, Belitser S, Roes KCB, Hoes AW, et al. Performance of instrumental variable methods in cohort and nested case–control studies: a simulation study. Pharmacoepidemiol Drug Saf. 2014;23:165–77.

    Article  PubMed  Google Scholar 

  58. Brookhart MA, Rassen JA, Schneeweiss S. Instrumental variable methods in comparative safety and effectiveness research. Pharmacoepidemiol Drug Saf. 2010;19(6):537–44.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Instrumental Variables (internet). Econometrics Academy (updated 2015; cited 2015 Dec 4). https://sites.google.com/site/econometricsacademy/econometrics-models/instrumental-variables.

  60. Lin NX, Logan S, Henley WE. Bias and sensitivity analysis when estimating treatment effects from the cox model with omitted covariates. Biometrics. 2013;69(4):850–60.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Groenwold RH, Nelson DB, Nichol KL, Hoes AW, Hak E. Sensitivity analyses to estimate the potential impact of unmeasured confounding in causal research. Int J Epidemiol. 2010;39(1):107–17.

    Article  PubMed  Google Scholar 

  62. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25(6):1107–16.

    Article  CAS  PubMed  Google Scholar 

  63. Lee WC. Detecting and correcting the bias of unmeasured factors using perturbation analysis: a data-mining approach. BMC Med Res Methodol. 2014;14(1):18.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Vanderweele TJ, Arah OA. Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology. 2011;22(1):42–52.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Wen SW, Kramer MS. Uses of ecologic studies in the assessment of intended treatment effects. J Clin Epidemiol. 1999;52(1):7–12.

    Article  CAS  PubMed  Google Scholar 

  66. Morgenstern H. Ecologic studies in epidemiology: concepts, principles, and methods. Annu Rev Public Health. 1995;16:61–81.

    Article  CAS  PubMed  Google Scholar 

  67. Groome PA, Mackillop WJ, Naylor CD. Uses of ecologic studies in the assessment of intended treatment effects multiple letters. J Clin Epidemiol. 1999;52(9):903–4.

    Article  CAS  PubMed  Google Scholar 

  68. Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin Pharmacol Ther. 2007;82(2):143–56.

    Article  CAS  PubMed  Google Scholar 

  69. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.

    Article  Google Scholar 

  70. Ali MS, Groenwold RH, Belitser SV, Pestman WR, Hoes AW, Roes KC, et al. Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: a systematic review. J Clin Epidemiol. 2015;68(2):112–21.

    Article  PubMed  Google Scholar 

  71. Stürmer T, Schneeweiss S, Avorn J, Glynn RJ. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am J Epidemiol. 2005;162(3):279–89.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Sturmer T, Schneeweiss S, Rothman KJ, Avorn J, Glynn RJ. Performance of propensity score calibration—a simulation study. Am J Epidemiol. 2007;165(10):1110–8.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Lunt M, Glynn RJ, Rothman KJ, Avorn J, Stürmer T. Propensity score calibration in the absence of surrogacy. Am J Epidemiol. 2012;175(12):1294–302.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Collet JP, Schaubel D, Hanley J, Sharpe C, Boivin JF. Controlling confounding when studying large pharmacoepidemiologic databases: a case study of the two-stage sampling design. Epidemiology. 1998;9(3):309–15.

    Article  CAS  PubMed  Google Scholar 

  75. Hanley JA, Dendukuri N. Efficient sampling approaches to address confounding in database studies. Stat Methods Med Res. 2009;18(1):81–105.

    Article  PubMed  Google Scholar 

  76. Stürmer T, Schneeweiss S, Rothman KJ, Avorn J, Glynn RJ. Comparison of performance of propensity score calibration (PSC) and multiple imputation (MI) to control for unmeasured confounding using an internal validation study. Pharmacoepidemiol Drug Saf. 2006;15:S39–40 (abstract).

    Article  Google Scholar 

  77. Faries D, Peng X, Pawaskar M, Price K, Stamey JD, Seaman JW. Evaluating the impact of unmeasured confounding with internal validation data: an example cost evaluation in type 2 diabetes. Value Health. 2013;16(2):259–66.

    Article  PubMed  Google Scholar 

  78. Yao X, Lix L. A flexible method to apply multiple imputation using SAS/IML® studio. SAS Global Forum 2013; Paper 283-2013.

  79. Soley-Bori M. Dealing with missing data: key assumptions and methods for applied analysis 2013.

  80. Horton NJ, Lipsitz SR. Multiple imputation in practice: comparison of software packages for regression models with missing variables. Am Stat. 2001;55(3):244–54.

    Article  Google Scholar 

  81. Miettinen OS. The need for randomization in the study of intended effects. Stat Med. 1983;2(2):267–71.

    Article  CAS  PubMed  Google Scholar 

  82. Joseph K, Mehrabadi A, Lisonkova S. Confounding by indication and related concepts. Curr Epidemiol Rep. 2014;1(1):1–8.

    Article  Google Scholar 

Download references

Funding

None.

Author information

Authors and Affiliations

  1. Department of Statistics (Biostatistics and Epidemiology), Shahjalal University of Science and Technology, Sylhet, 3114, Bangladesh

    Md. Jamal Uddin

  2. Division of Pharmacoepidemiology and Clinical Pharmacology, University of Utrecht, Utrecht, The Netherlands

    Md. Jamal Uddin, Anthonius de Boer & Olaf H. Klungel

  3. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands

    Rolf H. H. Groenwold, Mohammed Sanni Ali & Kit C. B. Roes

  4. Department of Biostatistics, Robert Stempel College of Public Health and Social Work, Florida International University, Miami, FL, USA

    Muhammad A. B. Chowdhury

Authors
  1. Md. Jamal Uddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rolf H. H. Groenwold
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Sanni Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthonius de Boer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kit C. B. Roes
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Muhammad A. B. Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Olaf H. Klungel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Jamal Uddin.

Ethics declarations

Conflicts of interest

There are no financial, personal, political, academic or other relations that could lead to a conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Uddin, M.J., Groenwold, R.H.H., Ali, M.S. et al. Methods to control for unmeasured confounding in pharmacoepidemiology: an overview. Int J Clin Pharm 38, 714–723 (2016). https://doi.org/10.1007/s11096-016-0299-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11096-016-0299-0

Keywords

Search

Navigation