Prevention Science

, Volume 14, Issue 6, pp 570–580 | Cite as

An Introduction to Sensitivity Analysis for Unobserved Confounding in Nonexperimental Prevention Research

  • Weiwei LiuEmail author
  • S. Janet Kuramoto
  • Elizabeth A. Stuart


Despite the fact that randomization is the gold standard for estimating causal relationships, many questions in prevention science are often left to be answered through nonexperimental studies because randomization is either infeasible or unethical. While methods such as propensity score matching can adjust for observed confounding, unobserved confounding is the Achilles heel of most nonexperimental studies. This paper describes and illustrates seven sensitivity analysis techniques that assess the sensitivity of study results to an unobserved confounder. These methods were categorized into two groups to reflect differences in their conceptualization of sensitivity analysis, as well as their targets of interest. As a motivating example, we examine the sensitivity of the association between maternal suicide and offspring’s risk for suicide attempt hospitalization. While inferences differed slightly depending on the type of sensitivity analysis conducted, overall, the association between maternal suicide and offspring’s hospitalization for suicide attempt was found to be relatively robust to an unobserved confounder. The ease of implementation and the insight these analyses provide underscores sensitivity analysis techniques as an important tool for nonexperimental studies. The implementation of sensitivity analysis can help increase confidence in results from nonexperimental studies and better inform prevention researchers and policy makers regarding potential intervention targets.


Sensitivity analysis Causal inference Unobserved confounding Suicide prevention 



The authors wish to acknowledge the NARSAD Young Investigator Award to Dr. Holly C. Wilcox, and Dr. Holly C. Wilcox for allowing us to use the motivating example. We also thank the National Institute of Drug Abuse for the training support for S. Janet Kuramoto (1F31DA0263182), the National Institute of Mental Health (NIMH) Prevention Research T32 Training Grant for the training support for Weiwei Liu (T32 MH18834), and NIMH for the support of Elizabeth Stuart’s time (K25 MH083846). This work was performed while Weiwei Liu was a postdoctoral fellow and S. Janet Kuramoto was a student at Johns Hopkins Bloomberg School of Public Health.

Supplementary material

11121_2012_339_MOESM1_ESM.docx (73 kb)
ESM 1 (DOCX 72 kb)


  1. Arah, O. A., Chiba, Y., & Greeland, S. (2008). Bias formulas for external adjustment and sensitivity analysis of unmeasured confounders. Annals of Epidemiology, 18, 637–646. doi: 10.1016/j.annepidem.2008.04.003.CrossRefPubMedGoogle Scholar
  2. Brent, D. A., & Mann, J. J. (2005). Family genetic studies, suicide, and suicidal behavior. American Journal of Medical Genetics, 133, 13–24. doi: 10.1002/ajmg.c.30042.CrossRefGoogle Scholar
  3. Cornfield, J., Haenszel, W., Hammon, E., Lilienfeld, A., Shimkin, M., & Wynder, E. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute, 22, 173–203. Available at Scholar
  4. DiPrete, T. A., & Gangl, M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology, 34, 271–310. doi: 10.1111/j.0081-1750.2004.00154.x.CrossRefGoogle Scholar
  5. Flay, B., Biglan, A., Boruch, R., Castro, F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175. doi: 10.1007/s11121-005-5553-y.CrossRefPubMedGoogle Scholar
  6. Gangl, M. (2004). Rbounds: Stata module to perform Rosenbaum sensitivity analysis for average treatment effects on the treated. Available at
  7. Gastwirth, J., Krieger, A., & Rosenbaum, P. (1998). Dual and simultaneous sensitivity analysis for matched pairs. Biometrika, 85, 907–920. doi: 10.1093/biomet/85.4.907.CrossRefGoogle Scholar
  8. Greenland, S. (1996). Basic methods for sensitivity analysis of biases. International Journal of Epidemiology, 25, 1107–1116. doi: 10.1093/ije/25.6.1107.CrossRefPubMedGoogle Scholar
  9. Harding, D. (2003). Counterfactual models of neighborhood effects: The effect of neighborhood poverty on dropping out and teenage pregnancy. The American Journal of Sociology, 109, 676–719. doi: 10.1086/379217.CrossRefGoogle Scholar
  10. Harding, D. J. (2009). Collateral consequences of violence in disadvantaged neighborhoods. Social Forces, 88, 757–784. doi: 10.1353/sof.0.0281.CrossRefGoogle Scholar
  11. Haviland, A., Nagin, D., & Rosenbaum, P. (2007). Combining propensity score matching and group-based trajectory analysis in an observational study. Psychological Methods, 12, 247–267. doi: 10.1037/1082-989X.12.3.247.CrossRefPubMedGoogle Scholar
  12. Jo, B., & Stuart, E. A. (2009). On the use of propensity scores in principal causal effect estimation. Statistics in Medicine, 28, 2857–2875. PMCID PMC 2757143.PubMedCentralCrossRefPubMedGoogle Scholar
  13. Keele, L. (2010). Rbounds: An R package for sensitivity analysis with matched data. R package. Available at
  14. Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongo, N. S., Wang, W., Toyinbo, P., et al. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95, S5–S28. doi: 10.1016/j.drugalcdep.2008.01.004.PubMedCentralCrossRefPubMedGoogle Scholar
  15. Kitahata, M. M., Gange, S. J., Abraham, A. G., Merriman, B., Saag, M. S., Justice, A. C., et al. (2009). Effect of early versus deferred antiretroviral therapy for HIV on survival. The New England Journal of Medicine, 360, 1815–1826. Available at Scholar
  16. Kuramoto, S. J., Stuart, E. A., Runeson, B., Lichtenstein, P., Langstrom, N., & Wilcox, H. C. (2010). Maternal or paternal suicide and offspring’s psychiatric and suicide-attempt hospitalization risk. Pediatrics, 126, e1026–e1032. doi: 10.1542/peds.2010-0974.PubMedCentralCrossRefPubMedGoogle Scholar
  17. Lieb, R., Bronisch, T., Hofler, M., Schreier, A., & Wittchen, H.-U. (2005). Maternal suicidality and risk of suicidality in offspring: Findings from a community study. The American Journal of Psychiatry, 162, 1665–1671. doi: 10.1176/appi.ajp.162.9.1665.CrossRefPubMedGoogle Scholar
  18. Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998). Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics, 54, 948–963. Available at Scholar
  19. Liu, W. (2012). The adult offending and school dropout nexus: A life course analysis. El Paso: LFB Scholarly Publishing LLC (in press).Google Scholar
  20. Love, T. (2008). Spreadsheet-based sensitivity analysis calculations for matched samples. Center for Health Care Research & Policy, Case Western Reserve University, Cleveland. Available at
  21. Luiz, R., & Cabral, M. (2010). Sensitivity analysis for an unmeasured confounder: A review of two independent methods. Revista Brasileira de Epidemiologia, 13, 188–198. Available at Scholar
  22. Manski, C. F., Sandefur, G. D., McLanahan, S., & Power, D. (1992). Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association, 87, 25–37. Available at Scholar
  23. McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9, 403–425. doi: 10.1037/1082-989X.9.4.403.CrossRefPubMedGoogle Scholar
  24. McCandless, L. C., Gustafson, P., & Levy, A. (2007). Bayesian sensitivity analysis for unmeasured confounding in observational studies. Statistics in Medicine, 26, 2331–2347. doi: 10.1002/sim.CrossRefPubMedGoogle Scholar
  25. McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153–157. doi: 10.1007/BF02295996.CrossRefPubMedGoogle Scholar
  26. Niederkrotenthaler, T., Floderus, B., Alexanderson, K., Rasmussen, F., & Mittendorfer-Rutz, E. (2012). Exposure to parental mortality and markers of morbidity, and the risks of attempted and completed suicide in offspring: An analysis of sensitive life periods. Journal of Epidemiology and Community Health, 66, 233–239. doi: 10.1136/jech.2010.109595.CrossRefPubMedGoogle Scholar
  27. Ridgeway, G. (2006). Assessing the effect of race bias in post-traffic stop outcomes using propensity scores. Journal of Quantitative Criminology, 22, 1–29. doi: 10.1007/s10940-005-9000-9.CrossRefGoogle Scholar
  28. Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer.CrossRefGoogle Scholar
  29. Rosenbaum, P. R. (2010). Design of observational studies. New York: Springer.CrossRefGoogle Scholar
  30. Rothman, K. J., Greenland, S., & Lash, T. L. (2008). Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins.Google Scholar
  31. Schneeweiss, S. (2006). Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiology and Drug Safety, 15, 291–303. doi: 10.1002/pds.1200.CrossRefPubMedGoogle Scholar
  32. Steenland, K., & Greenland, S. (2004). Monte Carlo sensitivity analysis and Bayesian analysis of smoking as an unmeasured confounder in a study of silica and lung cancer. American Journal of Epidemiology, 160, 384–392. doi: 10.1093/aje/kwh211.CrossRefPubMedGoogle Scholar
  33. Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21. doi: 10.1214/09-STS313.PubMedCentralCrossRefPubMedGoogle Scholar
  34. Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in nonexperimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Developmental Psychology, 44, 395–406.CrossRefPubMedGoogle Scholar
  35. VanderWeele, T. J., & Arah, O. A. (2011). Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology, 22, 42–52. doi: 10.1097/EDE.0b013e3181f74493.PubMedCentralCrossRefPubMedGoogle Scholar
  36. Wilcox, H. C., Kuramoto, S. J., Lichtenstein, P., Långström, N., Brent, D. A., & Runeson, B. (2010). Psychiatric morbidity, violent crime and suicide among children and adolescents exposed to parental death. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 514–523. doi: 10.1016/j.jaac.2010.01.020.PubMedGoogle Scholar

Copyright information

© Society for Prevention Research 2013

Authors and Affiliations

  • Weiwei Liu
    • 1
    Email author
  • S. Janet Kuramoto
    • 2
  • Elizabeth A. Stuart
    • 3
  1. 1.NORC at the University of ChicagoBethesdaUSA
  2. 2.American Psychiatric Institute for Research and EducationArlingtonUSA
  3. 3.Department of Mental Health and Department of Biostatistics, Bloomberg School of Public HealthJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations