Abstract
In observational studies of treatments or interventions, propensity score (PS) adjustment is often useful for controlling bias in estimation of treatment effects. Regression on PS is used most often and can be highly efficient, but it can lead to biased results when model assumptions are violated. The validity of stratification on PS depends on fewer model assumptions, but this approach is less efficient than regression adjustment when the regression assumptions hold. To investigate these issues, we compare stratification and regression adjustments in a Monte Carlo simulation study. We consider two stratification approaches: equal frequency strata and an approach that attempts to choose strata that minimize the mean squared error (MSE) of the treatment effect estimate. The regression approach that we consider is a generalized additive model (GAM) that estimates treatment effect controlling for a potentially nonlinear association between PS and outcome. We find that under a wide range of plausible data generating distributions the GAM approach outperforms stratification in treatment effect estimation with respect to bias, variance, and thereby MSE. We illustrate each approach in an analysis of insurance plan choice and its relation to satisfaction with asthma care.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Austin, P.C., Grootendorst, P., Anderson, G.M.: A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26, 734–753 (2007)
Billewicz, W.: The efficiency of matched samples: an emperical investigation. Biometrics 21, 623–643 (1965)
Brookhart, M., Schneeweiss, S., Rothman, K., Glynn, R., Avorn, J., Sturmer, T.: Variable selection for propensity score models. Am. J. Epidemiol. 163, 1149 (2006)
Byrd, R., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)
Cochran, W.G.: The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24, 295–313 (1968)
D’Agostino, R. Jr.: Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat. Med. 17, 2265–2281 (1998)
Dehejia, R.H., Wahba, S.: Propensity score-matching methods for nonexperimental causal studies. Rev. Econ. Stat. 84, 151–161 (2002)
Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall, London (1990)
Hellevik, O.: Linear versus logistic regression when the dependent variable is a dichotomy. Qual. Quant. 43, 59–74 (2008)
Hill, J.: Discussion of research using propensity-score matching: comments on ‘a critical appraisal of propensity-score matching in the medical literature between 1996 and 2003’ by Peter Austin. Stat. Med. 27, 2055–2061 (2008)
Huang, I., Frangakis, C., Dominici, F., Diette, G., Wu, A.: Application of a propensity score approach for risk adjustment in profiling multiple physician groups on asthma care. Health Serv. Res. 40, 253–278 (2005)
Hullsiek, K.H., Louis, T.A.: Propensity score modeling strategies for the causal analysis of observational data. Biostatistics 2, 179–193 (2002)
Little, R., An, H.: Robust likelihood-based analysis of multivariate data with missing values. Stat. Sin. 14, 949–968 (2004)
Masland, M., Wu, A., Diette, G., Dominici, F., Skinner, E.: The 1998 Asthma Outcomes Survey. Pacific Business Group on Health, San Francisco, CA (2000)
R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0 (2010)
Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)
Rosenbaum, P.R., Rubin, D.B.: Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)
Rosenbaum, P.R., Rubin, D.B.: Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am. Stat. 39, 33–38 (1985)
Rubin, D.B.: Practical implications of modes of statistical inference for causal effects and the critical role of the assignment mechanism. Biometrics 47, 1213–1234 (1991)
Rubin, D.: Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv. Outcomes Res. Methodol. 2, 169–188 (2001)
Rubin, D.: The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat. Med. 26, 20 (2007)
Shah, B., Laupacis, A., Hux, J., Austin, P.: Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J. Clin. Epidemiol. 58, 550–559 (2005)
Sommer, A., Zeger, S.L.: On estimating efficacy from clinical trials. Stat. Med. 10, 45–52 (1991)
Tu, W., Zhou, X.: A bootstrap confidence interval procedure for the treatment effect using propensity score subclassification. Health Serv. Outcomes Res. Methodol. 3, 135–147 (2002)
Weitzen, S., Lapane, K., Toledano, A., Hume, A., Mor, V.: Principles for modeling propensity scores in medical research: a systematic literature review. Pharmacoepidemiol. Drug Saf. 13, 841–853 (2004)
Woo, M., Reiter, J., Karr, A.: Estimation of propensity scores using generalized additive models. Stat. Med. 27, 3805–3816 (2008)
Wood, S.: Stable and efficient multiple smoothing parameter estimation for generalized additive models. J. Am. Stat. Assoc. 99, 673–687 (2004)
Wood, S.N.: Thin-plate regression splines. J. R. Stat. Soc. B 65, 95–114 (2003)
Acknowledgments
This research was supported by Grant 5T32ES012871 from the U.S. National Institute of Environmental Health Sciences and Grant R01 DK061662 from the U.S. National Institute of Diabetes, Digestive and Kidney Diseases. The authors wish to thank I-Chang Huang and Constantine Frangakis for supplying the data analyzed in this paper.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Myers, J.A., Louis, T.A. Comparing treatments via the propensity score: stratification or modeling?. Health Serv Outcomes Res Method 12, 29–43 (2012). https://doi.org/10.1007/s10742-012-0080-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-012-0080-3