Abstract
Confounding present in observational data impede community psychologists’ ability to draw causal inferences. This paper describes propensity score methods as a conceptually straightforward approach to drawing causal inferences from observational data. A step-by-step demonstration of three propensity score methods—weighting, matching, and subclassification—is presented in the context of an empirical examination of the causal effect of preschool experiences (Head Start vs. parental care) on reading development in kindergarten. Although the unadjusted population estimate indicated that children with parental care had substantially higher reading scores than children who attended Head Start, all propensity score adjustments reduce the size of this overall causal effect by more than half. The causal effect was also defined and estimated among children who attended Head Start. Results provide no evidence for improved reading if those children had instead received parental care. We carefully define different causal effects and discuss their respective policy implications, summarize advantages and limitations of each propensity score method, and provide SAS and R syntax so that community psychologists may conduct causal inference in their own research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allison, P. D. (2002). Missing data. Thousand Oaks: Sage.
Austin, P. C. (2011a). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46, 399–424.
Austin, P. C. (2011b). A tutorial and case study in propensity score analysis: An application to estimating the effect of in-hospital smoking cessation counseling on mortality. Multivariate Behavioral Research, 46, 119–151.
Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Sturmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 1149–1156.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: LEA.
Cook, T. D. (2003). Why have educational evaluators chosen not to do randomized experiments? Annals of the American Academy of Political and Social Science, 589, 114–149.
D’Agostino, R. B. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265–2281.
Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234–249.
Hirano, K., & Imbens, G. W. (2001). Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services and Outcomes Research Methodology, 2, 259–278.
Imai, K., King, G., & Stuart, E. A. (2008). Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society, 171(2), 481–502.
Imbens, G. W. (2003). Sensitivity to exogeneity assumptions in program evaluation. The American Economic Review, 93(2), 126–132.
King, G., & Zeng, L. (2006). The dangers of extreme counterfactuals. Political Analysis, 14(2), 131–159.
Lee, B., Lessler, J., & Stuart, E. A. (2009). Improving propensity score weighting using machine learning. Statistics in Medicine, 29(3), 337–346.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data. New York: Wiley.
Luellen, J. K., Shadish, W. R., & Clark, M. H. (2005). Propensity scores: An introduction and experiemental test. Evaluation Review, 29(6), 530–558.
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9, 403–425.
McCall, R. B., & Green, B. (2004). Beyond the methodological gold standards of behavioral research: Considerations for practice and policy. Social Policy Report, 18(2), 3–19.
Mercer, S. L., DeVinney, B. J., Fine, L. J., Green, L. W., & Dougherty, D. (2007). Study designs for effectiveness and translation research: Identifying trade-offs. American Journal of Preventative Medicine, 33(2), 139–154.
Pearl, J. (2010). On a class of bias-amplifying covariates that endanger effect estimates. UCLA Cognitive Systems Laboratory, Technical Report (R-356). In P. Grunwald & P. Spirtes (Eds.), Proceedings of the twenty-sixth conference on uncertainty in artificial intelligence (pp. 417–424). Corvallis, OR.
Potter, F. J. (1993). The effect of weight trimming on nonlinear survey estimates. In Proceedings of the section on survey research methods of American Statistical Association. American Statistical Association, San Francisco, CA.
Rhoades, B. L., Warren, H. K., Domitrovich, C. E., & Greenberg, M. T. (2010). Examining the link between preschool social-emotional competence and first grade academic achievement: The role of attention skills. Early Childhood Research Quarterly. doi:10.1016/j.ecresq.2010.07.003.
Rosenbaum, P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society Series A (General), 147, 656–666.
Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524.
Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. American Statistician, 39, 33–38.
Sanson-Fisher, R. W., Bonevski, B., Green, L. W., & D’Este, C. (2007). Limitations of the randomized controlled trial in evaluating population-based health interventions. American Journal of Preventative Medicine, 33(2), 155–161.
Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: Chapman & Hall.
Schafer, J. L., & Kang, J. D. Y. (2008). Average causal effects from non-randomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin.
Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15, 250–267.
Stuart, E. A. (2010). Matching methods for causal inference: A review and look forward. Statistical Science, 25(1), 1–21.
Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in nonexperimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Developmental Psychology, 44(2), 395–406.
Stuart, E. A., Perry, D. F., Le, H.-N., & Ialongo, N. S. (2008). Estimating intervention effects of prevention programs: Accounting for noncompliance. Prevention Science, 9, 288–298.
US Department of Education, National Center for Education Statistics. (2009). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K) Kindergarten through Eighth Grade Full Sample Public-Use Data and Documentation (DVD). (NCES 2009-005). Washington, DC: Author.
VanderWeele, T. (2006). The use of propensity scores in psychiatric research. International Journal of Methods in Psychiatric Research, 15(2), 95–103.
West, S. G. (2009). Alternatives to randomized experiments. Current Directions in Psychological Science, 18(5), 299–304.
Acknowledgments
Preparation of this manuscript was supported by National Institute on Drug Abuse (NIDA) Center grant P50 DA10075 and NIDA grant R03 DA026543-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse (NIDA) or the National Institutes of Health (NIH). The authors thank Donna Coffman, Brittany Rhoades, and Bethany Bray for feedback on an early draft of this manuscript, and John Dziak for providing the SAS macro used for subclassification outcome analysis.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Lanza, S.T., Moore, J.E. & Butera, N.M. Drawing Causal Inferences Using Propensity Scores: A Practical Guide for Community Psychologists. Am J Community Psychol 52, 380–392 (2013). https://doi.org/10.1007/s10464-013-9604-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-013-9604-4