Abstract
The challenge of making causal inferences from observational data consists not only in mastering the statistical matching techniques but also in meeting the causal assumptions. In this chapter, we have explained what the causal estimands are and how they can be identified and then estimated via propensity score (PS)-matching, PS-stratification, or PS-weighting in single-level or multilevel settings. We have also highlighted that the selection of baseline covariates and their reliable measurement is much more important than the choice of a specific matching technique. Sensitivity analyses can be very helpful in assessing the effect of unobserved confounding.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park: Sage.
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424.
Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295–313.
Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya: The Indian Journal of Statistics, 35(4), 417–446.
Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40, 31–53.
Griffin, B. A., Ridgeway, G., Morral, A. R., Burgette, L. F., Martin, C., Almirall, D., Ramchand, R., Jaycox, L. H., & McCaffrey, D. F. (2014). Toolkit for Weighting and Analysis of Nonequivalent Groups (TWANG) Website. Santa Monica: RAND Corporation. Retrieved from http://www.rand.org/statistics/twang. Accessed 28 July 2017.
Hansen, B. B., & Klopfer, S. O. (2006). Optimal full matching and related designs via network flows. Journal of Computational and Graphical Statistics, 15, 609–627.
Ho, D., Imai, K., King, G., & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), 1–28.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960.
Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101(475), 901–910.
Imbens, G. W., & Rubin, D. B. (2015). Causal inference for statistics, social, and biomedical sciences: An introduction. New York: Cambridge University Press.
Kang, J. D., & Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22(4), 523–539.
Keele, L., & Pimentel, S. (2016). matchMulti: Optimal multilevel matching using a network algorithm. R package version 1.1.5. https://CRAN.R-project.org/package=matchMulti
Kelcey, B. (2011). Assessing the effects of teachers’ reading knowledge on students’ achievement using multilevel propensity score stratification. Educational Evaluation and Policy Analysis, 33(4), 458–482.
Keller, B., Kim, J.-S., & Steiner, P. M. (2015). Neural networks for propensity score estimation: Simulation results and recommendations. In L. A. van der Ark, D. M. Bolt, S.-M. Chow, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology research (pp. 279–291). New York: Springer.
Kim, J.-S., & Steiner, P. M. (2015). Multilevel propensity score methods for estimating causal effects: A latent class modeling strategy. In L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & S.-M. Chow (Eds.), Quantitative psychology research: Proceedings of the 79th annual meeting of the psychometric society (pp. 293–306). New York: Springer.
Kim, Y., & Steiner, P. M. (2017, March). The mechanics of omitted variable bias and the effect of measurement error. Invited talk at the 2017 ENAR Spring Meeting of the Eastern North American Region International Biometric Society. Washington, DC.
Kosanke, J., & Bergstralh, B. (2004). gmatch: Match 1 or more controls to cases using the greedy algorithm. Retrieved from http://www.mayo.edu/research/documents/gmatchsas/doc-10027248; vmatch: Match cases to controls using variable optimal matching. Retrieved from http://www.mayo.edu/research/documents/vmatchsas-05-14-14/doc-20094471. Accessed 26 July 2017.
Lara, B., Mizala, A., & Repetto, A. (2011). The effectiveness of private voucher education: Evidence from structural school switches. Educational Evaluation and Policy Analysis, 33(2), 119–137.
Lee, J., & Reeves, T. (2012). Revisiting the impact of NCLB high-stakes school accountability, capacity, and resources state NAEP 1990–2009 reading and math achievement gaps and trends. Educational Evaluation and Policy Analysis, 34(2), 209–231.
Leuven, E., & Sianesi, B. (2003). Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. Statistical Software Components S432001, Boston College, Department of Economics.
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9, 403–425.
Pearl, J. (2010). On a class of bias-amplifying variables that endanger effect estimates. In P. Grunwald & P. Spirtes (Eds.), Proceedings of the 26th conference on uncertainty in artificial intelligence (pp. 425–432). Corvallis: AUAI Press.
Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer-Verlag.
Rosenbaum, P. R. (2005). Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. The American Statistician, 59(2), 147–152.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.
Rubin, D. B. (1973). The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics, 29(1), 185–203.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701.
Rubin, D. B. (1980). Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75(371), 591–593.
Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine, 26, 20–36.
Schafer, J. L., & Kang, J. D. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313.
Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random and nonrandom assignments. Journal of the American Statistical Association, 103(484), 1334–1344.
StataCorp. (2015). Stata treatment-effects reference manual: Potential outcomes/counterfactual outcomes. College Station: Stata Press. Retrieved from http://www.stata.com/manuals14/te.pdf. Accessed 26 July 2017.
Steiner, P. M., & Cook, D. (2013). Matching and propensity scores. In T. Little (Ed.), The Oxford handbook of quantitative methods in psychology (Vol. 1, pp. 237–259). New York: Oxford University Press.
Steiner, P. M., & Kim, Y. (2016). The mechanics of omitted variable bias: Bias amplification and cancellation of offsetting biases. Journal of Causal Inference, 4(2). DOI: https://doi.org/10.1515/jci-2016-0009. Advance online publication.
Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15(3), 250.
Steiner, P. M., Cook, T. D., & Shadish, W. R. (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36(2), 213–236.
Steiner, P. M., Kim, J.-S., & Thoemmes, F. J. (2012). Matching strategies for observational multilevel data. In Joint statistical meetings proceedings (pp. 5020–5032). Alexandria: American Statistical Association.
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21.
Thoemmes, F. (2012). Propensity score matching in SPSS. Unpublished manuscript. Retrieved from https://sourceforge.net/projects/psmspss/. Accessed 26 July 2017.
Thoemmes, F., & West, S. G. (2011). The use of propensity scores for nonrandomized designs with clustered data. Multivariate Behavioral Research, 46(3), 514–543.
West, S. G., & Hughes, J. N. (2008). Effect of retention in first grade on children’s achievement trajectories over 4 years. Journal of Educational and Psychological Measurement, 100(4), 727–740.
Wyse, A. E., Keesler, V., & Schneider, B. (2008). Assessing the effects of small school size on mathematics achievement: A propensity score-matching approach. Teachers College Record, 110(9), 1879–1900.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 The Author(s)
About this chapter
Cite this chapter
Kim, Y., Lubanski, S.A., Steiner, P.M. (2018). Matching Strategies for Causal Inference with Observational Data in Education. In: Lochmiller, C. (eds) Complementary Research Methods for Educational Leadership and Policy Studies. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-93539-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-93539-3_9
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-319-93538-6
Online ISBN: 978-3-319-93539-3
eBook Packages: EducationEducation (R0)