Abstract
Propensity score-based methods have been widely developed to adjust for confounders in observational studies to estimate causal treatment effect for binary treatments. We generalize these causal inference methods to the multi-level treatment case. We review the generalized causal inference framework and several propensity score estimation methods. We conduct a comprehensive simulation study to evaluate the performance of multinomial logistic regression, generalized boosted models, random forest and data adaptive matching score for estimating propensity scores based on inverse probability of treatment weighting. From our findings, multinomial logistic regression is susceptible to yielding extreme weights while a mis-specified model is assumed, which results in poor performance of the inverse probability weighted estimator. On the other hand, machine-learned propensity scores tend to have less biased and more stable performance, and the data adaptive matching score tends to perform the best overall. The above-mentioned propensity score based methods are applied to the Taobao dataset to evaluate the causal effect of reputation on sales.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Brookhart, M.A., van der Laan, M.J.: A semiparametric model selection criterion with applications to the marginal structural model. Comput. Stat. Data Anal. 50(2), 475–498 (2006)
Crump, R.K., Hotz, V.J., Imbens, G.W., Mitnik, O.A.: Dealing with limited overlap in estimation of average treatment effects. Biometrika 96(1), 187–199 (2009)
Fan, Y., Ju, J., Xiao, M.: Reputation premium and reputation management: evidence from the largest e-commerce platform in china. Int. J. Ind. Organ. 46, 63–76 (2016)
Feng, P., Zhou, X.H., Zou, Q.M., Fan, M.Y., Li, X.S.: Generalized propensity score for estimating the average treatment effect of multiple treatments. Stat. Med. 31(7), 681–697 (2012)
Frölich, M.: Programme evaluation with multiple treatments. J. Econ. Surv. 18(2), 181–224 (2004)
Imai, K., Van Dyk, D.: Causal inference with general treatment regimes. J. Am. Stat. Assoc. 99(467), 854–866 (2004)
Imbens, G.W.: The role of the propensity score in estimating dose-response functions. Biometrika 87(3), 706–710 (2000)
Lechner, M.: Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In: Econometric Evaluation of Labour Market Policies pp. 43–58 (2001)
Lechner, M.: Program heterogeneity and propensity score matching: an application to the evaluation of active labor market policies. Rev. Econ. Stat. 84(2), 205–220 (2002)
Lee, B.K., Lessler, J., Stuart, E.A.: Improving propensity score weighting using machine learning. Stat. Med. 29(3), 337–346 (2010)
Lee, B.K., Lessler, J., Stuart, E.A.: Weight trimming and propensity score weighting. PloS one 6(3), e18–174 (2011)
Liaw, A., Wiener, M.: Classification and regression by randomforest. R News 2(3), 18–22 (2002). http://CRAN.R-project.org/doc/Rnews/
Lunceford, J.K., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23(19), 2937–2960 (2004)
Luo, W., Zhu, Y., Ghosh, D.: On estimating regression-based causal effects using sufficient dimension reduction. Biometrika 104(1), 51–65 (2017)
McCaffrey, D.F., Griffin, B.A., Almirall, D., Slaughter, M.E., Ramchand, R., Burgette, L.F.: A tutorial on propensity score estimation for multiple treatments using generalized boosted models. Stat. Med. 32(19), 3388–3414 (2013)
McCaffrey, D.F., Ridgeway, G., Morral, A.R.: Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychol. Methods 9(4), 403–425 (2004)
McFadden, D.: Conditional logit analysis of qualitative choice behavior. In: Zarembka, P. (ed.) Frontiers in Econometrics, pp. 105–142. Academic Press, New York (1973)
Ridgeway, G.: gbm: Generalized Boosted Regression Models (2012). R package version 1.6-3.2. http://CRAN.R-project.org/package=gbm. Accessed 8 Nov 2016
Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66(5), 688–701 (1974)
Spreeuwenberg, M.D., Bartak, A., Croon, M.A., Hagenaars, J.A., Busschbach, J.J., Andrea, H., Twisk, J., Stijnen, T.: The multiple propensity score as control for bias in the comparison of more than two treatment arms: an introduction from a case study in mental health. Med. Care 48(2), 166–174 (2010)
Tchernis, R., Horvitz-Lennon, M., Normand, S.L.T.: On the use of discrete choice models for causal inference. Stat. Med. 24(14), 2197–2212 (2005)
Tu, C., Koh, W.Y., Jiao, S.: Using generalized doubly robust estimator to estimate average treatment effects of multiple treatments in observational studies. J. Stat. Comput. Simul. 83(8), 1518–1526 (2013)
Uysal, S.D.: Doubly robust estimation of causal effects with multivalued treatments: an application to the returns to schooling. J. Appl. Econom. 30(5), 763–786 (2014)
van der Laan, M.J., Polley, E.C., Hubbard, A.E.: Super learner. Stat. Appl. Genet. Mol. Biol. 6(1), 1–21 (2007)
Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S, 4th edn. Springer, New York (2002)
Xie, Y., Zhu, Y., Cotton, C.A., Wu, P.: A model averaging approach for estimating propensity scores by optimizing balance. Stat. Methods Med. Res. (2017). https://doi.org/10.1177/0962280217715487
Zhu, Y., Ghosh, D., Mitra, N., Mukherjee, B.: A data-adaptive strategy for inverse weighted estimation of causal effects. Health Serv. Outcomes Res. Methodol. 14(3), 69–91 (2014)
Zhu, Y., Lin, L.L.: Propensity score modeling and evaluation. In: Statistical Causal Inferences and Their Applications in Public Health Research, pp. 111–124. Springer (2016)
Zhu, Y., Schonbach, M., Coffman, D.L., Williams, J.S.: Variable selection for propensity score estimation via balancing covariates. Epidemiology 26(2), e14–e15 (2015)
Funding
This study was funded by Social Sciences and Humanities Research Council Insight Development Grant (Grant number: 430-2016-00163) and by National Sciences and Engineering Research Council of Canada (Grant number: RGPIN-2017-04064)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declares that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Lin, L., Zhu, Y. & Chen, L. Causal inference for multi-level treatments with machine-learned propensity scores. Health Serv Outcomes Res Method 19, 106–126 (2019). https://doi.org/10.1007/s10742-018-0187-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-018-0187-2