Abstract
Treatment effect estimation, which refers to the estimation of causal effects and aims to measure the strength of the causal relationship, is of great importance in many fields but is a challenging problem in practice. As present, data-driven causal effect estimation faces two main challenges, i.e., selection bias and the missing of counterfactual. To address these two issues, most of the existing approaches tend to reduce the selection bias by learning a balanced representation, and then to estimate the counterfactual through the representation. However, they heavily rely on the finely hand-crafted metric functions when learning balanced representations, which generally doesn’t work well for the situations where the original distribution is complicated. In this paper, we propose a CETransformer model for casual effect estimation via transformer based representation learning. To learn the representation of covariates (features) robustly, a self-supervised transformer is proposed, by which the correlation between covariates can be well exploited through self-attention mechanism. In addition, an adversarial network is adopted to balance the distribution of the treated and control groups in the representation space. Experimental results on three real-world datasets demonstrate the advantages of the proposed CETransformer, compared with the state-of-the-art treatment effect estimation methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alaa, A., Schaar, M.: Limits of estimating heterogeneous treatment effects: guidelines for practical algorithm design. In: International Conference on Machine Learning, pp. 129–138. PMLR (2018)
Alaa, A.M., van der Schaar, M.: Bayesian inference of individualized treatment effects using multi-task gaussian processes. arXiv preprint arXiv:1704.02801 (2017)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN (2017)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Chipman, H.A., George, E.I., McCulloch, R.E., et al.: BART: Bayesian additive regression trees. Ann. Appl. Stat. 4(1), 266–298 (2010)
Crump, R.K., Hotz, V.J., Imbens, G.W., Mitnik, O.A.: Nonparametric tests for treatment effect heterogeneity. Rev. Econ. Stat. 90(3), 389–405 (2008)
Domingos, P.: Every model learned by gradient descent is approximately a kernel machine. arXiv preprint arXiv:2012.00152 (2020)
D’Amour, A., Ding, P., Feller, A., Lei, L., Sekhon, J.: Overlap in observational studies with high-dimensional covariates. J. Econometrics 221(2), 644–654 (2021)
Gangl, M.: Causal inference in sociological research. Ann. Rev. Sociol. 36, 21–47 (2010)
Glass, T.A., Goodman, S.N., Hernán, M.A., Samet, J.M.: Causal inference in public health. Annu. Rev. Public Health 34, 61–75 (2013)
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of Wasserstein GANs. arXiv preprint arXiv:1704.00028 (2017)
Hill, J.L.: Bayesian nonparametric modeling for causal inference. J. Comput. Graph. Stat. 20(1), 217–240 (2011)
Johansson, F., Shalit, U., Sontag, D.: Learning representations for counterfactual inference. In: International Conference on Machine Learning, pp. 3020–3029. PMLR (2016)
Kuang, K., Cui, P., Li, B., Jiang, M., Yang, S.: Estimating treatment effect in the wild via differentiated confounder balancing. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 265–274 (2017)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)
Pearl, J., et al.: Causal inference in statistics: an overview. Stat. Surv. 3, 96–146 (2009)
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66(5), 688 (1974)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. arXiv preprint arXiv:1606.03498 (2016)
Schwab, P., Linhardt, L., Bauer, S., Buhmann, J.M., Karlen, W.: Learning counterfactual representations for estimating individual dose-response curves. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 5612–5619 (2020)
Shalit, U., Johansson, F.D., Sontag, D.: Estimating individual treatment effect: generalization bounds and algorithms. In: International Conference on Machine Learning, pp. 3076–3085. PMLR (2017)
Splawa-Neyman, J., Dabrowska, D.M., Speed, T.: On the application of probability theory to agricultural experiments. essay on principles. Section 9. Stat. Sci. 465–472 (1990)
Vaswani, A., et al.: Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)
Wager, S., Athey, S.: Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113(523), 1228–1242 (2018)
Wang, P., Sun, W., Yin, D., Yang, J., Chang, Y.: Robust tree-based causal inference for complex ad effectiveness analysis. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 67–76 (2015)
Yao, L., Li, S., Li, Y., Huai, M., Gao, J., Zhang, A.: Representation learning for treatment effect estimation from observational data. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Yao, L., Li, S., Li, Y., Huai, M., Gao, J., Zhang, A.: Ace: Adaptively similarity-preserved representation learning for individual treatment effect estimation. In: 2019 IEEE International Conference on Data Mining (ICDM), pp. 1432–1437. IEEE (2019)
Yin, X., Hong, L.: The identification and estimation of direct and indirect effects in a/b tests through causal mediation analysis. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2989–2999 (2019)
Zhang, K., Gong, M., Schölkopf, B.: Multi-source domain adaptation: a causal view. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)
Zhang, Y., Bellot, A., Schaar, M.: Learning overlapping representations for the estimation of individualized treatment effects. In: International Conference on Artificial Intelligence and Statistics, pp. 1005–1014. PMLR (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, Z., Zheng, S., Liu, Z., Yan, K., Zhu, Z. (2021). CETransformer: Casual Effect Estimation via Transformer Based Representation Learning. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13022. Springer, Cham. https://doi.org/10.1007/978-3-030-88013-2_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-88013-2_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88012-5
Online ISBN: 978-3-030-88013-2
eBook Packages: Computer ScienceComputer Science (R0)