Abstract
In observational studies, a key problem is to estimate the causal effect of a treatment on some outcome. Counterfactual inference tries to handle it by directly learning the treatment exposure surfaces. One of the biggest challenges in counterfactual inference is the existence of unobserved confounders, which are latent variables that affect both the treatment and outcome variables. Building on recent advances in latent variable modelling and efficient Bayesian inference techniques, deep latent variable models, such as variational auto-encoders (VAEs), have been used to ease the challenge by learning the latent confounders from the observations. However, for the sake of tractability, the posterior of latent variables used in existing methods is assumed to be Gaussian with diagonal covariance matrix. This specification is quite restrictive and even contradictory with the underlying truth, limiting the quality of the resulting generative models and the causal effect estimation. In this paper, we propose to take advantage of implicit generative models to detour this limitation by using black-box inference models. To make inference for the implicit generative model with intractable likelihood, we adopt recent implicit variational inference based on adversary training to obtain a close approximation to the true posterior. Experiments on simulated and real data show the proposed method matches the state-of-art.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Imbens, G.W., Rubin, D.B.: Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press, Cambridge (2015)
Hernán, M.A., Robins, J.M.: Causal Inference. Chapman & Hall/CRC, Boca Raton (2018, forthcoming)
Swaminathan, A., Joachims, T.: Batch learning from logged bandit feedback through counterfactual risk minimization. J. Mach. Learn. Res. 16, 1731–1755 (2015)
Swaminathan, A., Joachims, T.: Counterfactual risk minimization: learning from logged bandit feedback. In: ICML, pp. 814–823 (2015)
Bottou, L., et al.: Counterfactual reasoning and learning systems: the example of computational advertising. J. Mach. Learn. Res. 14, 3207–3260 (2013)
Swaminathan, A., Joachims, T.: The self-normalized estimator for counterfactual learning. In: NIPS, pp. 3231–3239 (2015)
Johansson, F.D., Shalit, U., Sontag, D.: Learning representations for counterfactual inference. In: ICML (2016)
Shalit, U., Johansson, F.D., Sontag, D.: Estimating individual treatment effect: generalization bounds and algorithms. In: ICML, vol. 1050, p. 28 (2017)
Louizos, C., Shalit, U., Mooij, J., Sontag, D., Zemel, R., Welling, M.: Causal effect inference with deep latent-variable models. In: NIPS (2017)
Pearl, J.: Causality: Models, Reasoning and Inference. Cambridge University Press, Cambridge (2000)
Ullman, J.B., Bentler, P.M.: Structural equation modeling. In: Handbook of Psychology, 2nd Edn (2012)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)
Mohamed, S., Lakshminarayanan, B.: Learning in implicit generative models. arXiv preprint arXiv:1610.03483 (2016)
Tran, D., Ranganath, R., Blei, D.: Hierarchical implicit models and likelihood-free variational inference. In: NIPS, pp. 5527–5537 (2017)
Tran, D., Blei, D.M.: Implicit causal models for genome-wide association studies. In: ICLR (2018)
Pearl, J.: An introduction to causal inference. Int. J. Biostat. 6(2) (2010). Article 7
Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Sig. Syst. 2, 303–314 (1989)
Izbicki, R., Lee, A.B., Pospisil, T.: ABC-CDE: towards approximate bayesian computation with complex high-dimensional data and limited simulations. arXiv:1805.05480 [stat.ME] (2018)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Jethava, V., Dubhashi, D.: GANs for LIFE: generative adversarial networks for likelihood free inference. arXiv preprint arXiv:1711.11139 (2017)
Tran, D., Hoffman, M.D., Saurous, R.A., Brevdo, E., Murphy, K., Blei, D.M.: Deep probabilistic programming. In: ICLR (2017)
Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. Wiley, Hoboken (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Mescheder, L., Nowozin, S., Geiger, A.: Adversarial variational bayes: unifying variational autoencoders and generative adversarial networks. arXiv preprint arXiv:1701.04722 (2017)
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
Hill, J.L.: Bayesian nonparametric modeling for causal inference. J. Comput. Graph. Stat. 20, 217–240 (2011)
Van der Laan, M.J., Rose, S.: Targeted Learning: Causal Inference for Observational and Experimental Data. Springer Science & Business Media, New York (2011). https://doi.org/10.1007/978-1-4419-9782-1
Chipman, H.A., George, E.I., McCulloch, R.E.: BART: Bayesian additive regression trees. Ann. Appl. Stat. 4, 266–298 (2010)
Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)
Athey, S., Tibshirani, J., Wager, S.: Generalized random forests. Ann. Stat. (2018, forthcoming)
Wager, S., Athey, S.: Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113, 1228–1242 (2018)
Huszár, F.: Variational inference using implicit distributions. arXiv preprint arXiv:1702.08235 (2017)
Shi, J., Sun, S., Zhu, J.: Kernel implicit variational inference. In: ICLR (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, F., Lin, A., Zhang, G., Lu, J. (2018). Counterfactual Inference with Hidden Confounders Using Implicit Generative Models. In: Mitrovic, T., Xue, B., Li, X. (eds) AI 2018: Advances in Artificial Intelligence. AI 2018. Lecture Notes in Computer Science(), vol 11320. Springer, Cham. https://doi.org/10.1007/978-3-030-03991-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-03991-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03990-5
Online ISBN: 978-3-030-03991-2
eBook Packages: Computer ScienceComputer Science (R0)