Semiparametric Theory and Empirical Processes in Causal Inference

  • Edward H. Kennedy
Part of the ICSA Book Series in Statistics book series (ICSABSS)


In this paper we review important aspects of semiparametric theory and empirical processes that arise in causal inference problems. We begin with a brief introduction to the general problem of causal inference, and go on to discuss estimation and inference for causal effects under semiparametric models, which allow parts of the data-generating process to be unrestricted if they are not of particular interest (i.e., nuisance functions). These models are very useful in causal problems because the outcome process is often complex and difficult to model, and there may only be information available about the treatment process (at best). Semiparametric theory gives a framework for benchmarking efficiency and constructing estimators in such settings. In the second part of the paper we discuss empirical process theory, which provides powerful tools for understanding the asymptotic behavior of semiparametric estimators that depend on flexible nonparametric estimators of nuisance functions. These tools are crucial for incorporating machine learning and other modern methods into causal inference analyses. We conclude by examining related extensions and future directions for work in semiparametric causal inference.


Propensity Score Tangent Space Causal Inference Empirical Process Influence Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Edward Kennedy acknowledges support from NIH grant R01-DK090385, and thanks Jason Roy and Bret Zeldow for very helpful comments and discussion.


  1. 1.
    Andrews, D.W.K.: Empirical process methods in econometrics. Handb. Econ. 4, 2247–2294 (1994)MathSciNetGoogle Scholar
  2. 2.
    Andrews, D.W.K.: Asymptotics for semiparametric econometric models via stochastic equicontinuity. Econometrica 62, 43–72 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Angrist, J.D., Imbens, G.W., Rubin, D.B.: Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91, 444–455 (1996)CrossRefzbMATHGoogle Scholar
  4. 4.
    Begun, J.M., Hall, W.J., Huang, W.M., Wellner, J.A.: Information and asymptotic efficiency in parametric-nonparametric models. Ann. Stat. 11, 432–452 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Belloni, A., Chernozhukov, V., Hansen, C.: Inference on treatment effects after selection among high-dimensional controls. Rev. Econ. Stud. 81, 608–650 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Belloni, A., Chernozhukov, V., Kato, K.: Uniform post-selection inference for least absolute deviation regression and other Z-estimation problems. Biometrika 102, 77–94 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bickel, P.J., Klaassen, C.A.J., Ritov, Y., Wellner, J.A.: Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York (1993)zbMATHGoogle Scholar
  8. 8.
    Carone, M., Diaz, I., van der Laan, M.J.: Higher-order targeted minimum loss-based estimation. U.C. Berkeley Division of Biostatistics Working Paper Series, vol. 331, pp. 1–39 (2015)Google Scholar
  9. 9.
    Chakraborty, B., Moodie, E.E.M.: Statistical Methods for Dynamic Treatment Regimes. Springer, New York (2013)CrossRefzbMATHGoogle Scholar
  10. 10.
    Dawid, P.A.: Causal inference without counterfactuals. J. Am. Stat. Assoc. 95, 407–424 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Diaz, I., van der Laan, M.J.: Targeted data adaptive estimation of the causal dose - response curve. J. Causal Inf. 1, 171–192 (2013)Google Scholar
  12. 12.
    Diaz, I., Carone, M., van der Laan, M.J.: Second order inference for the mean of a variable missing at random. U.C. Berkeley Division of Biostatistics Working Paper Series, vol. 337, pp. 1–22 (2015)Google Scholar
  13. 13.
    Gill, R.D., van der Laan, M.J., Wellner, J.A.: Inefficient estimators of the bivariate survival function for three models. Ann. Inst. Henri Poincare. 31, 545–597 (1995)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Gill, R.D., van der Laan, M.J., Robins, J.M.: Coarsening at random: characterizations, conjectures, counter-examples. In: Proceedings of the First Seattle Symposium in Biostatistics, pp. 255–294. Springer, New York (1997)Google Scholar
  15. 15.
    Hahn, J.: On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica 66, 315–333 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hernan, M.A., Robins, J.M.: Instruments for causal inference: an epidemiologist’s dream? Epidemiology 17, 360–372 (2006)CrossRefGoogle Scholar
  17. 17.
    Horowitz, J.L.: Semiparametric and Nonparametric Methods in Econometrics. Springer, New York (2009)CrossRefzbMATHGoogle Scholar
  18. 18.
    Hudgens, M.G., Halloran, M.E.: Toward causal inference with interference. J. Am. Stat. Assoc. 103, 832–842 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Kennedy, E.H., Sjolander, A., Small, D.S.: Semiparametric causal inference in matched cohort studies. Biometrika 102, 739–746 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Kennedy, E.H., Ma, Z., McHugh, M.D., Small, D.S.: Nonparametric methods for doubly robust estimation of continuous treatment effects. arXiv preprint, arXiv:1507.00747 (2015)Google Scholar
  21. 21.
    Kosorok, M.R.: Introduction to Empirical Processes and Semiparametric Inference. Springer, New York (2007)zbMATHGoogle Scholar
  22. 22.
    Luedtke, A.R., van der Laan, M.J.: Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy. U.C. Berkeley Division of Biostatistics Working Paper Series, vol. 332, pp. 1–37 (2014)Google Scholar
  23. 23.
    Manski, C.F.: Partial Identification of Probability Distributions. Springer, New York (2003)zbMATHGoogle Scholar
  24. 24.
    Murphy, S.A.: Optimal dynamic treatment regimes. J. R. Stat. Soc. B 65, 331–355 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Neugebauer, R., van der Laan, M.J.: Nonparametric causal effects based on marginal structural models. J. Stat. Plan. Infer. 137, 419–434 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Newey, W.K.: The asymptotic variance of semiparametric estimators. Econometrica 62, 1349–1382 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Newey, W.K., McFadden, D.: Large sample estimation and hypothesis testing. Handb. Econ. 4, 2111–2245 (1994)MathSciNetGoogle Scholar
  28. 28.
    Neyman, J.: On the application of probability theory to agricultural experiments: essay on principles. Excerpts reprinted (1990) in English (D. Dabrowska and T. Speed, trans.) Stat. Sci. 5, 463–472 (1923)Google Scholar
  29. 29.
    Ogburn, E.L., VanderWeele, T.J.: Causal diagrams for interference. Stat. Sci. 29, 559–578 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Pearl, J.: Causal diagrams for empirical research. Biometrika 82, 669–688 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Pearl, J.: Causality. Cambridge University Press, Cambridge (2009)CrossRefzbMATHGoogle Scholar
  32. 32.
    Petersen, M.L., Porter, K.E., Gruber, S., Wang, Y., van der Laan, M.J.: Diagnosing and responding to violations in the positivity assumption. Stat. Methods Med. Res. 21, 31–54 (2010)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Pfanzagl, J.: Contributions to a General Asymptotic Statistical Theory. Springer, New York (1982)CrossRefzbMATHGoogle Scholar
  34. 34.
    Pfanzagl, J.: Estimation in Semiparametric Models. Springer, New York (1990)CrossRefzbMATHGoogle Scholar
  35. 35.
    Pollard, D.: Convergence of stochastic processes. Springer, New York (1984)CrossRefzbMATHGoogle Scholar
  36. 36.
    Pollard, D.: Empirical processes: theory and applications. In: NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics and the American Statistical Association (1990)zbMATHGoogle Scholar
  37. 37.
    Robins, J.M.: A new approach to causal inference in mortality studies with a sustained exposure period - application to control of the healthy worker survivor effect. Math. Mod. 7, 1393–1512 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Robins, J.M., Rotnitzky, A., Zhao, L.P.: Estimation of regression coefficients when some regressors are not always observed. J. Am. Stat. Assoc. 89, 846–866 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Robins, J.M., Rotnitzky, A., Zhao, L.P.: Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. J. Am. Stat. Assoc. 90, 106–121 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Robins, J.M., Rotnitzky, A., Scharfstein, D.O.: Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In: Statistical Models in Epidemiology, the Environment, and Clinical Trials, pp. 1–94. Springer, New York (1999)Google Scholar
  41. 41.
    Robins, J.M., Hernan, M.A., Brumback, B.: Marginal structural models and causal inference in epidemiology. Epidemiology 11, 550–560 (2000)CrossRefGoogle Scholar
  42. 42.
    Robins, J.M., Li, L., Tchetgen, E., van der Vaart, A.W.: Higher order influence functions and minimax estimation of nonlinear functionals. In: Probability and Statistics: Essays in Honor of David A. Freedman, pp. 335–421. Beachwood, Ohio, USA, Institute of Mathematical Statistics (2008)Google Scholar
  43. 43.
    Robins, J.M., Hernan, M.A.: Estimation of the causal effects of time-varying exposures. In: Fitzmaurice, G., Davidian, M., Verbeke, G., Molenberghs, G. (eds.) Longitudinal Data Analysis, pp. 553–600. Chapman & Hall, London (2009)Google Scholar
  44. 44.
    Robins, J.M., Li, L., Tchetgen, E., van der Vaart, A.W.: Quadratic semiparametric von mises calculus. Metrika 69, 227–247 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Rose, S., van der Laan, M.J.: A double robust approach to causal effects in case-control studies. Am. J. Epid. 179, 662–669 (2014)CrossRefGoogle Scholar
  46. 46.
    Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688–701 (1974)CrossRefGoogle Scholar
  47. 47.
    Rubin, D.B.: Bayesian inference for causal effects: the role of randomization. Ann. Stat. 6, 34–58 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Shorack, G.R., Wellner, J.A.: Empirical Processes with Applications to Statistics. Wiley, New York (1986)zbMATHGoogle Scholar
  49. 49.
    Stefanski, L.A., Boos, D.D.: The calculus of M-estimation. Am. Stat. 56, 29–38 (2002)MathSciNetCrossRefGoogle Scholar
  50. 50.
    Tchetgen, E., Rotnitzky, A.: Double-robust estimation of an exposure-outcome odds ratio adjusting for confounding in cohort and case-control studies. Stat. Med. 30, 335–347 (2011)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Tchetgen, E., Shpitser, I.: Semiparametric theory for causal mediation analysis: efficiency bounds, multiple robustness and sensitivity analysis. Ann. Stat. 40, 1816–1845 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Tchetgen, E., VanderWeele, T.J.: On causal inference in the presence of interference. Stat. Methods Med. Res. 21, 55–75 (2012)MathSciNetCrossRefGoogle Scholar
  53. 53.
    Tsiatis, A.A.: Semiparametric Theory and Missing Data. Springer, New York (2006)zbMATHGoogle Scholar
  54. 54.
    van der Laan, M.J.: Estimation based on case-control designs with known prevalence probability. Int. J. Biostat. 4 (2008). Article 17Google Scholar
  55. 55.
    van der Laan, M.J.: Causal inference for a population of causally connected units. J. Causal Inf. 2, 13–74 (2014)Google Scholar
  56. 56.
    van der Laan, M.J.: Targeted estimation of nuisance parameters to obtain valid statistical inference. Int. J. Biostat. 10, 29–57 (2014)MathSciNetGoogle Scholar
  57. 57.
    van der Laan, M.J.: Targeted learning: From MLE to TMLE. In: Lin, X., Genest, C., Banks, D.L., et al. (eds.) Past, Present, and Future of Statistical Science, pp. 465–480. Chapman & Hall, London (2014)CrossRefGoogle Scholar
  58. 58.
    van der Laan, M.J., Dudoit, S.: Unified cross-validation methodology for selection among estimators and a general cross-validated adaptive epsilon-net estimator: finite sample oracle inequalities and examples. U.C. Berkeley Division of Biostatistics Working Paper Series. vol. 130, pp. 1–103 (2003)Google Scholar
  59. 59.
    van der Laan, M.J., Robins, J.M.: Unified Methods for Censored Longitudinal Data and Causality. Springer, New York (2003)CrossRefzbMATHGoogle Scholar
  60. 60.
    van der Laan, M.J., Rose, S.: Targeted Learning: Causal Inference for Observational and Experimental Data. Springer, New York (2011)CrossRefGoogle Scholar
  61. 61.
    van der Laan, M.J., Rubin, D.: Targeted maximum likelihood learning. Int. J. Biostat. 2, 1–38 (2006)MathSciNetGoogle Scholar
  62. 62.
    van der Laan, M.J., Polley, E.C., Hubbard, A.E.: Super learner. Stat. Appl. Genet. Mol. 6, 1–21 (2007)MathSciNetzbMATHGoogle Scholar
  63. 63.
    van der Laan, M.J., Petersen, M., Zheng, W.: Estimating the effect of a community-based intervention with two communities. J. Causal Inf. 1, 83–106 (2013)Google Scholar
  64. 64.
    van der Vaart, A.W.: Asymptotic Statistics. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  65. 65.
    van der Vaart, A.W.: Part III: Semiparametric Statistics. In: Bernard, P. (ed.) Lectures on Probability Theory and Statistics, pp. 331–457. Springer, New York (2002)Google Scholar
  66. 66.
    van der Vaart, A.W.: Higher order tangent spaces and influence functions. Stat. Sci. 29, 679–686 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  67. 67.
    van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes. Springer, New York (1996)CrossRefzbMATHGoogle Scholar
  68. 68.
    VanderWeele, T.J.: Concerning the consistency assumption in causal inference. Epidemiology 20, 880–883 (2009)CrossRefGoogle Scholar
  69. 69.
    VanderWeele, T.J.: Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press, Oxford (2015)Google Scholar
  70. 70.
    VanderWeele, T.J., Vansteelandt, S.: A weighting approach to causal effects and additive interaction in case-control studies: marginal structural linear odds models. Am. J. Epidemiol. 174, 1197–1203 (2011)CrossRefGoogle Scholar
  71. 71.
    VanderWeele, T.J., Vansteelandt, S.: Invited commentary: some advantages of the relative excess risk due to interaction (RERI) - towards better estimators of additive interaction. Am. J. Epidemiol. 179, 670–671 (2014)CrossRefGoogle Scholar
  72. 72.
    Zheng, W., van der Laan, M.J.: Asymptotic theory for cross-validated targeted maximum likelihood estimation. U.C. Berkeley Division of Biostatistics Working Paper Series, vol. 273, pp. 1–58 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.University of PennsylvaniaPhiladelphiaUSA

Personalised recommendations