Advertisement

Higher Order Equivalence of Bayes Cross Validation and WAIC

  • Sumio Watanabe
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 252)

Abstract

It was proved in the previous paper (Watanabe, J Mach Learn Res, 11:3571–3591, (2010), [16]) that Bayes cross validation is asymptotically equivalent to the widely applicable information criterion (WAIC), even if the posterior distribution can not be approximated by any normal distribution. In the present paper, we prove that they are equivalent to each other according to the second order asymptotics, if the posterior distribution can be approximated by some normal distribution. Based on this equivalence, it is also shown that the Bayes cross validation and WAIC are asymptotically equivalent as functions of a hyperparameter of a prior.

Keywords

Cross validation WAIC Asymptotic equivalence Second order 

Notes

Acknowledgements

This research was partially supported by the Ministry of Education, Science, Sports and Culture in Japan, Grant-in-Aid for Scientific Research 23500172, 25120013, and 15K00331.

References

  1. 1.
    Akaike, H.: Likelihood and Bayes procedure. In: Bernald, J.M. (ed.) Bayesian Statistics, pp. 143–166 (1980)Google Scholar
  2. 2.
    Clarke, B.S., Barron, A.R.: Jeffreys’ prior is asymptotically least favorable under entropy risk. J. Stat. Plan. Inference 41, 36–60 (1994)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Epifani, I., MacEachern, S.N., Peruggia, M.: Case-deletion importance sampling estimators: central limit theorems and related results. Electron. J. Stat. 2, 774–806 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Gelfand, A.E., Dey, D.K., Chang, H.: Model determination using predictive distributions with implementation via sampling-based method. Bayesian Stat. 4, 147–167 (1992)MathSciNetGoogle Scholar
  5. 5.
    Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis, 3rd edn. Chapman and Hall/CRC, New York (2013)zbMATHGoogle Scholar
  6. 6.
    Gelman, A., Hwang, J., Vehtari, A.: Understanding predictive information criteria for bayesian models. Stat. Comput. 24, 997–1016 (2014).  https://doi.org/10.1007/s11222-013-9416-2MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Good, I.J.: Rational decisions. J. R. Stat. Soc. Ser. B 14, 107–114 (1952)MathSciNetGoogle Scholar
  8. 8.
    Komaki, F.: On asymptotic properties of predictive distributions. Biometrika 83(2), 299–313 (1996)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Peruggia, M.: On the variability of case-detection importance sampling weights in the bayesian linear model. J. Am. Stat. Assoc. 92, 199–207 (1997)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A.: Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B 64(4), 583–639 (2002)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A.: The deviance information criterion: 12 years on. J. R. Stat. Soc. Ser. B (2014).  https://doi.org/10.1111/rssb.12062MathSciNetCrossRefGoogle Scholar
  12. 12.
    Vehtari, A., Lampinen, J.: Bayesian model assessment and comparison using cross-validation predictive densities. Neural Comput. 14(10), 2439–2468 (2002)CrossRefGoogle Scholar
  13. 13.
    Vehtari, A., Ojanen, J.: A survey of Bayesian predictive methods for model assessment, selection and comparison. Stat. Surv. 6, 142–228 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Comput. 13(4), 899–933 (2001)CrossRefGoogle Scholar
  15. 15.
    Watanabe, S.: Algebraic Geometry and Statistical Learning Theory. Cambridge University Press, Cambridge (2009)CrossRefGoogle Scholar
  16. 16.
    Watanabe, S.: Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J. Mach. Learn. Res. 11, 3571–3591 (2010)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Watanabe, S.: Equations of states in singular statistical estimation. Neural Netw. 23(1), 20–34 (2010)CrossRefGoogle Scholar
  18. 18.
    Watanabe, S.: Bayesian cross validation and WAIC for predictive prior design in regular asymptotic theory (2015). arXiv:1503.07970

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Tokyo Institute of TechnologyTokyoJapan

Personalised recommendations