Skip to main content
SpringerLink
Log in

Higher Order Equivalence of Bayes Cross Validation and WAIC

  • Conference paper
  • First Online:
Information Geometry and Its Applications (IGAIA IV 2016)

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 252))

Included in the following conference series:

Abstract

It was proved in the previous paper (Watanabe, J Mach Learn Res, 11:3571–3591, (2010), [16]) that Bayes cross validation is asymptotically equivalent to the widely applicable information criterion (WAIC), even if the posterior distribution can not be approximated by any normal distribution. In the present paper, we prove that they are equivalent to each other according to the second order asymptotics, if the posterior distribution can be approximated by some normal distribution. Based on this equivalence, it is also shown that the Bayes cross validation and WAIC are asymptotically equivalent as functions of a hyperparameter of a prior.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akaike, H.: Likelihood and Bayes procedure. In: Bernald, J.M. (ed.) Bayesian Statistics, pp. 143–166 (1980)

    Google Scholar 

  2. Clarke, B.S., Barron, A.R.: Jeffreys’ prior is asymptotically least favorable under entropy risk. J. Stat. Plan. Inference 41, 36–60 (1994)

    Article  MathSciNet  Google Scholar 

  3. Epifani, I., MacEachern, S.N., Peruggia, M.: Case-deletion importance sampling estimators: central limit theorems and related results. Electron. J. Stat. 2, 774–806 (2008)

    Article  MathSciNet  Google Scholar 

  4. Gelfand, A.E., Dey, D.K., Chang, H.: Model determination using predictive distributions with implementation via sampling-based method. Bayesian Stat. 4, 147–167 (1992)

    MathSciNet  Google Scholar 

  5. Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis, 3rd edn. Chapman and Hall/CRC, New York (2013)

    MATH  Google Scholar 

  6. Gelman, A., Hwang, J., Vehtari, A.: Understanding predictive information criteria for bayesian models. Stat. Comput. 24, 997–1016 (2014). https://doi.org/10.1007/s11222-013-9416-2

    Article  MathSciNet  MATH  Google Scholar 

  7. Good, I.J.: Rational decisions. J. R. Stat. Soc. Ser. B 14, 107–114 (1952)

    MathSciNet  Google Scholar 

  8. Komaki, F.: On asymptotic properties of predictive distributions. Biometrika 83(2), 299–313 (1996)

    Article  MathSciNet  Google Scholar 

  9. Peruggia, M.: On the variability of case-detection importance sampling weights in the bayesian linear model. J. Am. Stat. Assoc. 92, 199–207 (1997)

    Article  MathSciNet  Google Scholar 

  10. Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A.: Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B 64(4), 583–639 (2002)

    Article  MathSciNet  Google Scholar 

  11. Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A.: The deviance information criterion: 12 years on. J. R. Stat. Soc. Ser. B (2014). https://doi.org/10.1111/rssb.12062

    Article  MathSciNet  Google Scholar 

  12. Vehtari, A., Lampinen, J.: Bayesian model assessment and comparison using cross-validation predictive densities. Neural Comput. 14(10), 2439–2468 (2002)

    Article  Google Scholar 

  13. Vehtari, A., Ojanen, J.: A survey of Bayesian predictive methods for model assessment, selection and comparison. Stat. Surv. 6, 142–228 (2012)

    Article  MathSciNet  Google Scholar 

  14. Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Comput. 13(4), 899–933 (2001)

    Article  Google Scholar 

  15. Watanabe, S.: Algebraic Geometry and Statistical Learning Theory. Cambridge University Press, Cambridge (2009)

    Book  Google Scholar 

  16. Watanabe, S.: Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J. Mach. Learn. Res. 11, 3571–3591 (2010)

    MathSciNet  MATH  Google Scholar 

  17. Watanabe, S.: Equations of states in singular statistical estimation. Neural Netw. 23(1), 20–34 (2010)

    Article  Google Scholar 

  18. Watanabe, S.: Bayesian cross validation and WAIC for predictive prior design in regular asymptotic theory (2015). arXiv:1503.07970

Download references

Acknowledgements

This research was partially supported by the Ministry of Education, Science, Sports and Culture in Japan, Grant-in-Aid for Scientific Research 23500172, 25120013, and 15K00331.

Author information

Authors and Affiliations

  1. Tokyo Institute of Technology, Tokyo, Japan

    Sumio Watanabe

Authors
  1. Sumio Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sumio Watanabe .

Editor information

Editors and Affiliations

  1. Information Theory of Cognitive Systems, MPI for Mathematics in the Sciences, Leipzig, Germany

    Nihat Ay

  2. Department of Economics and Finance, University of Rome “Tor Vergata”, Rome, Italy

    Paolo Gibilisco

  3. Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic, Prague, Czech Republic

    František Matúš

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Watanabe, S. (2018). Higher Order Equivalence of Bayes Cross Validation and WAIC. In: Ay, N., Gibilisco, P., Matúš, F. (eds) Information Geometry and Its Applications . IGAIA IV 2016. Springer Proceedings in Mathematics & Statistics, vol 252. Springer, Cham. https://doi.org/10.1007/978-3-319-97798-0_3

Download citation

Publish with us

Policies and ethics

Search

Navigation