Skip to main content
SpringerLink
Log in

On Accuracy of Gaussian Approximation in Bayesian Semiparametric Problems

  • Conference paper
  • First Online:
Foundations of Modern Statistics (FMS 2019)

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 425))

Included in the following conference series:

  • 350 Accesses

Abstract

We consider the problem of Bayesian semiparametric inference and aim to obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter. This type of result can be seen as a nonasymptotic version of semiparametric Bernstein–von Mises (BvM). The provided bound is explicit in the dimension of the target parameter and in the dimension of sieve approximation of the full parameter. As a result, we can introduce the so-called critical dimension \(\,p_n\,\) of the sieve approximation, the maximal dimension for which the BvM result remains valid. In various particular statistical models, we show the necessity of the condition “\(\,p_n^2 q / n\,\) is small”, where \(\,q\,\) is the dimension of the target parameter and \(\,n\,\) is the sample size, for the BvM result to be valid under the general assumptions on the model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andresen, A., Spokoiny, V.: Critical dimension in profile semiparametric estimation. Electron. J. Statist. 8(2), 3077–3125 (2014). https://doi.org/10.1214/14-EJS982

    Article  MathSciNet  MATH  Google Scholar 

  2. Barron, A., Schervish, M.J., Wasserman, L.: The consistency of posterior distributions in nonparametric problems. Ann. Stat. 27, 536–561 (1996). https://doi.org/10.1214/aos/1018031206

    Article  MathSciNet  MATH  Google Scholar 

  3. Belloni, A., Chernozhukov, V.: Posterior inference in curved exponential families under increasing dimensions. Econ. J. 17(2), S75–S100 (2014). https://doi.org/10.1111/ectj.12027

    Article  MathSciNet  MATH  Google Scholar 

  4. Bernstein, S.: Lecture Notes on Probability Theory. Kharkiv University (1917)

    Google Scholar 

  5. Bickel, P.J., Kleijn, B.J.K.: The semiparametric Bernstein-von Mises theorem. Ann. Stat. 40(1), 206–237 (2012). https://doi.org/10.1214/11-AOS921

    Article  MathSciNet  MATH  Google Scholar 

  6. Bochkina, N.: Consistency of the posterior distribution in generalized linear inverse problems. Inverse Probl. 29(9), 095010 (2013). https://doi.org/10.1088/0266-5611/29/9/095010

  7. Boucheron, S., Gassiat, E.: A Bernstein-von Mises theorem for discrete probability distributions. Electron. J. Stat. 3, 114–148 (2009). https://doi.org/10.1214/08-EJS262

    Article  MathSciNet  MATH  Google Scholar 

  8. Boucheron, S., Massart, P.: A high-dimensional Wilks phenomenon. Probab. Theory Relat. Fields. 150, 405–433 (2011). https://doi.org/10.1007/s00440-010-0278-7

    Article  MathSciNet  MATH  Google Scholar 

  9. Buhlmann, P., van de Geer, S.: Statistics for High-Dimensional Data: Methods, Theory and Applications, 1st edn. Springer Publishing Company, Incorporated (2011). https://doi.org/10.1007/978-3-642-20192-9

  10. Burnashev, M.V.: Investigation of second order properties of statistical estimators in a scheme of independent observations. Izv. Akad. Nauk USSR Ser. Mat. 45(3), 509–539 (1981)

    MathSciNet  MATH  Google Scholar 

  11. Castillo, I.: A semiparametric Bernstein—von Mises theorem for Gaussian process priors. Probab. Theory Relat. Fields. 152, 53–99 (2012). https://doi.org/10.1007/s00440-010-0316-5

    Article  MathSciNet  MATH  Google Scholar 

  12. Castillo, I., Nickl, R.: Nonparametric Bernstein-von Mises theorems in Gaussian white noise. Ann. Stat. 41(4), 1999–2028 (2013). https://doi.org/10.1214/13-AOS1133

    Article  MathSciNet  MATH  Google Scholar 

  13. Castillo, I., Rousseau, J.: A general bernstein–von mises theorem in semiparametric models (2013). arXiv:1305.4482 [math.ST]

  14. Cheng, G., Kosorok, M.R.: General frequentist properties of the posterior profile distribution. Ann. Stat. 36(4), 1819–1853 (2008). https://doi.org/10.1214/07-AOS536

    Article  MathSciNet  MATH  Google Scholar 

  15. Chentsov, N.N.: A bound for an unknown distribution density in terms of the observations. Dokl. Akad. Nauk USSR 147(1), 45–48 (1962)

    MathSciNet  Google Scholar 

  16. Chernozhukov, V., Hong, H.: An mcmc approach to classical estimation. J. Econ. 115(2), 293–346 (2003). https://doi.org/10.1016/S0304-4076(03)00100-3

    Article  MathSciNet  MATH  Google Scholar 

  17. Cox, D.D.: An analysis of Bayesian inference for nonparametric regression. Ann. Stat. 21(2), 903–923 (1993). https://doi.org/10.1214/aos/1176349157

    Article  MathSciNet  MATH  Google Scholar 

  18. Ermakov, M.: On semiparametric statistical inferences in the moderate deviation zone. J. Math. Sci. 152(6), 869–874 (2008). https://doi.org/10.1007/s10958-008-9104-5

    Article  MathSciNet  MATH  Google Scholar 

  19. Freedman, D.: On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Stat. 27(4), 1119–1140 (1999). https://doi.org/10.1214/aos/1017938917

    Article  MathSciNet  MATH  Google Scholar 

  20. Ghosal, S.: Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5(2), 315–331 (1999). https://doi.org/10.2307/3318438

    Article  MathSciNet  MATH  Google Scholar 

  21. Gusev S.I.: Asymptotic expansions associated with some statistical estimators in the smooth case. i. expansions of random variables. Theory Probab. Appl. 20(3), 488–514 (1975)

    Google Scholar 

  22. Gusev S.I.: Asymptotic expansions associated with some statistical estimators in the smooth case. ii. expansions of moments and distributions. Theory Probab. Appl. 21(1), 16–33 (1976)

    Google Scholar 

  23. Ibragimov, I., Khas’minskij, R.: Statistical Estimation. Springer-Verlag, Asymptotic theory. New York - Heidelberg -Berlin (1981)

    Book  MATH  Google Scholar 

  24. Kim, Y.: The Bernstein—von Mises theorem for the proportional hazard model. Ann. Stat. 34(4), 1678–1700 (2006). https://doi.org/10.1214/009053606000000533

    Article  MathSciNet  MATH  Google Scholar 

  25. Kim, Y., Lee, J.: A Bernstein—von Mises theorem in the nonparametric right-censoring model. Ann. Stat. 32(4), 1492–1512 (2004). https://doi.org/10.1214/009053604000000526

    Article  MathSciNet  MATH  Google Scholar 

  26. Kleijn, B.J.K., van der Vaart, A.W.: Misspecification in infinite-dimensional Bayesian statistics. Ann. Stat. 34(2), 837–877 (2006). https://doi.org/10.1214/009053606000000029

    Article  MathSciNet  MATH  Google Scholar 

  27. Kleijn, B.J.K., van der Vaart, A.W.: The Bernstein-von-Mises theorem under misspecification. Electron. J. Stat. 6, 354–381 (2012). https://doi.org/10.1214/12-EJS675

    Article  MathSciNet  MATH  Google Scholar 

  28. Kosorok, M.R.: Springer series in statistics. Introduction to Empirical Processes and Semiparametric Inference (2008). https://doi.org/10.1007/978-0-387-74978-5

  29. Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28(5), 1302–1338 (2000). https://doi.org/10.1214/aos/1015957395

    Article  MathSciNet  MATH  Google Scholar 

  30. Le Cam, L., Yang, G.L.: Asymptotics in Statistics: Some Basic Concepts. Springer in Statistics (1990). https://doi.org/10.1007/978-1-4612-1166-2

  31. Leahu, H.: On the bernstein-von mises phenomenon in the gaussian white noise model. Electron. J. Stat. 5, 373–404 (2011). https://doi.org/10.1214/11-EJS611

    Article  MathSciNet  MATH  Google Scholar 

  32. Mises, R.: Wahrscheinlichkeitsrechnung und ihre Anwendung in der Statistik und theoretischen Physik. Mary S, Rosenberg (1931)

    MATH  Google Scholar 

  33. Panov, M.: Nonasymptotic approach to bayesian semiparametric inference. Dokl. Math. 93(2), 155–158 (2016). https://doi.org/10.1134/S1064562416020101

    Article  MathSciNet  MATH  Google Scholar 

  34. Panov, M., Spokoiny, V.: Finite sample Bernstein—von Mises theorem for semiparametric problems. Bayesian Anal. 10(3), 665–710 (2015). https://doi.org/10.1214/14-BA926

    Article  MathSciNet  MATH  Google Scholar 

  35. Rivoirard, V., Rousseau, J.: Bernstein—von Mises theorem for linear functionals of the density. Ann. Stat. 40(3), 1489–1523 (2012). https://doi.org/10.1214/12-AOS1004

    Article  MathSciNet  MATH  Google Scholar 

  36. Rivoirard, V., Rousseau, J.: Posterior concentration rates for infinite dimensional exponential families. Bayesian Anal. 7(2), 311–334 (2012). https://doi.org/10.1214/12-BA710

    Article  MathSciNet  MATH  Google Scholar 

  37. Schwartz, L.: On bayes procedures. Probab. Theory Relat. Fields. 4(1), 10–26 (1965)

    MathSciNet  MATH  Google Scholar 

  38. Shen, X.: Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Am. Stat. Assoc. 97(457), 222–235 (2002). https://doi.org/10.1198/016214502753479365

    Article  MathSciNet  MATH  Google Scholar 

  39. Spokoiny, V.: Parametric estimation. Finite sample theory. Ann. Stat. 40(6), 2877–2909 (2012). https://doi.org/10.1214/12-AOS1054

  40. Spokoiny, V.: Bernstein—von Mises Theorem for growing parameter dimension (2013). Manuscript. arXiv:1302.3430

  41. Spokoiny, V.: Penalized maximum likelihood estimation and effective dimension. AIHP (2015). https://doi.org/10.1214/15-AIHP720

    Article  MATH  Google Scholar 

  42. Spokoiny, V., Panov, M.: Accuracy of gaussian approximation in nonparametric bernstein–von mises theorem (2019). arXiv preprint arXiv:1910.06028

  43. Spokoiny, V., Zhilova, M.: Sharp deviation bounds for quadratic forms. Math. Methods Stat. 22(2), 100–113 (2013). https://doi.org/10.3103/S1066530713020026

    Article  MathSciNet  MATH  Google Scholar 

  44. van der Vaart, A.W.: Asymptotic Statistics. Cambridge University Press (2000). https://doi.org/10.1017/CBO9780511802256

Download references

Acknowledgements

The research was supported by the Russian Science Foundation grant 20-71-10135.

Author information

Authors and Affiliations

  1. Skolkovo Institute of Science and Technology, Moscow, Russia

    Maxim Panov

Authors
  1. Maxim Panov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maxim Panov .

Editor information

Editors and Affiliations

  1. Faculty of Mathematics, University of Duisburg-Essen, Essen, Germany

    Denis Belomestny

  2. Institut Polytechnique de Paris, CREST, ENSAE, Palaiseau, France

    Cristina Butucea

  3. Institute for Applied Mathematics, Heidelberg University, Heidelberg, Baden-Württemberg, Germany

    Enno Mammen

  4. CMAP, Ecole Polytechnique, Palaiseau, France

    Eric Moulines

  5. Institute of Mathematics, Humboldt-Universität zu Berlin, Berlin, Germany

    Markus Reiß

  6. Faculty of Computer Science, HSE University and Moscow State University, Moscow, Russia

    Vladimir V. Ulyanov

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Panov, M. (2023). On Accuracy of Gaussian Approximation in Bayesian Semiparametric Problems. In: Belomestny, D., Butucea, C., Mammen, E., Moulines, E., Reiß, M., Ulyanov, V.V. (eds) Foundations of Modern Statistics. FMS 2019. Springer Proceedings in Mathematics & Statistics, vol 425. Springer, Cham. https://doi.org/10.1007/978-3-031-30114-8_11

Download citation

Publish with us

Policies and ethics

Search

Navigation