Advertisement

Proceedings of the Steklov Institute of Mathematics

, Volume 287, Issue 1, pp 232–255 | Cite as

Critical dimension in the semiparametric Bernstein—von Mises theorem

  • Maxim E. Panov
  • Vladimir G. Spokoiny
Article

Abstract

The classical parametric and semiparametric Bernstein-von Mises (BvM) results are reconsidered in a nonclassical setup allowing finite samples and model misspecification. In the parametric case and in the case of a finite-dimensional nuisance parameter, we establish an upper bound on the error of Gaussian approximation of the posterior distribution of the target parameter; the bound depends explicitly on the dimension of the full and target parameters and on the sample size. This helps to identify the so-called critical dimension p n of the full parameter for which the BvM result is applicable. In the important special i.i.d. case, we show that the condition “p n 3 /n is small” is sufficient for the BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension p n approaches n 1/3.

Keywords

Posterior Distribution STEKLOV Institute Maximum Likelihood Estimator Critical Dimension Asymptotic Normality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A. Barron, M. J. Schervish, and L. Wasserman, “The consistency of posterior distributions in nonparametric problems,” Ann. Stat. 27(2), 536–561 (1996).MathSciNetGoogle Scholar
  2. 2.
    P. J. Bickel and B. J. K. Kleijn, “The semiparametric Bernstein-von Mises theorem,” Ann. Stat. 40(1), 206–237 (2012).CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    N. Bochkina and P. J. Green, “The Bernstein-von Mises theorem and nonregular models,” Ann. Stat. 42(5), 1850–1878 (2014).CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    D. Bontemps, “Bernstein-von Mises theorems for Gaussian regression with increasing number of regressors,” Ann. Stat. 39(5), 2557–2584 (2011).CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    S. Boucheron and E. Gassiat, “A Bernstein-von Mises theorem for discrete probability distributions,” Electron. J. Stat. 3, 114–148 (2009).CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    S. Boucheron and P. Massart, “A high-dimensional Wilks phenomenon,” Probab. Theory Relat. Fields 150(3–4), 405–433 (2011).CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    I. Castillo, “A semiparametric Bernstein-von Mises theorem for Gaussian process priors,” Probab. Theory Relat. Fields 152(1–2), 53–99 (2012).CrossRefzbMATHGoogle Scholar
  8. 8.
    I. Castillo and R. Nickl, “Nonparametric Bernstein-von Mises theorems in Gaussian white noise,” Ann. Stat. 41(4), 1999–2028 (2013).CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    I. Castillo and J. Rousseau, “A general Bernstein-von Mises theorem in semiparametric models,” arXiv: 1305.4482v1 [math.ST].Google Scholar
  10. 10.
    G. Cheng and M. R. Kosorok, “General frequentist properties of the posterior profile distribution,” Ann. Stat. 36(4), 1819–1853 (2008).CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    V. Chernozhukov and H. Hong, “An MCMC approach to classical estimation,” J. Econom. 115(2), 293–346 (2003).CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    D. D. Cox, “An analysis of Bayesian inference for nonparametric regression,” Ann. Stat. 21(2), 903–923 (1993).CrossRefzbMATHGoogle Scholar
  13. 13.
    D. Freedman, “On the Bernstein-von Mises theorem with infinite-dimensional parameters,” Ann. Stat. 27(4), 1119–1140 (1999).zbMATHGoogle Scholar
  14. 14.
    S. Ghosal, “Asymptotic normality of posterior distributions in high-dimensional linear models,” Bernoulli 5(2), 315–331 (1999).CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    S. Ghosal, “Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity,” J. Multivariate Anal. 74(1), 49–68 (2000).CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    I. M. Johnstone, “High dimensional Bernstein-von Mises: Simple examples,” in Borrowing Strength: Theory Powering Applications-A Festschrift for Lawrence D. Brown (Inst. Math. Stat., Beachwood, OH, 2010), Inst. Math. Stat. Collect. 6, pp. 87–98.Google Scholar
  17. 17.
    Y. Kim, “The Bernstein-von Mises theorem for the proportional hazard model,” Ann. Stat. 34(4), 1678–1700 (2006).CrossRefzbMATHGoogle Scholar
  18. 18.
    Y. Kim and J. Lee, “A Bernstein-von Mises theorem in the nonparametric right-censoring model,” Ann. Stat. 32(4), 1492–1512 (2004).CrossRefzbMATHGoogle Scholar
  19. 19.
    B. J. K. Kleijn and A. W. van der Vaart, “Misspecification in infinite-dimensional Bayesian statistics,” Ann. Stat. 34(2), 837–877 (2006).CrossRefzbMATHGoogle Scholar
  20. 20.
    B. J. K. Kleijn and A. W. van der Vaart, “The Bernstein-von Mises theorem under misspecification,” Electron. J. Stat. 6, 354–381 (2012).CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    L. Le Cam and G. L. Yang, Asymptotics in Statistics: Some Basic Concepts (Springer, New York, 1990), Springer Ser. Stat.CrossRefzbMATHGoogle Scholar
  22. 22.
    H. Leahu, “On the Bernstein-von Mises phenomenon in the Gaussian white noise model,” Electron. J. Stat. 5, 373–404 (2011).CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    J. Polzehl and V. Spokoiny, “Propagation-separation approach for local likelihood estimation,” Probab. Theory Relat. Fields 135(3), 335–362 (2006).CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    V. Rivoirard and J. Rousseau, “Bernstein-von Mises theorem for linear functionals of the density,” Ann. Stat. 40(3), 1489–1523 (2012).CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    L. Schwartz, “On Bayes procedures,” Z. Wahrscheinlichkeitstheor. Verw. Geb. 4(1), 10–26 (1965).CrossRefzbMATHGoogle Scholar
  26. 26.
    X. Shen, “Asymptotic normality of semiparametric and nonparametric posterior distributions,” J. Am. Stat. Assoc. 97(457), 222–235 (2002).CrossRefzbMATHGoogle Scholar
  27. 27.
    V. Spokoiny, “Parametric estimation. Finite sample theory,” Ann. Stat. 40(6), 2877–2909 (2012); arXiv: 1111.3029 [math.ST].CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    V. Spokoiny and M. Zhilova, “Sharp deviation bounds for quadratic forms,” Math. Methods Stat. 22(2), 100–113 (2013); arXiv: 1302.1699 [math.PR].CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    A. W. van der Vaart, Asymptotic Statistics (Cambridge Univ. Press, Cambridge, 2000), Cambridge Ser. Stat. Probab. Math. 3.Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2014

Authors and Affiliations

  1. 1.Moscow Institute of Physics and Technology (State University)Dolgoprudnyi, Moscow oblastRussia
  2. 2.Institute for Information Transmission Problems (Kharkevich Institute)Russian Academy of SciencesMoscowRussia
  3. 3.Datadvance CompanyMoscowRussia
  4. 4.Weierstrass Institute for Applied Analysis and StochasticsBerlinGermany
  5. 5.Humboldt-Universität zu BerlinBerlinGermany

Personalised recommendations