Proceedings of the Steklov Institute of Mathematics

, Volume 287, Issue 1, pp 232–255 | Cite as

Critical dimension in the semiparametric Bernstein—von Mises theorem

Article
  • 58 Downloads

Abstract

The classical parametric and semiparametric Bernstein-von Mises (BvM) results are reconsidered in a nonclassical setup allowing finite samples and model misspecification. In the parametric case and in the case of a finite-dimensional nuisance parameter, we establish an upper bound on the error of Gaussian approximation of the posterior distribution of the target parameter; the bound depends explicitly on the dimension of the full and target parameters and on the sample size. This helps to identify the so-called critical dimension p n of the full parameter for which the BvM result is applicable. In the important special i.i.d. case, we show that the condition “p n 3 /n is small” is sufficient for the BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension p n approaches n 1/3.

Keywords

Posterior Distribution STEKLOV Institute Maximum Likelihood Estimator Critical Dimension Asymptotic Normality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A. Barron, M. J. Schervish, and L. Wasserman, “The consistency of posterior distributions in nonparametric problems,” Ann. Stat. 27(2), 536–561 (1996).MathSciNetGoogle Scholar
  2. 2.
    P. J. Bickel and B. J. K. Kleijn, “The semiparametric Bernstein-von Mises theorem,” Ann. Stat. 40(1), 206–237 (2012).CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    N. Bochkina and P. J. Green, “The Bernstein-von Mises theorem and nonregular models,” Ann. Stat. 42(5), 1850–1878 (2014).CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    D. Bontemps, “Bernstein-von Mises theorems for Gaussian regression with increasing number of regressors,” Ann. Stat. 39(5), 2557–2584 (2011).CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    S. Boucheron and E. Gassiat, “A Bernstein-von Mises theorem for discrete probability distributions,” Electron. J. Stat. 3, 114–148 (2009).CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    S. Boucheron and P. Massart, “A high-dimensional Wilks phenomenon,” Probab. Theory Relat. Fields 150(3–4), 405–433 (2011).CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    I. Castillo, “A semiparametric Bernstein-von Mises theorem for Gaussian process priors,” Probab. Theory Relat. Fields 152(1–2), 53–99 (2012).CrossRefMATHGoogle Scholar
  8. 8.
    I. Castillo and R. Nickl, “Nonparametric Bernstein-von Mises theorems in Gaussian white noise,” Ann. Stat. 41(4), 1999–2028 (2013).CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    I. Castillo and J. Rousseau, “A general Bernstein-von Mises theorem in semiparametric models,” arXiv: 1305.4482v1 [math.ST].Google Scholar
  10. 10.
    G. Cheng and M. R. Kosorok, “General frequentist properties of the posterior profile distribution,” Ann. Stat. 36(4), 1819–1853 (2008).CrossRefMATHMathSciNetGoogle Scholar
  11. 11.
    V. Chernozhukov and H. Hong, “An MCMC approach to classical estimation,” J. Econom. 115(2), 293–346 (2003).CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    D. D. Cox, “An analysis of Bayesian inference for nonparametric regression,” Ann. Stat. 21(2), 903–923 (1993).CrossRefMATHGoogle Scholar
  13. 13.
    D. Freedman, “On the Bernstein-von Mises theorem with infinite-dimensional parameters,” Ann. Stat. 27(4), 1119–1140 (1999).MATHGoogle Scholar
  14. 14.
    S. Ghosal, “Asymptotic normality of posterior distributions in high-dimensional linear models,” Bernoulli 5(2), 315–331 (1999).CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    S. Ghosal, “Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity,” J. Multivariate Anal. 74(1), 49–68 (2000).CrossRefMATHMathSciNetGoogle Scholar
  16. 16.
    I. M. Johnstone, “High dimensional Bernstein-von Mises: Simple examples,” in Borrowing Strength: Theory Powering Applications-A Festschrift for Lawrence D. Brown (Inst. Math. Stat., Beachwood, OH, 2010), Inst. Math. Stat. Collect. 6, pp. 87–98.Google Scholar
  17. 17.
    Y. Kim, “The Bernstein-von Mises theorem for the proportional hazard model,” Ann. Stat. 34(4), 1678–1700 (2006).CrossRefMATHGoogle Scholar
  18. 18.
    Y. Kim and J. Lee, “A Bernstein-von Mises theorem in the nonparametric right-censoring model,” Ann. Stat. 32(4), 1492–1512 (2004).CrossRefMATHGoogle Scholar
  19. 19.
    B. J. K. Kleijn and A. W. van der Vaart, “Misspecification in infinite-dimensional Bayesian statistics,” Ann. Stat. 34(2), 837–877 (2006).CrossRefMATHGoogle Scholar
  20. 20.
    B. J. K. Kleijn and A. W. van der Vaart, “The Bernstein-von Mises theorem under misspecification,” Electron. J. Stat. 6, 354–381 (2012).CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    L. Le Cam and G. L. Yang, Asymptotics in Statistics: Some Basic Concepts (Springer, New York, 1990), Springer Ser. Stat.CrossRefMATHGoogle Scholar
  22. 22.
    H. Leahu, “On the Bernstein-von Mises phenomenon in the Gaussian white noise model,” Electron. J. Stat. 5, 373–404 (2011).CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    J. Polzehl and V. Spokoiny, “Propagation-separation approach for local likelihood estimation,” Probab. Theory Relat. Fields 135(3), 335–362 (2006).CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    V. Rivoirard and J. Rousseau, “Bernstein-von Mises theorem for linear functionals of the density,” Ann. Stat. 40(3), 1489–1523 (2012).CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    L. Schwartz, “On Bayes procedures,” Z. Wahrscheinlichkeitstheor. Verw. Geb. 4(1), 10–26 (1965).CrossRefMATHGoogle Scholar
  26. 26.
    X. Shen, “Asymptotic normality of semiparametric and nonparametric posterior distributions,” J. Am. Stat. Assoc. 97(457), 222–235 (2002).CrossRefMATHGoogle Scholar
  27. 27.
    V. Spokoiny, “Parametric estimation. Finite sample theory,” Ann. Stat. 40(6), 2877–2909 (2012); arXiv: 1111.3029 [math.ST].CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    V. Spokoiny and M. Zhilova, “Sharp deviation bounds for quadratic forms,” Math. Methods Stat. 22(2), 100–113 (2013); arXiv: 1302.1699 [math.PR].CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    A. W. van der Vaart, Asymptotic Statistics (Cambridge Univ. Press, Cambridge, 2000), Cambridge Ser. Stat. Probab. Math. 3.Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2014

Authors and Affiliations

  1. 1.Moscow Institute of Physics and Technology (State University)Dolgoprudnyi, Moscow oblastRussia
  2. 2.Institute for Information Transmission Problems (Kharkevich Institute)Russian Academy of SciencesMoscowRussia
  3. 3.Datadvance CompanyMoscowRussia
  4. 4.Weierstrass Institute for Applied Analysis and StochasticsBerlinGermany
  5. 5.Humboldt-Universität zu BerlinBerlinGermany

Personalised recommendations