Abstract
We apply the Bayes approach to the problem of projection estimation of a signal observed in the Gaussian white noise model and we study the rate at which the posterior distribution concentrates about the true signal from the space ℓ2 as the information in observations tends to infinity. A benchmark is the rate of a so-called oracle projection risk, i.e., the smallest risk of an unknown true signal over all projection estimators. Under an appropriate hierarchical prior, we study the performance of the resulting (appropriately adjusted by the empirical Bayes approach) posterior distribution and establish that the posterior concentrates about the true signal with the oracle projection convergence rate. We also construct a Bayes estimator based on the posterior and show that it satisfies an oracle inequality. The results are nonasymptotic and uniform over ℓ2. Another important feature of our approach is that our results on the oracle projection posterior rate are always stronger than any result about posterior convergence with the minimax rate over all nonparametric classes for which the corresponding projection oracle estimator is minimax over this class. We also study implications for the model selection problem, namely, we propose a Bayes model selector and assess its quality in terms of the so-called false selection probability.
Similar content being viewed by others
References
E. Belitser and F. Enikeeva, “Empirical Bayesian Test for the Smoothness”, Math. Methods Statist. 17, 1–18 (2008).
E. Belitser and S. Ghosal, “Adaptive Bayesian Inference on the Mean of an Infinite-Dimensional Normal Distribution”, Ann. Statist. 31, 536–559 (2003).
E. Belitser and B. Levit, “On Minimax Filtering over Ellipsoids”, Math. Methods Statist. 3, 259–273 (1995).
L. Birgé and P. Massart, “Gaussian Model Selection”, J. Eur. Math. Soc. 3, 203–268 (2001).
L. D. Brown and M. G. Low, “Asymptotic Equivalence of Nonparametric Regression and White Noise”, Ann. Statist. 24, 2384–2398 (1995).
L. Cavalier, G. K. Golubev, D. Picard, and A. B. Tsybakov, “Oracle Inequalities for Inverse Problems”, Ann. Statist. 30, 843–874 (2002).
L. Cavalier and A. Tsybakov, “Penalized Blockwise Stein’s Method, Monotone Oracles and Sharp Adaptive Estimation”, Math. Methods Statist. 10, 247–282 (2001).
S. Efromovich and M. Pinsker, “A Learning Algorithm for Nonparametric Filtering”, Automat. Remote Control. 24, 1434–1440 (1984).
S. Ghosal, J. K. Ghosh and A. W. van der Vaart, “Convergence Rates of Posterior Distributions”, Ann. Statist. 28, 500–531 (2000).
G. K. Golubev, “On a Method for Minimizing Empirical Risk”, Problems Inform. Transmission. 40, 202–211 (2004).
Y. Golubev and B. Levit, “An Oracle Approach to Adaptive Estimation of Linear Functionals in a Gaussian Model”, Math. Methods Statist. 13, 392–408 (2004).
M. Hoffmann and O. Lepski, “Random Rates in Anisotropic Regression”, Ann. Statist. 28, 325–396 (2002).
I. A. Ibragimov and R. Z. Khasminski, Statistical Estimation: Asymptotic Theory (Springer, New York, 1981).
I. Johnstone, Function Estimation in Gaussian Noise: Sequence models (Monograph draft, 1999), http://www-stat.stanford.edu/~imj/.
A. Kneip, “Ordered Linear Smoothers”, Ann. Statist. 22, 835–866 (1994).
M. Pinsker, “Optimal Filtration of Square-Integrable Signal in Gaussian White Noise”, Problems Inform. Transmission. 16, 120–133 (1980).
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Babenko, A., Belitser, E. Oracle convergence rate of posterior under projection prior and Bayesian model selection. Math. Meth. Stat. 19, 219–245 (2010). https://doi.org/10.3103/S1066530710030026
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S1066530710030026
Key words
- Bayes approach
- Bayes model selector
- false selection probability
- oracle projection posterior rate
- posterior-randomized estimator