Abstract
In the setting of high-dimensional linear models with Gaussian noise, we investigate the possibility of confidence statements connected to model selection. Although there exist numerous procedures for adaptive (point) estimation, the construction of adaptive confidence regions is severely limited (cf. Li in Ann Stat 17:1001–1008, 1989). The present paper sheds new light on this gap. We develop exact and adaptive confidence regions for the best approximating model in terms of risk. One of our constructions is based on a multiscale procedure and a particular coupling argument. Utilizing exponential inequalities for noncentral χ 2-distributions, we show that the risk and quadratic loss of all models within our confidence region are uniformly bounded by the minimal risk times a factor close to one.
Article PDF
Similar content being viewed by others
References
Baraud Y.: Confidence balls in Gaussian regression. Ann. Stat. 32, 528–551 (2004)
Beran R.: Confidence sets centered at C p estimators. Ann. Inst. Stat. Math. 48, 1–15 (1996)
Beran R.: REACT scatterplot smoothers: superefficiency through basis economy. J. Am. Stat. Assoc. 95, 155–169 (2000)
Beran R., Dümbgen L.: Modulation of estimators and confidence sets. Ann. Stat. 26, 1826–1856 (1998)
Birgé L., Massart P.: Gaussian model selection. J. Eur. Math. Soc. 3, 203–268 (2001)
Cai T.T.: Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Stat. 26, 1783–1799 (1999)
Cai T.T.: On block thresholding in wavelet regression: adaptivity, block size, and threshold level. Stat. Sin. 12, 1241–1273 (2002)
Cai T.T., Low M.G.: Adaptive confidence balls. Ann. Stat. 34, 202–228 (2006)
Cai, T.T., Low, M.G.: Adaptive estimation and confidence intervals for convex functions and monotone functions (2007) (manuscript in preparation)
Dahlhaus R., Polonik W.: Nonparametric quasi-maximum likelihood estimation for Gaussian locally stationary processes. Ann. Stat. 34, 2790–2824 (2006)
Donoho D.L., Johnstone I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425–455 (1994)
Donoho D.L., Johnstone I.M.: Adapting to unknown smoothness via wavelet shrinkage. JASA 90, 1200–1224 (1995)
Donoho D.L., Johnstone I.M.: Minimax estimation via wavelet shrinkage. Ann. Stat. 26, 879–921 (1998)
Dümbgen L.: Optimal confidence bands for shape-restricted curves. Bernoulli 9, 423–449 (2003)
Dümbgen L., Spokoiny V.G.: Multiscale testing of qualitative hypotheses. Ann. Stat. 29, 124–152 (2001)
Dümbgen, L., Walther, G.: Multiscale inference about a density. Technical report 56, IMSV, University of Bern (2007)
Efromovich S.: Simultaneous sharp estimation of functions and their derivatives. Ann. Stat. 26, 273–278 (1998)
Genovese C.R., Wassermann L.: Confidence sets for nonparametric wavelet regression. Ann. Stat. 33, 698–729 (2005)
Giné E., Nickl R.: Confidence bands in density estimation. Ann. Stat. 38, 1122–1170 (2010)
Hengartner N.W., Stark P.B.: Finite-sample confidence envelopes for shape-restricted densities. Ann. Stat. 23, 525–550 (1995)
Hoffmann, M., Nickl, R.: On adaptive inference and confidence bands. Ann. Stat. 39, 2383–2409 (2011)
Juditsky A., Lambert-Lacroix S.: Nonparametric confidence set estimation. Math. Methods Stat. 19, 410–428 (2003)
Lepski O.V., Mammen E., Spokoiny V.G.: Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors. Ann. Stat. 25, 929–947 (1997)
Li K.-C.: Honest confidence regions for nonparametric regression. Ann. Stat. 17, 1001–1008 (1989)
Polyak B.T., Tsybakov A.B.: Asymptotic optimality of the C p -test for the orthogonal series estimation of regression. Theory Prob. Appl. 35, 293–306 (1991)
Robins J., van der Vaart A.: Adaptive nonparametric confidence sets. Ann. Stat. 34, 229–253 (2006)
Rohde, A., Dümbgen, L.: Adaptive confidence sets for the optimal approximating model. Technical report 73, IMSV, Univ. of Bern (2009)
Stone C.J.: An asymptotically optimal window selection rule for kernel density estimates. Ann. Stat. 12, 1285–1297 (1984)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rohde, A., Dümbgen, L. Statistical inference for the optimal approximating model. Probab. Theory Relat. Fields 155, 839–865 (2013). https://doi.org/10.1007/s00440-012-0414-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-012-0414-7
Keywords
- Adaptivity
- Confidence regions
- Coupling
- Exponential inequality
- Model selection
- Multiscale inference
- Risk optimality