Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
  1. Home
  2. Probability Theory and Related Fields
  3. Article
Model selection and sharp asymptotic minimaxity
Download PDF
Download PDF
  • Published: 17 March 2012

Model selection and sharp asymptotic minimaxity

  • Zheyang Wu1 &
  • Harrison H. Zhou2 

Probability Theory and Related Fields volume 156, pages 165–191 (2013)Cite this article

  • 331 Accesses

  • 8 Citations

  • Metrics details

Abstract

We obtain sharp minimax results for estimation of an n-dimensional normal mean under quadratic loss. The estimators are chosen by penalized least squares with a penalty that grows like ck log(n/k), for k equal to the number of nonzero elements in the estimating vector. For a wide range of sparse parameter spaces, we show that the penalized estimator achieves the exact minimax rate with the correct multiplication constant if and only if c equals 2. Our results unify the theory obtained by many other authors for penalized estimation of normal means. In particular we establish that a conjecture by Abramovich et al. (Ann Stat 34:584–653, 2006) is true.

Download to read the full article text

Working on a manuscript?

Avoid the common mistakes

References

  1. Abramovich F., Benjamini Y., Donoho D.L., Johnstone I.M.: Adapting to unknown sparsity by controlling the false discovery rate. Ann. Stat. 34, 584–653 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Abramovich F., Grinshtein V., Pensky M.: On optimality of Bayesian testimation in the normal means problem. Ann. Stat. 35, 2261–2286 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Akaike H.: Information theory and an extension of the maximum likelihood principle. In: Petrov, B.N., Czáki, F. (eds) Second International Symposium on Information Theory, pp. 267–281. Akadémiai Kiadó, Budapest (1973)

    Google Scholar 

  4. Akaike H.: A new look at the statistical model identification. IEEE Trans. Automa. Control. 19(6), 716–723 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  5. Barron A.R., Birgé L., Massart P.: Risk bounds for model selection via penalization. Probab. Theory. Relat. Fields 113, 301–413 (1999)

    Article  MATH  Google Scholar 

  6. Benjamini Y., Hochberg Y.: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. B. 57, 289–300 (1995)

    MathSciNet  MATH  Google Scholar 

  7. Benjamini Y., Gavrilov Y.: A simple forward selection procedure based on false discovery rate control. Ann. Appl. Stat. 3, 1, 179–198 (2009)

    Article  MathSciNet  Google Scholar 

  8. Birgé l., Massart P.: Gaussian model selection. J. Eur. Math. Soc. 3, 203–268 (2001)

    Article  MATH  Google Scholar 

  9. Birgé L., Massart P.: Minimal penalties for Gaussian model selection. Probab. Theory. Relat. Fields 138, 33–73 (2007)

    Article  MATH  Google Scholar 

  10. Brown L.D., Low M.G.: Asymptotic equivalence of nonparametric regression and white noise. Ann. Stat. 24, 2384–2398 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  11. Brown L.D., Cai T.T., Zhou H.H.: Robust nonparametric estimation via wavelet median regression. Ann. Stat. 36, 2055–2084 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  12. Brown L.D., Cai T., Zhou H.H.: Nonparametric regression in exponential families. Ann. Stat. 38, 2005–2046 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Cai T.T., Zhou H.H.: Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Ann. Stat. 37, 3204–3235 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Cörgő S., Mason D.: Central limit theorems for sums of extreme values. Math. Proc. Camb. 98, 547– (1985)

    Article  Google Scholar 

  15. Donoho D.L., Johnstone I.M., Hoch J.C., Stern A.S.: Maximum entropy and the nearly black object. J. R. Stat. Soc. B. 54(1), 41–81 (1992)

    MathSciNet  MATH  Google Scholar 

  16. Donoho D.L., Johnstone I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  17. Donoho D.L., Johnstone I.M.: Minimax Risk Over l p -Balls for l q -Error. Probab. Theory. Relat. Fields 99, 277–303 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  18. Donoho D.L., Johnstone I.M.: Adapting to unknown smoothness via wavelet Shrinkage. J. Am. Stat. Assoc. 90, 1200–1224 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  19. Donoho D.L., Johnstone I.M.: Minimax estimation via wavelet shrinkage. Ann. Stat. 26, 879–921 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  20. David H.A., Nagaraja H.N.: Order Statistics, 3rd edn. Wiley & Sons, New York (2003)

    Book  MATH  Google Scholar 

  21. Efron B.: Robbins, empirical Bayes and microarrays. Ann. Stat. 31, 366–378 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  22. Foster D.P., George E.I.: The risk inflation criterion for multiple regression. Ann. Stat. 22, 1947–1975 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  23. Foster D.P., Stine R.A.: Local asymptotic coding and the minimum description length. IEEE Trans. Inf. Theory 45, 1289–1293 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  24. George E.I., Foster D.P.: Calibration and empirical Bayes variable selection. Biometrika 87, 731–747 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  25. Golubev G.K., Nussbaum M., Zhou H.H.: Asymptotic equivalence of spectral density estimation and Gaussian white noise. Ann. Stat. 38, 181–214 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Johnstone I.M.: Minimax Bayes asymptotic minimax and sparse wavelet priors. In: Gupta, S., Berger, J. (eds) Statistical Decision Theory and Related Topics V, pp. 303–326. Springer, Berlin (1994)

    Chapter  Google Scholar 

  27. Johnstone, I.M.: Gaussian Estimation: Sequence and Multiresolution Models. Unpublished manuscript. http://www-stat.stanford.edu/~imj/ (2011)

  28. Nussbaum M.: Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Stat. 24, 2399–2430 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  29. Tibshirani R., Knight K.: The covariance inflation criterion for adaptive model selection. J. R. Stat. Soc. B. 61, 529–546 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  30. Yang Y., Barron A.R.: An asymptotic property of model selection criteria. IEEE Trans. Inf. Theory 44, 117–133 (1998)

    Article  MathSciNet  Google Scholar 

  31. Yang Y.: Model selection for nonparametric regression. Stat. Sinica. 9, 475–499 (1999)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Worcester Polytechnic Institute, Worcester, MA, 01609, USA

    Zheyang Wu

  2. Department of Statistics, Yale University, New Haven, CT, 06511, USA

    Harrison H. Zhou

Authors
  1. Zheyang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Harrison H. Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Harrison H. Zhou.

Additional information

The research of Z. Wu was supported in part by NIH Grant GM590507. The research of H. Zhou was supported in part by NSF Grants DMS-0645676 and DMS-0854975.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wu, Z., Zhou, H.H. Model selection and sharp asymptotic minimaxity. Probab. Theory Relat. Fields 156, 165–191 (2013). https://doi.org/10.1007/s00440-012-0424-5

Download citation

  • Received: 12 March 2010

  • Accepted: 13 December 2011

  • Published: 17 March 2012

  • Issue Date: June 2013

  • DOI: https://doi.org/10.1007/s00440-012-0424-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • FDR
  • Minimax estimation
  • Model selection
  • Multiple comparisons
  • Sharp asymptotic minimaxity
  • Smoothing parameter selection
  • Thresholding
  • Wavelet denoising
  • Wavelets

Mathematics Subject Classification

  • Primary 62G08
  • Secondary 62G20
Download PDF

Working on a manuscript?

Avoid the common mistakes

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature