Constructive Approximation

, Volume 31, Issue 2, pp 195–229 | Cite as

Maxisets for Model Selection

  • F. Autin
  • E. Le Pennec
  • J. M. Loubes
  • V. Rivoirard
Article

Abstract

We address the statistical issue of determining the maximal spaces (maxisets) where model selection procedures attain a given rate of convergence. By considering first general dictionaries, then orthonormal bases, we characterize these maxisets in terms of approximation spaces. These results are illustrated by classical choices of wavelet model collections. For each of them, the maxisets are described in terms of functional spaces. We give special attention to the issue of calculability and measure the induced loss of performance in terms of maxisets.

Keywords

Approximation spaces Approximation theory Besov spaces Estimation Maxiset Model selection Rates of convergence 

Mathematics Subject Classification (2000)

62G05 62G20 41A25 42C40 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Second International Symposium on Information Theory, Tsahkadsor, 1971, pp. 267–281. Akadémiai Kiadó, Budapest (1973) Google Scholar
  2. 2.
    Autin, F.: Maxiset for density estimation on ℝ. Math. Methods Stat. 15(2), 123–145 (2006) MathSciNetGoogle Scholar
  3. 3.
    Autin, F.: Maxisets for μ-thresholding rules. Test 17(2), 332–349 (2008) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Autin, F., Picard, D., Rivoirard, V.: Large variance Gaussian priors in Bayesian nonparametric estimation: a maxiset approach. Math. Methods Stat. 15(4), 349–373 (2006) MathSciNetGoogle Scholar
  5. 5.
    Baraud, Y.: Model selection for regression on a fixed design. Probab. Theory Relat. Fields 117(4), 467–493 (2000) MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Baraud, Y.: Model selection for regression on a random design. ESAIM Probab. Stat. 6, 127–146 (2002) MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Barron, A., Birgé, L., Massart, P.: Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113(3), 301–413 (1999) MATHCrossRefGoogle Scholar
  8. 8.
    Bertin, K., Rivoirard, V.: Maxiset in sup-norm for kernel estimators. Test (2009). Doi:10.1007/s11749-008-0109-7
  9. 9.
    Birgé, L., Massart, P.: An adaptive compression algorithm in Besov spaces. Constr. Approx. 16(1), 1–36 (2000) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Birgé, L., Massart, P.: Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3(3), 203–268 (2001) MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Birgé, L., Massart, P.: Minimal penalties for Gaussian model selection. Probab. Theory Relat. Fields 138(1–2), 33–73 (2007) MATHCrossRefGoogle Scholar
  12. 12.
    Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification: a survey of some recent advances. ESAIM Probab. Stat. 9, 323–375 (2005) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Cohen, A., DeVore, R.A., Kerkyacharian, G., Picard, D.: Maximal spaces with given rate of convergence for thresholding algorithms. Appl. Comput. Harmon. Anal. 11(2), 167–191 (2001) MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992) MATHGoogle Scholar
  15. 15.
    Kerkyacharian, G., Picard, D.: Thresholding algorithms, maxisets and well-concentrated bases. Test 9(2), 283–344 (2000) MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Loubes, J.-M., Ludeña, C.: Adaptive complexity regularization for linear inverse problems. Electron. J. Stat. 2, 661–677 (2008) CrossRefMathSciNetGoogle Scholar
  17. 17.
    Loubes, J.-M., Ludeña, C.: Penalized estimators for non linear inverse problems. ESAIM Probab. Stat. (2008). DOI:10.1051/ps:2008024
  18. 18.
    Mallows, C.L.: Some comments on C p. Technometrics 15, 661–675 (1973) MATHCrossRefGoogle Scholar
  19. 19.
    Massart, P.: Concentration inequalities and model selection. In: Lectures on Probability Theory and Statistics, Saint-Flour, 2003. Lecture Notes in Math., vol. 1896. Springer, Berlin (2007) Google Scholar
  20. 20.
    Meyer, Y.: Ondelettes et opérateurs. I. Hermann, Paris (1990) MATHGoogle Scholar
  21. 21.
    Rivoirard, V.: Maxisets for linear procedures. Stat. Probab. Lett. 67(3), 267–275 (2004) MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Rivoirard, V.: Bayesian modeling of sparse sequences and maxisets for Bayes rules. Math. Methods Stat. 14(3), 346–376 (2005) MathSciNetGoogle Scholar
  23. 23.
    Rivoirard, V., Tribouley, K.: The maxiset point of view for estimating integrated quadratic functionals. Stat. Sin. 18(1), 255–279 (2008) MATHMathSciNetGoogle Scholar
  24. 24.
    Nussbaum, M.: Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Stat. 24(6), 2399–2430 (1996) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • F. Autin
    • 1
  • E. Le Pennec
    • 2
  • J. M. Loubes
    • 3
  • V. Rivoirard
    • 4
    • 5
  1. 1.Centre de Mathématiques et d’InformatiqueMarseille Cedex 13France
  2. 2.Laboratoire de Probabilités et Modèles Aléatoires, UMR 7599Université Paris DiderotParisFrance
  3. 3.Institut de Mathématiques de Toulouse, Equipe de Probabilités et de StatistiqueUniversité de Toulouse Paul SabatierToulouseFrance
  4. 4.Laboratoire de Mathématiques, UMR 8628Université Paris-Sud.Orsay cedexFrance
  5. 5.Département de Mathématiques et Applications, UMR 8553Ecole Normale SupérieureParis Cedex 05France

Personalised recommendations