Advertisement

Efficiency of the V -Fold Model Selection for Localized Bases

  • F. NavarroEmail author
  • A. Saumard
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 250)

Abstract

Many interesting functional bases, such as piecewise polynomials or wavelets, are examples of localized bases. We investigate the optimality of V -fold cross-validation and a variant called V -fold penalization in the context of the selection of linear models generated by localized bases in a heteroscedastic framework. It appears that while V -fold cross-validation is not asymptotically optimal when V is fixed, the V -fold penalization procedure is optimal. Simulation studies are also presented.

References

  1. 1.
    Antoniadis, A., Gregoire, G., & McKeague, I. (1994). Wavelet methods for curve estimation. Journal of the American Statistical Association, 89(428), 1340–1353.Google Scholar
  2. 2.
    Arlot, S. (2008). V -fold cross-validation improved: V -fold penalization. ArXiv:0802.0566v2. http://hal.archives-ouvertes.fr/hal-00239182/en/
  3. 3.
    Arlot, S., & Célisse, A. (2010). A survey of cross-validation procedures for model selection. Statistical Surveys, 4, 40–79.Google Scholar
  4. 4.
    Arlot, S., & Massart, P. (2009) Data-driven calibration of penalties for least-squares regression. Journal of Machine Learning Research, 10, 245–279 (electronic).Google Scholar
  5. 5.
    Barron, A., Birgé, L., & Massart, P. (1999). Risk bounds for model selection via penalization. Probability Theory and Related Fields, 113(3), 301–413.Google Scholar
  6. 6.
    Birgé, L., & Massart, P. (1998). Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli, 4(3), 329–375.Google Scholar
  7. 7.
    Cai, T. (1999). Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Annals of Statistics, 27(3), 898–924.Google Scholar
  8. 8.
    Cai, T., & Brown, L. (1998). Wavelet shrinkage for nonequispaced samples. Annals of Statistics, 26, 1783–1799.Google Scholar
  9. 9.
    Cai, T., & Brown, L. (1999). Wavelet estimation for samples with random uniform design. Statistics and Probability Letters, 42(3), 313–321.Google Scholar
  10. 10.
    Donoho D., Maleki, A., & Shahram, M. (2006). Wavelab 850. http://statweb.stanford.edu/~wavelab/
  11. 11.
    Hall, P., & Turlach, B. (1997). Interpolation methods for nonlinear wavelet regression with irregularly spaced design. Annals of Statistics, 25(5), 1912–1925.Google Scholar
  12. 12.
    Kulik, R., & Raimondo, M. (2009). Wavelet regression in random design with heteroscedastic dependent errors. Annals of Statistics, 37(6A), 3396–3430.Google Scholar
  13. 13.
    Mallat, S. (2008). A wavelet tour of signal processing: The sparse way. New York: Academic.Google Scholar
  14. 14.
    Marron, J., Adak, S., Johnstone, I., Neumann, M., & Patil, P. (1998). Exact risk analysis of wavelet regression. Journal of Computational and Graphical Statistics, 7(3), 278–309.Google Scholar
  15. 15.
    Nason, G. (1996). Wavelet shrinkage using cross-validation. Journal of the Royal Statistical Society, Series B, 58, 463–479.Google Scholar
  16. 16.
    Navarro, F., & Saumard, A. (2017). Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases. ESAIM: Probability and Statistics, 21, 412–451.Google Scholar
  17. 17.
    Saumard, A. (2012). Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression. Electronic Journal of Statistics, 6(1–2):579–655.Google Scholar
  18. 18.
    Saumard, A. (2013). Optimal model selection in heteroscedastic regression using piecewise polynomial functions. Electronic Journal of Statistics, 7, 1184–1223.Google Scholar
  19. 19.
    Saumard, A. (2017). On optimality of empirical risk minimization in linear aggregation. Bernoulli (to appear). arXiv:1605.03433Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.CREST-ENSAI-UBLBruzFrance

Personalised recommendations