Statistics and Computing

, Volume 11, Issue 4, pp 373–394 | Cite as

Adaptive wavelet series estimation in separable nonparametric regression models

  • Umberto Amato
  • Anestis Antoniadis


It is well-known that multivariate curve estimation suffers from the “curse of dimensionality.” However, reasonable estimators are possible, even in several dimensions, under appropriate restrictions on the complexity of the curve. In the present paper we explore how much appropriate wavelet estimators can exploit a typical restriction on the curve such as additivity. We first propose an adaptive and simultaneous estimation procedure for all additive components in additive regression models and discuss rate of convergence results and data-dependent truncation rules for wavelet series estimators. To speed up computation we then introduce a wavelet version of functional ANOVA algorithm for additive regression models and propose a regularization algorithm which guarantees an adaptive solution to the multivariate estimation problem. Some simulations indicate that wavelets methods complement nicely the existing methodology for nonparametric multivariate curve estimation.

wavelet series adaptive estimation additive model wavelet regularization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amato U. and Vuza, D.T. 1997. Wavelet approximation of a function from samples affected by noise. Revue Roumaine de Mathéematiques Pures et Appliquées 42: 481–493.Google Scholar
  2. Andrews D.W.K. 1991. Asymptotic optimality of generalizedC L, crossvalidation, and generalized cross-validation in regression with heteroskedastic errors. Journal of Econometrics 47: 359–377.Google Scholar
  3. Andrews D.W.K. 1991. Asymptotic normality of series estimators for nonparametric and semiparametric regression models. Econometrica 59: 307–345.Google Scholar
  4. Andrews D.W.K. and Whang Y.J. 1990. Additive interactive regression models: Circumvention of the curse of dimensionality. Econometric Theory 6: 466–479.Google Scholar
  5. Antoniadis A. 1996. Smoothing noisy data with tapered coiflet series. Scandinavian Journal of Statistics 23: 313–330.Google Scholar
  6. Antoniadis A. and Pham D.T. 1998. Wavelet regression for random or irregular design. Computational and Statistical Data Analysis 28: 353–370.Google Scholar
  7. Beylkin G., Coifmann R., and Rokhlin V. 1991. The fast wavelet transform and numerical algorithms. Communications in Pure and Applied Mathematics 44: 141–163.Google Scholar
  8. Breiman L. and Friedman J.H. 1985. Estimating optimal transformations for multiple regression and correlation (with discussion). Journal of the American Statistical Association 80: 580–619.Google Scholar
  9. Buja A., Hastie T.J., and Tibshirani R.J. 1989. Linear smoothers and additive models. Annals of Statistics 17: 453–510.Google Scholar
  10. Cohen A., Daubechies I., and Vial P. 1993. Wavelets on the interval and fast wavelet transforms. Applied and Computational Harmonic Analysis 1: 54–82.Google Scholar
  11. Craven P. and Wahba G. 1979. Smoothing noisy data with spline functions. Numerische Mathematik 59: 377–403.Google Scholar
  12. Daubechies I. 1992. Ten Lectures on Wavelets, SIAM, Philadelphia.Google Scholar
  13. Delyon B. and Ioudistski A. 1995. Estimating wavelet coefficients. In: Antoniadis A. and Oppenheim, G. (Eds.). Wavelets and Statistics, Lecture Notes in Statistics, 103. Springer-Verlag, Berlin, pp. 151–168.Google Scholar
  14. Fan J. and Gijbels I. 1995. Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. Journal of the Royal Statistical Society Series B 57: 371–394.Google Scholar
  15. Hastie T.J. and Tibshirani R.J. 1990. Generalized Additive Models, Chapman & Hall, London.Google Scholar
  16. Linton O. 1996. Efficient estimation of additive nonparametric regression models. Biometrika 84: 469–473.Google Scholar
  17. Linton O. and Nielsen J.P. 1995. A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82: 93–100.Google Scholar
  18. Mallat S.G. 1989. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattn. Anal. Mach. Intell. 11: 674–693.Google Scholar
  19. Meyer Y. 1990. Ondelettes Et Op´erateurs, Vol. I. Hermann, Paris.Google Scholar
  20. Muller H.G. and Stadtmuller U. 1987. Estimation of heteroscedasticity in regression analysis. Annals of Statistics 15: 610–625.Google Scholar
  21. Newey W.K. 1994. Kernel estimation of partial means. Econometric Theory 10: 233–253.Google Scholar
  22. Nychka D., Bailey B., Ellner S., Haaland P., and O'Connel P. 1993. FUNFITS: Data analysis and statistical tools for estimating functions. Raleigh, North Carolina State University. Raleigh. Technical report.Google Scholar
  23. Opsomer J.D. 1995. Optimal Bandwidth Selection for Fitting an Additive Model by Local Polynomial Regression. Cornell Univ. Ph.D. Dissertation.Google Scholar
  24. Opsomer J.D. and Ruppert D. 1997. Fitting a bivariate additive model by local polynomial regression. Annals of Statistics 25: 186–211.Google Scholar
  25. Opsomer J.D. and Ruppert D. 1998. A fully automated bandwidth selection method for fitting additive models, Journal of the American Statistical Association 93: 605–619.Google Scholar
  26. Sperlich S., Linton O.B. and Härdle W. 1997. A simulation comparison between integration and backfitting methods of estimating separable nonparametric regression models. Univ. Berlin. Humboldt. Technical Report.Google Scholar
  27. Sperlich S., Tjøstheim D., and Yang L. 1998. Nonparametric estimation and testing of interaction additive models. Univ. Berlin. Humboldt. Technical report.Google Scholar
  28. Stone C.J. 1985. Additive regression and other nonparametric models. Annals of Statistics 13: 689–705.Google Scholar
  29. Stone C.J. 1986. The dimensionality reduction principle for generalized additive models. Annals of Statistics 14: 590–606.Google Scholar
  30. Tjøstheim D. and Auestad B. 1994. Nonparametric identification of nonlinear time series: Projections. Journal of the American Statistical Association 89: 1398–1409.Google Scholar
  31. Venables W.N. and Ripley B. 1994. Modern Applied Statistics with S-Plus. Springer Verlag, New York.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Umberto Amato
    • 1
  • Anestis Antoniadis
    • 2
  1. 1.Istituto per Applicazioni della Matematica CNRNapoliItaly
  2. 2.Laboratoire LMC—IMAGUniversité Joseph FourierGrenoble Cedex 09France

Personalised recommendations