Skip to main content
SpringerLink
Log in

A new algorithm for fixed design regression and denoising

  • Regression
  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

In this paper, we present a new algorithm to estimate a regression function in a fixed design regression model, by piecewise (standard and trigonometric) polynomials computed with an automatic choice of the knots of the subdivision and of the degrees of the polynomials on each sub-interval. First we give the theoretical background underlying the method: the theoretical performances of our penalized least-squares estimator are based on non-asymptotic evaluations of a mean-square type risk. Then we explain how the algorithm is built and possibly accelerated (to face the case when the number of observations is great), how the penalty term is chosen and why it contains some constants requiring an empirical calibration. Lastly, a comparison with some well-known or recent wavelet methods is made: this brings out that our algorithm behaves in a very competitive way in term of denoising and of compression.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abramowitz, A. and Stegun, I. A. (1972).Handbook of Mathematical Functions, Dover Publications, New York.

    MATH  Google Scholar 

  • Antoniadis, A. and Fan, J. (2001). Regularization of wavelets approximations (with discussion),Journal of the American Statistical Association,96, 939–967.

    Article  MathSciNet  Google Scholar 

  • Antoniadis, A. and Pham, D. T. (1998). Wavelet regression for random or irregular design,Computational Statistics and Data Analysis,28, 353–369.

    Article  MathSciNet  Google Scholar 

  • Antoniadis, A., Bigot, J. and Sapatinas, T. (2002). Wavelet estimators in nonparametric regression: A comparative simulation study,Journal of Statistical Software,6 (see http://www.jstatsoft.org/v06/i06).

  • Baraud, Y. (2000). Model selection for regression on a fixed design,Probability Theory and Related Fields,117, 467–493.

    Article  MathSciNet  Google Scholar 

  • Baraud, Y. (2002). Model selection for regression on a random design,ESAIM Probability and Statistics,6, 127–146.

    Article  MathSciNet  Google Scholar 

  • Baraud, Y., Comte, F. and Viennet, G. (2001a). Adaptive estimation in an autoregressive and a geometrical β-mixing regression framework,The Annals of Statistics,39, 839–875.

    MathSciNet  Google Scholar 

  • Baraud, Y., Comte, F. and Viennet, G. (2001b). Model Selection for (auto-)regression with dependent data,ESAIM Probability and Statistics,5, 33–49.

    Article  MathSciNet  Google Scholar 

  • Barron, A. and Cover, T. M. (1991). Minimum complexity density estimation,IEEE Transactions on Information Theory,37, 1037–1054.

    Article  Google Scholar 

  • Barron, A., Birgé, L. and Massart, P. (1999). Risks bounds for model selection via penalization,Probability Theory and Related Fields,113, 301–413.

    Article  MathSciNet  Google Scholar 

  • Birgé, L. and Massart, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence,Bernoulli,4, 329–375.

    Article  MathSciNet  Google Scholar 

  • Birgé, L. and Massart, P. (2001). Gaussian model selection,Journal of the European Mathematical Society,3, 203–268.

    Article  Google Scholar 

  • Birgé, L. and Rozenholc, Y. (2002). How many bins should be put in a regular histogram. Preprint du LPMA 721, http://www.proba.jussieu.fr/mathdoc/preprints/index.html.

  • Breimann, L., Friedman, J. H., Olshen, R. H. and Stone, C. J. (1984).Classification and Regression Trees, Wadsworth, Belmont, California.

    Google Scholar 

  • Buckheit, J. B., Chen, S., Donoho, D. L., Johnstone, I. M. and Scargle, J. (1995). About WaveLab, Tech. Report, Department of Statistics, Stanford University, Stanford, California. availablehttp://www-stat.stanford.edu/wavelab

    Google Scholar 

  • Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach,The Annals of Statistics,27, 898–924.

    Article  MathSciNet  Google Scholar 

  • Cai, T. T. and Silverman, B. W. (2001). Incorporating information on neighboring coefficients into wavelet estimation,Sankhya, Series B,63, 127–148.

    MathSciNet  Google Scholar 

  • Castellan, G. (2000). Sélection d’histogrammes à l’aide d’un critère de type Akaike (Histograms selection with an Akaike type criterion),Comptes Rendus de l’Académie des Sciences. Paris. Série I. Mathematique,330, 729–732.

    Article  MathSciNet  Google Scholar 

  • Coifman, R. R. and Donoho, D. L. (1995). Translation-invariant de-noising (eds. Antoniadis, A. and Oppenheim, G.),Wavelets and Statistics, Lecture Notes in Statistics,103, 125–150, Springer-Verlag, New York.

    Google Scholar 

  • Comte, F. and Rozenholc, Y. (2002). Adaptive estimation of mean and volatility functions in (auto)-regressive models,Stochastic Processes and Their Applications,97, 111–145.

    Article  MathSciNet  Google Scholar 

  • Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998). Automatic Bayesian curve fitting,Journal of the Royal Statistical Society Series B,60, 333–350.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L. (1995). Denoising by soft-thresholding,IEEE Transactions on Information Theory,41, 613–627.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal space adaptation by wavelet shrinkage,Biometrika,81, 425–455.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia (with discussion),Journal of the Royal Statistical Society Series B,57, 371–394.

    MathSciNet  Google Scholar 

  • Efromovich, S. and Pinsker, M. (1984). Learning algorithm for nonparametric filtering,Automatic Remote Control,11, 1434–1440.

    Google Scholar 

  • Freidman, J. H. and Silverman, B. W. (1989). Flexible parsimonous smoothing and additive modeling,Technometrics,31, 3–39.

    Article  MathSciNet  Google Scholar 

  • Hastie, T. J. and Tibshirani, R. J. (1990).Generalized Additive Models, Chapman-Hall, London.

    MATH  Google Scholar 

  • Huang, S. Y. and Lu, H. H.-S. (2000). Bayesian wavelet shrinkage for nonparametric mixed-effects models,Statistica Sinica,10, 1021–1040.

    MathSciNet  Google Scholar 

  • Kanazawa, Y. (1992). An optimal variable cell histogram based on the sample spacings,The Annals of Statistics,20, 291–304.

    MathSciNet  Google Scholar 

  • Lohler, M. (1999). Nonparametric estimation of piecewise smooth regression functions,Statistics and Probability Letters,43, 49–55.

    Article  MathSciNet  Google Scholar 

  • Li, K. C. (1987). Asymptotic optimality forc p ,c l cross-validation and generalized cross-validation: discrete index set,The Annals of Statistics,15, 958–975.

    MathSciNet  Google Scholar 

  • Lindstrom, M. J. (1999). Penalized estimation of free-knot splines,Journal of Computational and Graphical Statistics,8, 333–352.

    Article  MathSciNet  Google Scholar 

  • Mallows, C. L. (1973). Some comments onC p ,Technometrics,15, 661–675.

    Article  Google Scholar 

  • Misiti, M., Oppenheim, G. and Poggi, J.-M. (1995).The Wavelet Toolbox (ed. The Math Works).

  • Polyak, B. T. and Tsybakov, A. B. (1990). Asymptotic normality of thec p test for the orthogonal series estimation of regression,Theory of Probability and Its Applications,35, 293–306.

    Article  MathSciNet  Google Scholar 

  • Shibata, R. (1981). An optimal selection of regression variables,Biometrika,68, 45–54.

    Article  MathSciNet  Google Scholar 

  • Vidakovic, B. and Ruggeri, F. (2001). BAMS method: Theory and simulations, Special Issue on Wavelets,Sankhya Series B,63(2), 234–249.

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MAP5, UMR CNRS 8145, Université Paris 5, 45 rue des Saints-Pères, 75270, Paris cedex 06, France

    F. Comte

  2. LPMA, UMR CNRS 7599, Université Paris VII, 175 rue du Chevaleret, 75013, Paris, France

    Y. Rozenholc

  3. Université du Maine, Le Mans, France

    Y. Rozenholc

Authors
  1. F. Comte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. Rozenholc
    View author publications

    You can also search for this author in PubMed Google Scholar

About this article

Cite this article

Comte, F., Rozenholc, Y. A new algorithm for fixed design regression and denoising. Ann Inst Stat Math 56, 449–473 (2004). https://doi.org/10.1007/BF02530536

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02530536

Key words and phrases

Search

Navigation