Adaptive lifting for nonparametric regression

Statistics and Computing volume 16, pages143–159(2006)Cite this article

Abstract

Many wavelet shrinkage methods assume that the data are observed on an equally spaced grid of length of the form 2J for some J. These methods require serious modification or preprocessed data to cope with irregularly spaced data. The lifting scheme is a recent mathematical innovation that obtains a multiscale analysis for irregularly spaced data.

A key lifting component is the “predict” step where a prediction of a data point is made. The residual from the prediction is stored and can be thought of as a wavelet coefficient. This article exploits the flexibility of lifting by adaptively choosing the kind of prediction according to a criterion. In this way the smoothness of the underlying ‘wavelet’ can be adapted to the local properties of the function. Multiple observations at a point can readily be handled by lifting through a suitable choice of prediction. We adapt existing shrinkage rules to work with our adaptive lifting methods.

We use simulation to demonstrate the improved sparsity of our techniques and improved regression performance when compared to both wavelet and non-wavelet methods suitable for irregular data. We also exhibit the benefits of our adaptive lifting on the real inductance plethysmography and motorcycle data.

This is a preview of subscription content, log in to check access.

References

  1. Abramovich F., Bailey T., and Sapatinas T. 2000. Wavelet analysis and its statistical applications. J. Roy. Statist. Soc. D 49: 1–29.

    Article  Google Scholar 

  2. Antoniadis A. and Fan J. 2001. Regularization of wavelet approximations. J. Am. Statist. Ass., 96: 939–967.

    Article  MathSciNet  Google Scholar 

  3. Barber S. and Nason G.P. 2004. Real nonparametric regression using complex wavelets. J. Roy. Statist. Soc. Series B 66: 927–939.

    Article  MathSciNet  Google Scholar 

  4. Boulgouris N.V., Tzovaras D., and Strintzis M.G. 2001. Lossless image compression based on optimal prediction, adaptive lifting and conditional arithmetic coding. IEEE Trans. Im. Proc. 10: 1–14.

    Article  Google Scholar 

  5. Cai T. and Brown L. 1998. Wavelet shrinkage for non-equispaced samples. Ann. Stat. 26: 1783–1799.

    Article  MathSciNet  Google Scholar 

  6. Cai T. and Brown L. 1999. Wavelet estimation for samples with random uniform design. Stat. Prob. Lett. 42: 313–321.

    Article  MathSciNet  Google Scholar 

  7. Claypoole R.L., Baraniuk R.G. and Nowak R.D. 1998. Adaptive wavelet transforms via lifting. In Transactions of the International Conference on Acoustics, Speech and Signal Processing. IEEE Trans. Im. Proc. 12: 1513–1516.

    Google Scholar 

  8. Claypoole R.L., Davis G.M., Sweldens W., and Baraniuk R.G. 2003. Nonlinear wavelet transforms for image coding via lifting. IEEE Trans. Im. Proc. 12: 1449–1459.

    Article  MathSciNet  Google Scholar 

  9. Daubechies I. 1992. Ten Lectures on Wavelets. Philadelphia: SIAM.

    Google Scholar 

  10. Delouille V., Franke J., and von Sachs R. 2004. Nonparametric stochastic regression with design-adapted wavelets. Sankhya, A, 63: 328–366.

    Google Scholar 

  11. Delouille V., Simoens J., and von Sachs R. 2004. Smooth design-adapted wavelets for nonparametric stochastic regression. J. Am. Statist. Soc. 99: 643–658.

    Google Scholar 

  12. Donoho D.L. and Johnstone I.M. 1994. Ideal spatial adaptation by wavelet shrinkage. Biometrika 81: 425–455.

    Article  MathSciNet  Google Scholar 

  13. Donoho D.L. and Johnstone I.M. 1995a. Adapting to unknown smoothness via wavelet shrinkage. J. Am. Statist. Soc. 90: 1200–1224.

    MathSciNet  Google Scholar 

  14. Donoho D.L., Johnstone I.M., Kerkyacharian G., and Picard D. 1995b. Wavelet shrinkage: asymptopia? (with discussion) J. Roy. Statist. Soc. B 57: 301–337.

    Google Scholar 

  15. Friedman J.H. 1984. A variable span scatterplot smoother. Technical Report, No. 5, Laboratory for Computational Statistics, Stanford University, Stanford, CA, USA.

    Google Scholar 

  16. Green P.J. and Silverman B.W. 1994. Nonparametric regression and generalized linear models. Chapman and Hall: London.

    Google Scholar 

  17. Jansen M., Nason G.P. and Silverman B.W. 2001. Scattered data smoothing by empirical Bayesian shrinkage of second generation wavelet coefficients. In Unser, M. and Aldroubi, A. (eds) Wavelet applications in signal and image processing, Proceedings of SPIE 4478: 87–97.

  18. Jansen M., Nason G.P. and Silverman B.W. 2004. Multivariate nonparametric regression using lifting. Technical Report 04:17, Statistics Group, Department of Mathematics, University of Bristol, UK.

    Google Scholar 

  19. Johnstone I.M. and Silverman B.W. 2004a. Needles and hay in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32: 1594–1649.

    Article  MathSciNet  Google Scholar 

  20. Johnstone I.M. and Silverman B.W. 2004b. EBayesThresh: R and S-PLUS programs for Empirical Bayes thresholding. Unpublished manuscript (available from the CRAN archive).

  21. Johnstone I.M. and Silverman B.W. 2005. Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33: (to appear).

  22. Knight M.I. and Nason G.P. 2004. Improving prediction of hydrophobic segments along a transmembrane protein sequence using adaptive multiscale lifting. Technical Report 04:19, Statistics Group, Department of Mathematics, University of Bristol, UK.

    Google Scholar 

  23. Kovac A. 1998. Wavelet Thresholding for Unequally Time-Spaced Data. PhD Thesis, University of Bristol.

  24. Kovac A. and Silverman B.W. 2000. Extending the scope of wavelet regression methods by coefficient-dependent thresholding. J. Am. Statist. Ass. 95: 172–183.

    Google Scholar 

  25. Loader C. 1997. Locfit: an introduction. Stat. Comput. Graph. News. 8: 11–17.

    Google Scholar 

  26. Loader C. 1999. Local regression and likelihood. Springer: New York.

    Google Scholar 

  27. Mallat S.G. 1989. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattn. Anal. Mach. Intell. 11: 674–693.

    Article  MATH  Google Scholar 

  28. Nason G.P. 1996. Wavelet shrinkage using cross-validation. J. Roy. Statist. Soc. B 58: 463–479.

    MATH  MathSciNet  Google Scholar 

  29. Nason G.P. 2002. Choice of wavelet smoothness, primary resolution and threshold in wavelet shrinkage. Stat. Comput. 12: 219–227.

    Article  MathSciNet  Google Scholar 

  30. Nason G.P. and Silverman B.W. 1994. The discrete wavelet transform in S. J. Comp. Graph. Statist. 3: 163–191.

    Google Scholar 

  31. Nunes M.A. and Nason G.P. 2005. Stopping time in adaptive lifting. Technical Report 05:15, Statistics Group, Department of Mathematics, University of Bristol, UK.

    Google Scholar 

  32. Pensky M. and Vidakovic B. 2001. On non-equally spaced wavelet regression. Ann. Inst. Statist. Math. 53: 681–690.

    Article  MathSciNet  Google Scholar 

  33. Percival D.B. and Walden A.T. 2000. Wavelet methods for time series analysis. Cambridge University Press: Cambridge.

    Google Scholar 

  34. Piella G. and Heijmans, H.J.A.M. 2002. Adaptive lifting schemes with perfect reconstruction. IEEE Trans. Sig. Proc. 50: 1620–1630.

    Article  Google Scholar 

  35. Sardy S., Percival D.B., Bruce A.G., Gao H.-Y. and Stuetzle W. 1999. Wavelet de-noising for unequally spaced data. Stat. Comput. 9: 65–75.

    Article  Google Scholar 

  36. Silverman B.W. 1985. Some aspects of the spline smoothing approach to nonparametric curve fitting. J. Roy. Statist. Soc. B 47: 1–52.

    MATH  Google Scholar 

  37. Simoens J. and Vandewalle S. 2003. A stabilized construction of wavelets on irregular meshes on the interval. SIAM J. Scientific Computing 24: 1356–1378.

    Article  MathSciNet  Google Scholar 

  38. Sweldens W. 1996. Wavelets and the lifting scheme: A 5 minute tour. Z. Angew. Math. Mech. 76: 41–44.

    MATH  Google Scholar 

  39. Sweldens W. 1997. The lifting scheme: a construction of second generation wavelets. SIAM J. Math. Anal. 29: 511–546.

    Article  MathSciNet  Google Scholar 

  40. Trappe W. and Liu K.J.R. 2000. Denoising via adaptive lifting schemes. In Proceedings of SPIE, Wavelet applications in signal and image processing VIII, Aldroubi A., Laine M.A. and Unser M.A. (eds), 4119: 302–312.

  41. Vanraes E., Jansen M. and Bultheel A. 2002. Stabilised wavelet transforms for non-equispaced data smoothing. Sig. Proc. 82: 1979–1990.

    Article  Google Scholar 

  42. Vidakovic B. 1999. Statistical modeling by wavelets. Wiley: New York.

    Google Scholar 

Download references

Author information

Affiliations

  1. Department of Mathematics, University of Bristol, UK

    Matthew A. Nunes, Marina I. Knight & Guy P. Nason

Authors
  1. Matthew A. Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marina I. Knight
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guy P. Nason
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew A. Nunes.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nunes, M.A., Knight, M.I. & Nason, G.P. Adaptive lifting for nonparametric regression. Stat Comput 16, 143–159 (2006). https://doi.org/10.1007/s11222-006-6560-y

Download citation

Keywords

  • Curve estimation
  • Lifting
  • Nonparametric regression
  • Wavelets