Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
  1. Home
  2. Probability Theory and Related Fields
  3. Article
On the efficiency of selection criteria in spline regression
Download PDF
Download PDF
  • Published: 04 July 2003

On the efficiency of selection criteria in spline regression

  • S. C. Kou1 

Probability Theory and Related Fields volume 127, pages 153–176 (2003)Cite this article

  • 118 Accesses

  • 15 Citations

  • Metrics details

Abstract.

This paper concerns the cubic smoothing spline approach to nonparametric regression. After first deriving sharp asymptotic formulas for the eigenvalues of the smoothing matrix, the paper uses these formulas to investigate the efficiency of different selection criteria for choosing the smoothing parameter. Special attention is paid to the generalized maximum likelihood (GML), C p and extended exponential (EE) criteria and their marginal Bayesian interpretation. It is shown that (a) when the Bayesian model that motivates GML is true, using C p to estimate the smoothing parameter would result in a loss of efficiency with a factor of 10/3, proving and strengthening a conjecture proposed in Stein (1990); (b) when the data indeed come from the C p density, using GML would result in a loss of efficiency of ∞ ; (c) the loss of efficiency of the EE criterion is at most 1.543 when the data are sampled from its consistent density family. The paper not only studies equally spaced observations (the setting of Stein, 1990), but also investigates general sampling scheme of the design points, and shows that the efficiency results remain the same in both cases.

Download to read the full article text

Working on a manuscript?

Avoid the common mistakes

References

  1. Akaike, H.: A new look at statistical model identification. IEEE Trans. Auto. Cont. AU-19, 716–722 (1974)

    Google Scholar 

  2. Billingsley, P.: Probability and Measure, 3rd ed. Wiley, New York, 1995

  3. Bowman, A., Azzalini, A.: Applied smoothing techniques for data analysis: the kernel approach with S-Plus illustrations. Oxford University Press, New York, 1997

  4. Craven, P., Wahba, G.: Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377–403 (1979)

    MathSciNet  MATH  Google Scholar 

  5. Culpin, D.: Calculation of cubic smoothing splines for equally spaced data. Numer. Math. 48, 627–638 (1986)

    MathSciNet  MATH  Google Scholar 

  6. Demmler, A., Reinsch, C.: Oscillation matrices with spline smoothing. Numer. Math. 24, 375–382 (1975)

    MATH  Google Scholar 

  7. Efron, B.: Selection criteria for scatterplot smoothers. Ann. Statist. 29, 470–504 (2001)

    MathSciNet  MATH  Google Scholar 

  8. Eubank, R.: Spline Smoothing and Nonparametric Regression. Marcel Dekker, New York, 1988

  9. Eubank, R.: Nonparametric Regression and Spline Smoothing, 2nd ed, Marcel Dekker, New York, 1999

  10. Fan, J.: Prospects of nonparametric modeling. J. Amer. Statist. Assoc. 95, 1296–1300 (2000)

    MATH  Google Scholar 

  11. Fan, J., Gijbels, I.: Local Polynomial Modelling and Its Applications. Chapman and Hall, London, 1996

  12. Feller, W.: An Introduction to Probability Theory and Its Applications, Vol. II. Wiley, New York, 1971

  13. Gradshteyn, I., Ryzhik, I.: Table of Integrals, Series, and Products. Academic Press, Boston, 1994

  14. Green, P., Silverman, B.: Nonparametric Regression and Generalized Linear Models. Chapman and Hall, London, 1994

  15. Hall, P.: Biometrika century: nonparametrics. Biometrika 88, 143–165 (2001)

    MathSciNet  MATH  Google Scholar 

  16. Hall, P., Johnstone, I.: Empirical functionals and efficient smoothing parameter selection (with discussion). J. Roy. Statist. Soc. B 54, 475–530 (1992)

    MATH  Google Scholar 

  17. Härdle, W.: Applied Nonparametric Regression. Cambridge University Press, Cambridge, 1990

  18. Härdle, W., Hall, P., Marron, S.: How far are the optimally chosen smoothing parameters from their optimum? (with discussion.) J. Amer. Statist. Assoc. 83, 86–101 (1988)

    MathSciNet  Google Scholar 

  19. Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman and Hall, London, 1990

  20. Kimeldorf, G., Wahba, G.: A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Statist. 41, 495–502 (1970)

    MATH  Google Scholar 

  21. Kneip, A.: Ordered linear smoothers. Ann. Statist. 22, 835–866 (1994)

    MathSciNet  MATH  Google Scholar 

  22. Kou, S.C.: Extended exponential criterion: a new selection procedure for scatterplot smoothers. Ph. D. thesis, Stanford University, 2001

  23. Kou, S.C., Efron, B.: Smoothers and the C p , GML and EE criteria: A geometric approach. J. Amer. Statist. Assoc. 97, 766–782 (2002)

    Article  Google Scholar 

  24. Li, K.-C.: Asymptotic optimality of C L and generalized cross-validation in ridge regression with application to spline smoothing. Ann. Statist. 14, 1101–1112 (1986)

    MathSciNet  MATH  Google Scholar 

  25. Li, K.-C.: Asymptotic optimality for C p , C L , cross-validation and generalized cross-validation: discrete index set. Ann. Statist. 15, 958–975 (1987)

    MathSciNet  MATH  Google Scholar 

  26. Mallows, C.: Some comments on C p . Technometrics 15, 661–675 (1973)

    MATH  Google Scholar 

  27. Nussbaum, M: Spline smoothing in regression models and asymptotic efficiency in L 2. Ann. Statist. 13, 984–997 (1985)

    MathSciNet  MATH  Google Scholar 

  28. Rosenblatt, M.: Stochastic Curve Estimation. NSF-CBMS Regional Conference Series in Probability and Statistics, Volume 3. IMS, Hayward, 1991

  29. Reinsch, C.: Smoothing by spline functions. Numer. Math. 10, 177–183 (1967)

    MATH  Google Scholar 

  30. Schoenberg, I.: Spline functions and the problem of graduation. Proc. Nat. Acad. Sci. USA. 52, 947–950 (1964a)

    Google Scholar 

  31. Schoenberg, I.: On interpolation by spline functions and its minimum properties. Internat. Ser. Numer. Anal. 5, 109–129 (1964b)

    Google Scholar 

  32. Silverman, B.: A fast and efficient cross-validation method for smoothing parameter choice in spline regression. J. Amer. Statist. Assoc. 79, 584–589 (1984)

    MathSciNet  MATH  Google Scholar 

  33. Silverman, B.: Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with discussion). J. Roy. Statist. Soc. B 47, 1–52 (1985)

    MATH  Google Scholar 

  34. Simonoff, J.: Smoothing Methods in Statistics. Springer-Verlag, New York, 1996

  35. Speckman, P.: Efficient nonparametric regression with cross-validated smoothing splines. Unpublished manuscript, 1983

  36. Speckman, P.: Spline smoothing and optimal rates of convergence in nonparametric regression models. Ann. Statist. 13, 970–983 (1985)

    MathSciNet  MATH  Google Scholar 

  37. Speckman, P., Sun, D.: Asymptotic properties of smoothing parameter selection in spline regression. Preprint, 2001

  38. Stein, M.: A comparison of generalized cross validation and modified maximum likelihood for estimating the parameters of a stochastic process. Ann. Statist. 18, 1139–1157 (1990)

    MathSciNet  MATH  Google Scholar 

  39. Stein, M.: Spline smoothing with an estimated order parameter. Ann. Statist. 21, 1522–1544 (1993)

    MathSciNet  MATH  Google Scholar 

  40. Utreras, F.: Cross-validation techniques for smoothing spline functions in one or two dimensions. In: Smoothing Techniques for Curve Estimation, (T. Gasser, M. Rosenblatt, ed.), Springer-Verlag, Heidelberg, 1979, pp. 196–232

  41. Utreras, F.: Sur le choix du parametre d'ajustement dans le lissage par fonctions spline. Numer. Math. 34, 15–28 (1980)

    Google Scholar 

  42. Utreras, F.: Optimal smoothing of noisy data using spline functions. SIAM J. Sci. and Statist. Comput. 2, 349–362 (1981)

    Google Scholar 

  43. Utreras, F.: Boundary effects on convergence rates for Tikhonov regularization. J. Approx. Theor. 54, 235–249 (1988)

    MathSciNet  MATH  Google Scholar 

  44. Wahba, G.: Smoothing noisy data by spline functions. Numer. Math. 24, 383–393 (1975)

    MATH  Google Scholar 

  45. Wahba, G.: Optimal smoothing of density estimates. In: Classification and Clustering (J. Van Ryzin, ed.), Academic Press, New York, 1977a, pp. 423–458

  46. Wahba, G.: A survey of some smoothing problems and the method of generalized cross-validation for solving them. In: Applications of Statistics (P. R. Krishnaiah, ed.), North Holland, Amsterdam. 1977b, pp. 507–523

  47. Wahba, G.: A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Statist. 13, 1378–1402 (1985)

    MathSciNet  MATH  Google Scholar 

  48. Wahba, G.: Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, 59. SIAM, Philadelphia, 1990

  49. Wecker, W., Ansley, C.: The signal extraction approach to nonlinear regression and spline smoothing. J. Amer. Statist. Assoc. 78, 81–89 (1983)

    MathSciNet  MATH  Google Scholar 

  50. Whittaker, E.: On a new method of graduation. Proc. Edinburgh Math. Soc. 41, 63–75 (1923)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Harvard University, Science Center 6th Floor, Cambridge, MA, 02138, USA

    S. C. Kou

Authors
  1. S. C. Kou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. C. Kou.

Additional information

This work is supported in part by NSF grant DMS-0204674 and Harvard University Clark-Cooke Fund.

Mathematics Subject Classification (2000): Primary: 62G08; Secondary: 62G20

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kou, S. On the efficiency of selection criteria in spline regression. Probab. Theory Relat. Fields 127, 153–176 (2003). https://doi.org/10.1007/s00440-003-0277-z

Download citation

  • Received: 23 July 2003

  • Revised: 26 March 2003

  • Published: 04 July 2003

  • Issue Date: October 2003

  • DOI: https://doi.org/10.1007/s00440-003-0277-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Key words or phrases:

  • Smoothing splines
  • Extended exponential criterion
  • Cp
  • Generalized maximum likelihood
  • Eigenvalue
  • Robustness
  • Sampling scheme
Download PDF

Working on a manuscript?

Avoid the common mistakes

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature