Summary
Regression spline smoothing involves modelling a regression function as a piecewise polynomial with a high number of pieces relative to the sample size. Because the number of possible models is so large, efficient strategies for choosing among them are required. In this paper we review approaches to this problem and compare them through a simulation study. For simplicity and conciseness we restrict attention to the univariate smoothing setting with Gaussian noise and the truncated polynomial regression spline basis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agarwal, G.G. and Studden, W.J. (1978). An algorithm for selection of design and knots in the response curve estimation by spline functions. Technical Report No. 78-15, Purdue University.
Agarwal, G.G. and Studden, W.J. (1980). Asymptotic integrated mean square error using least squares and bias minimising splines. Ann. Statist, 8, 1307–1325.
de Boor, C. (1978) A Practical Guide to Splines. Springer, New York.
Denison, D.G.T, Mallick, B.K. and Smith, A.F.M. (1998). Automatic Bayesian curve fitting. J. Roy. Statist. Soc, B, 60, 333–350.
Eilers, P.H.C. and Marx, B.D. (1996). Flexible smoothing with B-splines and penalties (with discussion). Statist. Sci., 89, 89–121.
Friedman, J.H. and Silverman, B.W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics, 31, 3–39.
Friedman, J.H. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist., 19, 1–141.
Fuller, W.A. (1969). Grafting polynomials as approximating functions. Austr. J. Agric. Econ., 13, 35–46.
Furnival, G.M and Wilson, R.W. Jr. (1974). Regressions by leaps and bounds. Technometrics, 16, 499–511.
Gallant, A.R. and Fuller, W.A. (1973). Fitting segmented polynomial regression models whose join points have to be estimated. J. Amer. Statist. Assoc., 68, 144–147.
Gelfand, A.E. and Smith, A.F.M. (1990). Sampling-based approaches to calculating marginal densities, J. Amer. Statist. Assoc., 85, 398–409.
Green, P.J. and Silverman, B.W. (1994). Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, London: Chapman and Hall.
Green, P.J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 711–732.
He, X. and Ng, P. (1996). COBS: constrained smoothing made easy. Unpublished manuscript.
Hudson, D.J. (1966). Fitting segmented curves whose join points have to be estimated. J. Amer. Statist. Assoc., 61, 1097–1129.
Luo, Z. and Wahba, G. (1997). Hybrid adaptive splines. J. Amer. Statist. Assoc., 92, 107–116.
O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems (with discussion). Statist. Sex, 1, 505–527.
O’Sullivan, F. (1988). Fast computations of fully automated log-density and log-hazard estimators. SIAM J. Sci. Statist. Comput., 9, 363–379.
Reinsch, C. (1967). Smoothing by spline functions. Numer. Math., 10, 177–183.
Ruppert, D. and Carroll, R.J. (1998). Penalized regression splines. Unpublished manuscript.
Smith, M. and Kohn, R. (1996). Nonparametric regression using Bayesian variable selection. J. Econometrics, 75, 317–344.
Smith, P.L. (1979). Splines as a useful and convenient statistical tool. Amer. Statistician., 33, 57–62.
Smith, P.L. (1982). Curve fitting and modeling with splines using statistical variable selection techniques. Report NASA 166034, NASA, Langley Research Center, Hampton.
Stone, C.J., Hansen, M., Kooperberg, C. and Truong, Y. K. (1997). Polynomial splines and their tensor products in extended linear modeling, Ann. Statist., 25, 1371–1425.
Studden, W.J. and Van Arman, D.J. (1969). Admissible designs for polynomial spline regression. Ann. Math. Statist., 40, 1557–1569.
Tibshirani, R. (1996). Regression shrinkage via the lasso. J. Royal Statist. Soc., Ser. B, 58, 267–288.
Weisberg, S. (1985). Applied Linear Regression, 2nd ed. New York: Wiley.
Wold, S. (1971). Analysis of kinetic data by means of spline functions. Chemica Scripta, 1, 97–102.
Wold, S. (1974). Spline functions in data analysis. Technometrics, 16, 1–11.
Acknowledgments
I am grateful to Mark Hansen, Xuming He, Roger Koenker, Robert Kohn, Charles Kooperberg, Steve Portnoy, Doug Simpson and Mike Smith for helpful comments. The final version of this paper also benefited from the helpful comments of two referees and an associate editor. Mike Smith’s Bayesian regression code br was used in this study. The simulation settings are based on those originally developed by Steve Marron. Part of this research was carried out while the author enjoyed the hospitality of the Department of Statistics in the University of Illinois at Urbana-Champaign.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wand, M.P. A Comparison of Regression Spline Smoothing Procedures. Computational Statistics 15, 443–462 (2000). https://doi.org/10.1007/s001800000047
Published:
Issue Date:
DOI: https://doi.org/10.1007/s001800000047