Annals of the Institute of Statistical Mathematics

, Volume 61, Issue 3, pp 663–690 | Cite as

Efficient and fast spline-backfitted kernel smoothing of additive models

  • Jing WangEmail author
  • Lijian Yang


A great deal of effort has been devoted to the inference of additive model in the last decade. Among existing procedures, the kernel type are too costly to implement for high dimensions or large sample sizes, while the spline type provide no asymptotic distribution or uniform convergence. We propose a one step backfitting estimator of the component function in an additive regression model, using spline estimators in the first stage followed by kernel/local linear estimators. Under weak conditions, the proposed estimator’s pointwise distribution is asymptotically equivalent to an univariate kernel/local linear estimator, hence the dimension is effectively reduced to one at any point. This dimension reduction holds uniformly over an interval under assumptions of normal errors. Monte Carlo evidence supports the asymptotic results for dimensions ranging from low to very high, and sample sizes ranging from moderate to large. The proposed confidence band is applied to the Boston housing data for linearity diagnosis.


Bandwidths B spline Knots Local linear estimator Nadaraya-Watson estimator Nonparametric regression 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andrews D., Whang Y. (1990). Additive interactive regression models: circumvention of the curse of the dimensionality. Econometric Theory 6, 466–479CrossRefMathSciNetGoogle Scholar
  2. Breiman L., Friedman J.H. (1985). Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association 80, 580–619zbMATHCrossRefMathSciNetGoogle Scholar
  3. Bickel P.J., Rosenblatt M. (1973). On some global measures of the deviations of density function estimates. Annals of Statistics 1, 1071–1095zbMATHCrossRefMathSciNetGoogle Scholar
  4. Claeskens G., Van Keilegom I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Annals of Statistics 31: 1852–1884zbMATHCrossRefMathSciNetGoogle Scholar
  5. de Boor C. (2001). A practical guide to splines. New York, SpringerzbMATHGoogle Scholar
  6. Fan J., Chen J. (1999). One-step local quasi-likelihood estimation. Journal of the Royal Statistical Society: Series B 61: 927–934zbMATHCrossRefMathSciNetGoogle Scholar
  7. Fan J., Gijbels I. (1996). Local polynomial modelling and its applications. London, Chapman and HallzbMATHGoogle Scholar
  8. Fan J., Härdle W., Mammen E. (1998). Direct estimation of low-dimensional components in additive models. Annals of Statistics 26, 943–971zbMATHCrossRefMathSciNetGoogle Scholar
  9. Hall P., Titterington D.M. (1988). On confidence bands in nonparametric density estimation and regression. Journal of Multivariate Analysis 27, 228–254zbMATHCrossRefMathSciNetGoogle Scholar
  10. Härdle W. (1989). Asymptotic maximal deviation of M-smoothers. Journal of Multivariate Analysis 29, 163–179zbMATHCrossRefMathSciNetGoogle Scholar
  11. Härdle W. (1990). Applied nonparametric regression. Cambridge, Cambridge University PresszbMATHGoogle Scholar
  12. Härdle W., Hlávka Z., Klinke S. (2000). XploRe application guide. Berlin, SpringerzbMATHGoogle Scholar
  13. Härdle W., Huet S., Mammen E., Sperlich S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20, 265–300zbMATHCrossRefMathSciNetGoogle Scholar
  14. Härdle W., Sperlich S., Spokoiny V. (2001). Structural tests in additive regression. Journal of the American Statistical Association 96, 1333–1347zbMATHCrossRefMathSciNetGoogle Scholar
  15. Harrison D., Rubinfeld D.L. (1978). Hedonic housing prices and the demand for cleaning air. Journal of Economics and Management 5, 81–102zbMATHCrossRefGoogle Scholar
  16. Hastie T.J., Tibshirani R.J. (1990). Generalized additive models. London, Chapman and HallzbMATHGoogle Scholar
  17. Horowitz J.L., Mammen E. (2004). Nonparametric estimation of an additive model with a link function. Annals of Statistics 32, 2412–2443zbMATHCrossRefMathSciNetGoogle Scholar
  18. Horowitz J.L., Klemelä J., Mammen E. (2006). Optimal estimation in additive regression models. Bernoulli 12, 271–298zbMATHCrossRefMathSciNetGoogle Scholar
  19. Huang J.Z. (1998). Projection estimation in multiple regression with application to functional ANOVA models. Annals of Statistics 26, 242–272zbMATHCrossRefMathSciNetGoogle Scholar
  20. Huang J.Z. (2003). Local asymptotics for polynomial spline regression. Annals of Statistics 31, 1600–1635zbMATHCrossRefMathSciNetGoogle Scholar
  21. Huang J.Z., Yang L. (2004). Identification of nonlinear additive autoregression models. Journal of the Royal Statistical Society: Series B 66, 463–477zbMATHCrossRefMathSciNetGoogle Scholar
  22. Kim W., Linton O.B., Hengartner N. (1999). A computationally efficient oracle estimator for additive nonparametric regression with bootstrap confidence intervals. Journal of Computational and Graphical Statistics 8, 278–297CrossRefMathSciNetGoogle Scholar
  23. Linton O.B., Nielsen J.P. (1995). Estimating structured nonparametric regression models by the kernel method. Biometrika 82, 93–101zbMATHCrossRefMathSciNetGoogle Scholar
  24. Linton O.B., Härdle W. (1996). Estimating additive regression models with known links. Biometrika 83, 529–540zbMATHCrossRefMathSciNetGoogle Scholar
  25. Linton O.B. (1997). Efficient estimation of additive nonparametric regression models. Biometrika 84, 469–473zbMATHCrossRefMathSciNetGoogle Scholar
  26. Mammen E., Linton O., Nielsen J. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Annals of Statistics 27, 1443–1490zbMATHMathSciNetGoogle Scholar
  27. Nielsen J.P., Sperlich S. (2005). Smooth backfitting in practice. Journal of the Royal Statistical Society: Series B 67, 43–61zbMATHCrossRefMathSciNetGoogle Scholar
  28. Opsomer J.D. (2000). Asymptotic properties of backfitting estimators. Journal of Multivariate Analysis 73, 166–179zbMATHCrossRefMathSciNetGoogle Scholar
  29. Opsomer J.D., Ruppert D. (1997). Fitting a bivariate additive model by local polynomial regression. Annals of Statistics 25, 186–211zbMATHCrossRefMathSciNetGoogle Scholar
  30. Sperlich S., Tjøstheim D., Yang L. (2002). Nonparametric estimation and testing of interaction in additive models. Econometric Theory 18, 197–251zbMATHCrossRefMathSciNetGoogle Scholar
  31. Stone C.J. (1985). Additive regression and other nonparametric models. Annals of Statistics 13, 689–705zbMATHCrossRefMathSciNetGoogle Scholar
  32. Stone C.J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Annals of Statistics 22, 118–184zbMATHCrossRefMathSciNetGoogle Scholar
  33. Tjøstheim D., Auestad B. (1994). Nonparametric identification of nonlinear time series: projections. Journal of the American Statistical Association 89, 1398–1409CrossRefMathSciNetGoogle Scholar
  34. Tusnády G. (1977). A remark on the approximation of the sample df in the multidimensional case. Periodica Mathematica Hungarica 8, 53–55zbMATHCrossRefMathSciNetGoogle Scholar
  35. Wang, J., Yang, L. (2007a). Polynomial spline confidence bands for regression curves. Manuscript.Google Scholar
  36. Wang, J., Yang, L. (2007b). Efficient and fast spline-backfitted kernel smoothing of additive models.
  37. Xia Y. (1998). Bias-corrected confidence bands in nonparametric regression. Journal of the Royal Statistical Society: Series B 60, 797–811zbMATHCrossRefGoogle Scholar
  38. Xue L., Yang L. (2006a). Additive coefficient modeling via polynomial spline. Statistica Sinica 16, 1423–1446MathSciNetGoogle Scholar
  39. Xue L., Yang L. (2006b). Estimation of semiparametric additive coefficient model. Journal of Statistical Planning and Inference 136, 2506–2534zbMATHCrossRefMathSciNetGoogle Scholar
  40. Yang, L. (2007). Confidence band for additive regression model. Journal of Data Science, forthcoming.Google Scholar
  41. Yang L., Härdle W., Nielsen J.P. (1999). Nonparametric autoregression with multiplicative volatility and additive mean. Journal of Time Series Analysis 20, 579–604zbMATHCrossRefMathSciNetGoogle Scholar
  42. Yang L., Sperlich S., Härdle W. (2003). Derivative estimation and testing in generalized additive models. Journal of Statistical Planning and Inference 115: 521–542zbMATHCrossRefMathSciNetGoogle Scholar
  43. Yang L., Park B.U., Xue L., Härdle W. (2006). Estimation and testing of varying coefficients in additive models with marginal integration. Journal of the American Statistical Association 101: 1212–1227zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2007

Authors and Affiliations

  1. 1.Department of Mathematics, Statistics, and Computer ScienceUniversity of Illinois at ChicagoChicagoUSA
  2. 2.Department of Statistics and ProbabilityMichigan State UniversityEast LansingUSA

Personalised recommendations