Skip to main content

Efficient and fast spline-backfitted kernel smoothing of additive models

Abstract

A great deal of effort has been devoted to the inference of additive model in the last decade. Among existing procedures, the kernel type are too costly to implement for high dimensions or large sample sizes, while the spline type provide no asymptotic distribution or uniform convergence. We propose a one step backfitting estimator of the component function in an additive regression model, using spline estimators in the first stage followed by kernel/local linear estimators. Under weak conditions, the proposed estimator’s pointwise distribution is asymptotically equivalent to an univariate kernel/local linear estimator, hence the dimension is effectively reduced to one at any point. This dimension reduction holds uniformly over an interval under assumptions of normal errors. Monte Carlo evidence supports the asymptotic results for dimensions ranging from low to very high, and sample sizes ranging from moderate to large. The proposed confidence band is applied to the Boston housing data for linearity diagnosis.

This is a preview of subscription content, access via your institution.

References

  1. Andrews D., Whang Y. (1990). Additive interactive regression models: circumvention of the curse of the dimensionality. Econometric Theory 6, 466–479

    Article  MathSciNet  Google Scholar 

  2. Breiman L., Friedman J.H. (1985). Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association 80, 580–619

    MATH  Article  MathSciNet  Google Scholar 

  3. Bickel P.J., Rosenblatt M. (1973). On some global measures of the deviations of density function estimates. Annals of Statistics 1, 1071–1095

    MATH  Article  MathSciNet  Google Scholar 

  4. Claeskens G., Van Keilegom I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Annals of Statistics 31: 1852–1884

    MATH  Article  MathSciNet  Google Scholar 

  5. de Boor C. (2001). A practical guide to splines. New York, Springer

    MATH  Google Scholar 

  6. Fan J., Chen J. (1999). One-step local quasi-likelihood estimation. Journal of the Royal Statistical Society: Series B 61: 927–934

    MATH  Article  MathSciNet  Google Scholar 

  7. Fan J., Gijbels I. (1996). Local polynomial modelling and its applications. London, Chapman and Hall

    MATH  Google Scholar 

  8. Fan J., Härdle W., Mammen E. (1998). Direct estimation of low-dimensional components in additive models. Annals of Statistics 26, 943–971

    MATH  Article  MathSciNet  Google Scholar 

  9. Hall P., Titterington D.M. (1988). On confidence bands in nonparametric density estimation and regression. Journal of Multivariate Analysis 27, 228–254

    MATH  Article  MathSciNet  Google Scholar 

  10. Härdle W. (1989). Asymptotic maximal deviation of M-smoothers. Journal of Multivariate Analysis 29, 163–179

    MATH  Article  MathSciNet  Google Scholar 

  11. Härdle W. (1990). Applied nonparametric regression. Cambridge, Cambridge University Press

    MATH  Google Scholar 

  12. Härdle W., Hlávka Z., Klinke S. (2000). XploRe application guide. Berlin, Springer

    MATH  Google Scholar 

  13. Härdle W., Huet S., Mammen E., Sperlich S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20, 265–300

    MATH  Article  MathSciNet  Google Scholar 

  14. Härdle W., Sperlich S., Spokoiny V. (2001). Structural tests in additive regression. Journal of the American Statistical Association 96, 1333–1347

    MATH  Article  MathSciNet  Google Scholar 

  15. Harrison D., Rubinfeld D.L. (1978). Hedonic housing prices and the demand for cleaning air. Journal of Economics and Management 5, 81–102

    MATH  Article  Google Scholar 

  16. Hastie T.J., Tibshirani R.J. (1990). Generalized additive models. London, Chapman and Hall

    MATH  Google Scholar 

  17. Horowitz J.L., Mammen E. (2004). Nonparametric estimation of an additive model with a link function. Annals of Statistics 32, 2412–2443

    MATH  Article  MathSciNet  Google Scholar 

  18. Horowitz J.L., Klemelä J., Mammen E. (2006). Optimal estimation in additive regression models. Bernoulli 12, 271–298

    MATH  Article  MathSciNet  Google Scholar 

  19. Huang J.Z. (1998). Projection estimation in multiple regression with application to functional ANOVA models. Annals of Statistics 26, 242–272

    MATH  Article  MathSciNet  Google Scholar 

  20. Huang J.Z. (2003). Local asymptotics for polynomial spline regression. Annals of Statistics 31, 1600–1635

    MATH  Article  MathSciNet  Google Scholar 

  21. Huang J.Z., Yang L. (2004). Identification of nonlinear additive autoregression models. Journal of the Royal Statistical Society: Series B 66, 463–477

    MATH  Article  MathSciNet  Google Scholar 

  22. Kim W., Linton O.B., Hengartner N. (1999). A computationally efficient oracle estimator for additive nonparametric regression with bootstrap confidence intervals. Journal of Computational and Graphical Statistics 8, 278–297

    Article  MathSciNet  Google Scholar 

  23. Linton O.B., Nielsen J.P. (1995). Estimating structured nonparametric regression models by the kernel method. Biometrika 82, 93–101

    MATH  Article  MathSciNet  Google Scholar 

  24. Linton O.B., Härdle W. (1996). Estimating additive regression models with known links. Biometrika 83, 529–540

    MATH  Article  MathSciNet  Google Scholar 

  25. Linton O.B. (1997). Efficient estimation of additive nonparametric regression models. Biometrika 84, 469–473

    MATH  Article  MathSciNet  Google Scholar 

  26. Mammen E., Linton O., Nielsen J. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Annals of Statistics 27, 1443–1490

    MATH  MathSciNet  Google Scholar 

  27. Nielsen J.P., Sperlich S. (2005). Smooth backfitting in practice. Journal of the Royal Statistical Society: Series B 67, 43–61

    MATH  Article  MathSciNet  Google Scholar 

  28. Opsomer J.D. (2000). Asymptotic properties of backfitting estimators. Journal of Multivariate Analysis 73, 166–179

    MATH  Article  MathSciNet  Google Scholar 

  29. Opsomer J.D., Ruppert D. (1997). Fitting a bivariate additive model by local polynomial regression. Annals of Statistics 25, 186–211

    MATH  Article  MathSciNet  Google Scholar 

  30. Sperlich S., Tjøstheim D., Yang L. (2002). Nonparametric estimation and testing of interaction in additive models. Econometric Theory 18, 197–251

    MATH  Article  MathSciNet  Google Scholar 

  31. Stone C.J. (1985). Additive regression and other nonparametric models. Annals of Statistics 13, 689–705

    MATH  Article  MathSciNet  Google Scholar 

  32. Stone C.J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Annals of Statistics 22, 118–184

    MATH  Article  MathSciNet  Google Scholar 

  33. Tjøstheim D., Auestad B. (1994). Nonparametric identification of nonlinear time series: projections. Journal of the American Statistical Association 89, 1398–1409

    Article  MathSciNet  Google Scholar 

  34. Tusnády G. (1977). A remark on the approximation of the sample df in the multidimensional case. Periodica Mathematica Hungarica 8, 53–55

    MATH  Article  MathSciNet  Google Scholar 

  35. Wang, J., Yang, L. (2007a). Polynomial spline confidence bands for regression curves. Manuscript.

  36. Wang, J., Yang, L. (2007b). Efficient and fast spline-backfitted kernel smoothing of additive models. http://www.stt.msu.edu/~yangli/SBKAISMfull.pdf.

  37. Xia Y. (1998). Bias-corrected confidence bands in nonparametric regression. Journal of the Royal Statistical Society: Series B 60, 797–811

    MATH  Article  Google Scholar 

  38. Xue L., Yang L. (2006a). Additive coefficient modeling via polynomial spline. Statistica Sinica 16, 1423–1446

    MathSciNet  Google Scholar 

  39. Xue L., Yang L. (2006b). Estimation of semiparametric additive coefficient model. Journal of Statistical Planning and Inference 136, 2506–2534

    MATH  Article  MathSciNet  Google Scholar 

  40. Yang, L. (2007). Confidence band for additive regression model. Journal of Data Science, forthcoming.

  41. Yang L., Härdle W., Nielsen J.P. (1999). Nonparametric autoregression with multiplicative volatility and additive mean. Journal of Time Series Analysis 20, 579–604

    MATH  Article  MathSciNet  Google Scholar 

  42. Yang L., Sperlich S., Härdle W. (2003). Derivative estimation and testing in generalized additive models. Journal of Statistical Planning and Inference 115: 521–542

    MATH  Article  MathSciNet  Google Scholar 

  43. Yang L., Park B.U., Xue L., Härdle W. (2006). Estimation and testing of varying coefficients in additive models with marginal integration. Journal of the American Statistical Association 101: 1212–1227

    MATH  Article  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jing Wang.

Additional information

Supported in part by NSF awards DMS 0405330, 0706518, BCS 0308420 and SES 0127722.

About this article

Cite this article

Wang, J., Yang, L. Efficient and fast spline-backfitted kernel smoothing of additive models. Ann Inst Stat Math 61, 663–690 (2009). https://doi.org/10.1007/s10463-007-0157-x

Download citation

Keywords

  • Bandwidths
  • B spline
  • Knots
  • Local linear estimator
  • Nadaraya-Watson estimator
  • Nonparametric regression