Skip to main content
SpringerLink
Log in

Efficient and fast spline-backfitted kernel smoothing of additive models

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

A great deal of effort has been devoted to the inference of additive model in the last decade. Among existing procedures, the kernel type are too costly to implement for high dimensions or large sample sizes, while the spline type provide no asymptotic distribution or uniform convergence. We propose a one step backfitting estimator of the component function in an additive regression model, using spline estimators in the first stage followed by kernel/local linear estimators. Under weak conditions, the proposed estimator’s pointwise distribution is asymptotically equivalent to an univariate kernel/local linear estimator, hence the dimension is effectively reduced to one at any point. This dimension reduction holds uniformly over an interval under assumptions of normal errors. Monte Carlo evidence supports the asymptotic results for dimensions ranging from low to very high, and sample sizes ranging from moderate to large. The proposed confidence band is applied to the Boston housing data for linearity diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrews D., Whang Y. (1990). Additive interactive regression models: circumvention of the curse of the dimensionality. Econometric Theory 6, 466–479

    Article  MathSciNet  Google Scholar 

  • Breiman L., Friedman J.H. (1985). Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association 80, 580–619

    Article  MATH  MathSciNet  Google Scholar 

  • Bickel P.J., Rosenblatt M. (1973). On some global measures of the deviations of density function estimates. Annals of Statistics 1, 1071–1095

    Article  MATH  MathSciNet  Google Scholar 

  • Claeskens G., Van Keilegom I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Annals of Statistics 31: 1852–1884

    Article  MATH  MathSciNet  Google Scholar 

  • de Boor C. (2001). A practical guide to splines. New York, Springer

    MATH  Google Scholar 

  • Fan J., Chen J. (1999). One-step local quasi-likelihood estimation. Journal of the Royal Statistical Society: Series B 61: 927–934

    Article  MATH  MathSciNet  Google Scholar 

  • Fan J., Gijbels I. (1996). Local polynomial modelling and its applications. London, Chapman and Hall

    MATH  Google Scholar 

  • Fan J., Härdle W., Mammen E. (1998). Direct estimation of low-dimensional components in additive models. Annals of Statistics 26, 943–971

    Article  MATH  MathSciNet  Google Scholar 

  • Hall P., Titterington D.M. (1988). On confidence bands in nonparametric density estimation and regression. Journal of Multivariate Analysis 27, 228–254

    Article  MATH  MathSciNet  Google Scholar 

  • Härdle W. (1989). Asymptotic maximal deviation of M-smoothers. Journal of Multivariate Analysis 29, 163–179

    Article  MATH  MathSciNet  Google Scholar 

  • Härdle W. (1990). Applied nonparametric regression. Cambridge, Cambridge University Press

    MATH  Google Scholar 

  • Härdle W., Hlávka Z., Klinke S. (2000). XploRe application guide. Berlin, Springer

    MATH  Google Scholar 

  • Härdle W., Huet S., Mammen E., Sperlich S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20, 265–300

    Article  MATH  MathSciNet  Google Scholar 

  • Härdle W., Sperlich S., Spokoiny V. (2001). Structural tests in additive regression. Journal of the American Statistical Association 96, 1333–1347

    Article  MATH  MathSciNet  Google Scholar 

  • Harrison D., Rubinfeld D.L. (1978). Hedonic housing prices and the demand for cleaning air. Journal of Economics and Management 5, 81–102

    Article  MATH  Google Scholar 

  • Hastie T.J., Tibshirani R.J. (1990). Generalized additive models. London, Chapman and Hall

    MATH  Google Scholar 

  • Horowitz J.L., Mammen E. (2004). Nonparametric estimation of an additive model with a link function. Annals of Statistics 32, 2412–2443

    Article  MATH  MathSciNet  Google Scholar 

  • Horowitz J.L., Klemelä J., Mammen E. (2006). Optimal estimation in additive regression models. Bernoulli 12, 271–298

    Article  MATH  MathSciNet  Google Scholar 

  • Huang J.Z. (1998). Projection estimation in multiple regression with application to functional ANOVA models. Annals of Statistics 26, 242–272

    Article  MATH  MathSciNet  Google Scholar 

  • Huang J.Z. (2003). Local asymptotics for polynomial spline regression. Annals of Statistics 31, 1600–1635

    Article  MATH  MathSciNet  Google Scholar 

  • Huang J.Z., Yang L. (2004). Identification of nonlinear additive autoregression models. Journal of the Royal Statistical Society: Series B 66, 463–477

    Article  MATH  MathSciNet  Google Scholar 

  • Kim W., Linton O.B., Hengartner N. (1999). A computationally efficient oracle estimator for additive nonparametric regression with bootstrap confidence intervals. Journal of Computational and Graphical Statistics 8, 278–297

    Article  MathSciNet  Google Scholar 

  • Linton O.B., Nielsen J.P. (1995). Estimating structured nonparametric regression models by the kernel method. Biometrika 82, 93–101

    Article  MATH  MathSciNet  Google Scholar 

  • Linton O.B., Härdle W. (1996). Estimating additive regression models with known links. Biometrika 83, 529–540

    Article  MATH  MathSciNet  Google Scholar 

  • Linton O.B. (1997). Efficient estimation of additive nonparametric regression models. Biometrika 84, 469–473

    Article  MATH  MathSciNet  Google Scholar 

  • Mammen E., Linton O., Nielsen J. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Annals of Statistics 27, 1443–1490

    MATH  MathSciNet  Google Scholar 

  • Nielsen J.P., Sperlich S. (2005). Smooth backfitting in practice. Journal of the Royal Statistical Society: Series B 67, 43–61

    Article  MATH  MathSciNet  Google Scholar 

  • Opsomer J.D. (2000). Asymptotic properties of backfitting estimators. Journal of Multivariate Analysis 73, 166–179

    Article  MATH  MathSciNet  Google Scholar 

  • Opsomer J.D., Ruppert D. (1997). Fitting a bivariate additive model by local polynomial regression. Annals of Statistics 25, 186–211

    Article  MATH  MathSciNet  Google Scholar 

  • Sperlich S., Tjøstheim D., Yang L. (2002). Nonparametric estimation and testing of interaction in additive models. Econometric Theory 18, 197–251

    Article  MATH  MathSciNet  Google Scholar 

  • Stone C.J. (1985). Additive regression and other nonparametric models. Annals of Statistics 13, 689–705

    Article  MATH  MathSciNet  Google Scholar 

  • Stone C.J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Annals of Statistics 22, 118–184

    Article  MATH  MathSciNet  Google Scholar 

  • Tjøstheim D., Auestad B. (1994). Nonparametric identification of nonlinear time series: projections. Journal of the American Statistical Association 89, 1398–1409

    Article  MathSciNet  Google Scholar 

  • Tusnády G. (1977). A remark on the approximation of the sample df in the multidimensional case. Periodica Mathematica Hungarica 8, 53–55

    Article  MATH  MathSciNet  Google Scholar 

  • Wang, J., Yang, L. (2007a). Polynomial spline confidence bands for regression curves. Manuscript.

  • Wang, J., Yang, L. (2007b). Efficient and fast spline-backfitted kernel smoothing of additive models. http://www.stt.msu.edu/~yangli/SBKAISMfull.pdf.

  • Xia Y. (1998). Bias-corrected confidence bands in nonparametric regression. Journal of the Royal Statistical Society: Series B 60, 797–811

    Article  MATH  Google Scholar 

  • Xue L., Yang L. (2006a). Additive coefficient modeling via polynomial spline. Statistica Sinica 16, 1423–1446

    MathSciNet  Google Scholar 

  • Xue L., Yang L. (2006b). Estimation of semiparametric additive coefficient model. Journal of Statistical Planning and Inference 136, 2506–2534

    Article  MATH  MathSciNet  Google Scholar 

  • Yang, L. (2007). Confidence band for additive regression model. Journal of Data Science, forthcoming.

  • Yang L., Härdle W., Nielsen J.P. (1999). Nonparametric autoregression with multiplicative volatility and additive mean. Journal of Time Series Analysis 20, 579–604

    Article  MATH  MathSciNet  Google Scholar 

  • Yang L., Sperlich S., Härdle W. (2003). Derivative estimation and testing in generalized additive models. Journal of Statistical Planning and Inference 115: 521–542

    Article  MATH  MathSciNet  Google Scholar 

  • Yang L., Park B.U., Xue L., Härdle W. (2006). Estimation and testing of varying coefficients in additive models with marginal integration. Journal of the American Statistical Association 101: 1212–1227

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, Chicago, IL, 60607, USA

    Jing Wang

  2. Department of Statistics and Probability, Michigan State University, East Lansing, MI, 48824, USA

    Lijian Yang

Authors
  1. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lijian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Wang.

Additional information

Supported in part by NSF awards DMS 0405330, 0706518, BCS 0308420 and SES 0127722.

About this article

Cite this article

Wang, J., Yang, L. Efficient and fast spline-backfitted kernel smoothing of additive models. Ann Inst Stat Math 61, 663–690 (2009). https://doi.org/10.1007/s10463-007-0157-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-007-0157-x

Keywords

Search

Navigation