Skip to main content
Log in

Nonparametric regression using linear combinations of basis functions

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

This paper discusses a Bayesian approach to nonparametric regression initially proposed by Smith and Kohn (1996. Journal of Econometrics 75: 317–344). In this approach the regression function is represented as a linear combination of basis terms. The basis terms can be univariate or multivariate functions and can include polynomials, natural splines and radial basis functions. A Bayesian hierarchical model is used such that the coefficient of each basis term can be zero with positive prior probability. The presence of basis terms in the model is determined by latent indicator variables. The posterior mean is estimated by Markov chain Monte Carlo simulation because it is computationally intractable to compute the posterior mean analytically unless a small number of basis terms is used. The present article updates the work of Smith and Kohn (1996. Journal of Econometrics 75: 317–344) to take account of work by us and others over the last three years. A careful discussion is given to all aspects of the model specification, function estimation and the use of sampling schemes. In particular, new sampling schemes are introduced to carry out the variable selection methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Dennison D.G.T., Mallick B.K., and Smith A.F.M. 1998. Automatic Bayesian curve fitting. Journal of the Royal Statistical Society B 60: 333–350.

    Google Scholar 

  • Friedman J.H. 1991. Multivariate adaptive regression splines. The Annals of Statistics 19: 1–141.

    Google Scholar 

  • Friedman J.H. and Silverman B.W. 1989. Flexible parsimonious smoothing and additive modeling. Technometrics 31: 3–39.

    Google Scholar 

  • George E.I. and McCulloch R.E. 1993. Variable selection via Gibbs sampling. Journal of the American Statistical Association 88: 881–889.

    Google Scholar 

  • George E.I. and McCulloch R.E. 1997. Approaches for Bayesian variable selection. Statistica Sinica 7: 339–373.

    Google Scholar 

  • Green P.J. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.

    Google Scholar 

  • Hastie T.J. and Tibshirani R.J. 1990. Generalized Additive Models. Chapman Hall, New York.

    Google Scholar 

  • Hastings W.K. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57: 97–109.

    Google Scholar 

  • Holmes C.C. and Mallick B.K. 1998. Radial basis functions of variable dimension. Neural Computation 10: 1217–1233.

    Google Scholar 

  • Mitchell T.J. and Beauchamp J.J. 1988. Bayesian variable selection in linear regression. Journal of the American Statistical Association 83: 1023–1036.

    Google Scholar 

  • O'Hagan A. 1995. Fractional Bayes factors for model comparison (with discussion). Journal of the Royal Statistical Society B 57: 99–138.

    Google Scholar 

  • Peskun P.H. 1973. Optimum Monte Carlo sampling using Markov chains. Biometrika 60: 607–612.

    Google Scholar 

  • Powell M.J.D. 1987. Radial basis functions for multivariate interpolation. In: Mason J.M. and Cox M. (Eds.), Algorithms for Approximation. Oxford University Press, Oxford, pp. 143–167.

    Google Scholar 

  • Smith M. and Kohn R. 1996. Nonparametric regression via Bayesian variable selection. Journal of Econometrics. 75: 317–344.

    Google Scholar 

  • Smith M. and Kohn R. 1997a. A Bayesian approach to nonparametric bivariate regression. Journal of the American Statistical Association 92: 1522–1535.

    Google Scholar 

  • Smith M. and Kohn R. 2000. Nonparametric seemingly unrelated regression. Journal of Econometrics 98: 257–281.

    Google Scholar 

  • Stone C., Hansen M., Kooperberg C., and Truong Y. 1996. Polynomial splines and their tensor products in extended linear modeling. Annals of Statistics 25: 1371–1470.

    Google Scholar 

  • Wahba G. 1990. Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59. SIAM, Philadelphia.

    Google Scholar 

  • Zellner A. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In: P.K. Goel and A. Zellner (Eds.), Bayesian Inference and Decision Techniques—Essays in Honor of Bruno de Finetti. North Holland, Amsterdam, 11:233–243.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kohn, R., Smith, M. & Chan, D. Nonparametric regression using linear combinations of basis functions. Statistics and Computing 11, 313–322 (2001). https://doi.org/10.1023/A:1011916902934

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011916902934

Navigation