Abstract
This paper discusses a Bayesian approach to nonparametric regression initially proposed by Smith and Kohn (1996. Journal of Econometrics 75: 317–344). In this approach the regression function is represented as a linear combination of basis terms. The basis terms can be univariate or multivariate functions and can include polynomials, natural splines and radial basis functions. A Bayesian hierarchical model is used such that the coefficient of each basis term can be zero with positive prior probability. The presence of basis terms in the model is determined by latent indicator variables. The posterior mean is estimated by Markov chain Monte Carlo simulation because it is computationally intractable to compute the posterior mean analytically unless a small number of basis terms is used. The present article updates the work of Smith and Kohn (1996. Journal of Econometrics 75: 317–344) to take account of work by us and others over the last three years. A careful discussion is given to all aspects of the model specification, function estimation and the use of sampling schemes. In particular, new sampling schemes are introduced to carry out the variable selection methodology.
Similar content being viewed by others
References
Dennison D.G.T., Mallick B.K., and Smith A.F.M. 1998. Automatic Bayesian curve fitting. Journal of the Royal Statistical Society B 60: 333–350.
Friedman J.H. 1991. Multivariate adaptive regression splines. The Annals of Statistics 19: 1–141.
Friedman J.H. and Silverman B.W. 1989. Flexible parsimonious smoothing and additive modeling. Technometrics 31: 3–39.
George E.I. and McCulloch R.E. 1993. Variable selection via Gibbs sampling. Journal of the American Statistical Association 88: 881–889.
George E.I. and McCulloch R.E. 1997. Approaches for Bayesian variable selection. Statistica Sinica 7: 339–373.
Green P.J. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.
Hastie T.J. and Tibshirani R.J. 1990. Generalized Additive Models. Chapman Hall, New York.
Hastings W.K. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57: 97–109.
Holmes C.C. and Mallick B.K. 1998. Radial basis functions of variable dimension. Neural Computation 10: 1217–1233.
Mitchell T.J. and Beauchamp J.J. 1988. Bayesian variable selection in linear regression. Journal of the American Statistical Association 83: 1023–1036.
O'Hagan A. 1995. Fractional Bayes factors for model comparison (with discussion). Journal of the Royal Statistical Society B 57: 99–138.
Peskun P.H. 1973. Optimum Monte Carlo sampling using Markov chains. Biometrika 60: 607–612.
Powell M.J.D. 1987. Radial basis functions for multivariate interpolation. In: Mason J.M. and Cox M. (Eds.), Algorithms for Approximation. Oxford University Press, Oxford, pp. 143–167.
Smith M. and Kohn R. 1996. Nonparametric regression via Bayesian variable selection. Journal of Econometrics. 75: 317–344.
Smith M. and Kohn R. 1997a. A Bayesian approach to nonparametric bivariate regression. Journal of the American Statistical Association 92: 1522–1535.
Smith M. and Kohn R. 2000. Nonparametric seemingly unrelated regression. Journal of Econometrics 98: 257–281.
Stone C., Hansen M., Kooperberg C., and Truong Y. 1996. Polynomial splines and their tensor products in extended linear modeling. Annals of Statistics 25: 1371–1470.
Wahba G. 1990. Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59. SIAM, Philadelphia.
Zellner A. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In: P.K. Goel and A. Zellner (Eds.), Bayesian Inference and Decision Techniques—Essays in Honor of Bruno de Finetti. North Holland, Amsterdam, 11:233–243.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kohn, R., Smith, M. & Chan, D. Nonparametric regression using linear combinations of basis functions. Statistics and Computing 11, 313–322 (2001). https://doi.org/10.1023/A:1011916902934
Issue Date:
DOI: https://doi.org/10.1023/A:1011916902934