Bayesian MARS

Statistics and Computing volume 8, pages337–346(1998)Cite this article

Abstract

A Bayesian approach to multivariate adaptive regression spline (MARS) fitting (Friedman, 1991) is proposed. This takes the form of a probability distribution over the space of possible MARS models which is explored using reversible jump Markov chain Monte Carlo methods (Green, 1995). The generated sample of MARS models produced is shown to have good predictive power when averaged and allows easy interpretation of the relative importance of predictors to the overall fit.

This is a preview of subscription content, log in to check access.

References

  1. Becker, R., Chambers, J. M. and Wilks, A. (1988) The New S Language, Wadsworth, Belmont, California.

  2. Breiman, L. and Friedman, J. H. (1985) Estimating optimal transformations for multiple regression and correlation (with discussion). Journal of the American Statistical Association, 80, 580–619.

    Google Scholar 

  3. Breiman, L., Friedman, J. H., Olshen, R. and Stone, C. J. (1984) Classification and Regression Trees, Wadsworth, Belmont, California.

  4. Bruntz, S. M., Cleveland, W. S., Kleiner, B. and Warner, J. L. (1974) The dependence of ambient ozone on solar radiation, temperature and mixing height, in Symposium on atmospheric diffusion and air pollution. American Meteorological Society, Boston, pp. 125–8.

    Google Scholar 

  5. Chipman, H., George, E. I. and McCulloch, R. E. (1998) Bayesian CART model search (with discussion). Journal of the American Statistical Association 93, 935–960.

    Google Scholar 

  6. Cleveland, W. S. and Devlin, S. J. (1988) Locally-weighted regression: an approach to regression analysis by local fitting. Journal of the American Statistical Association, 83, 597–610.

    Google Scholar 

  7. Craven, P. and Wahba, G. (1979) Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of cross-validation. Numerische Mathematik, 31, 317–403.

    Google Scholar 

  8. Denison, D. G. T. (1997) Simulation based Bayesian nonparametric regression methods. Unpublished Ph.D. Thesis. Imperial College, London.

  9. Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998a) A Bayesian CART algorithm. Biometrika, 85, 363–377.

    Google Scholar 

  10. Denison, D. G. T., Mallick, B. K. and Smith, A. F. M. (1998b) Automatic Bayesian curve fitting. Journal of the Royal Statistical Society, Series B, 60, 333–350.

    Google Scholar 

  11. Friedman, J. H. (1991) Multivariate adaptive regression splines (with discussion). The Annals of Statistics, 19, 1–141.

    Google Scholar 

  12. Friedman, J. H., Grosse, E. and Stuetzle, W. (1983) Multidimensional additive spline approximation. SIAM Journal of Scientific and Statistical Computing, 291–301.

  13. Friedman, J. H. and Stuetzle, W. (1981) Projection pursuit regression. Journal of the American Statistical Association, 76, 817–823.

    Google Scholar 

  14. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.

    Google Scholar 

  15. George, E. I. and McCulloch, R. E. (1993) Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889.

    Google Scholar 

  16. Green, P. J. (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 771–32.

    Google Scholar 

  17. Hastie, T. J. and Tibshirani, R. J. (1990) Generalized Additive Models, Chapman & Hall, London.

    Google Scholar 

  18. Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109.

    Google Scholar 

  19. Holmes, C. C. and Mallick, B. K. (1997) Bayesian wavelet networks for non-parametric regression. Technical report. Imperial College, London.

    Google Scholar 

  20. Hwang, J-N, Lay, S-R, Maechler, M., Martin, D. and Schimert, J. (1994) Regression modeling in back-propagation and projection pursuit learning. IEEE Transactions on Neural Networks, 5, 342–53.

    Google Scholar 

  21. Kass, R. E. and Raftery, A. E. (1995) Bayes factors. Journal of the American Statistical Association, 90, 773–795.

    Google Scholar 

  22. Mallick, B. K., Denison, D. G. T. and Smith, A. F. M. (1997) Bayesian survival analysis using a MARS model. Technical report. Imperial College, London.

    Google Scholar 

  23. Mallick, B. K., Denison, D. G. T. and Smith, A. F. M. (1998) Semiparametric generalized linear models: Bayesian approaches, in Generalized Linear Models: A Bayesian Perspective, Dey, D. K., Ghosh, S. K. and Mallick, B. K. (eds) Marcel-Dekker (to appear).

  24. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–1091.

    Google Scholar 

  25. Morgan, J. N. and Sonquist, J. A. (1963) Problems in the analysis of survey data and a proposal. Journal of the American Statistical Association, 58, 415–434.

    Google Scholar 

  26. O'Hagan, A. (1994) Kendall's Advanced Theory of Statistics, Volume 2B, Edward Arnold, London.

    Google Scholar 

  27. Richardson, S. and Green, P. J. (1997) Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society (Ser. B), 59, 731–792.

    Google Scholar 

  28. Roosen, C. B. and Hastie, T. J. (1994) Automatic smoothing spline projection pursuit. Journal of Computational and Graphical Statistics, 3, 235–48.

    Google Scholar 

  29. Tierney, L. (1994) Markov chains for exploring posterior distributions (with discussion). The Annals of Statistics, 22, 1701–1762.

    Google Scholar 

Download references

Author information

Affiliations

  1. Department of Mathematics, Imperial College of Science, Technology and Medicine, 180 Queen's Gate, London, SW7 2BZ, UK

    D. G. T. DENISON

  2. Department of Statistics, Texas A & M University, College Station, TX 77843-3143, USA

    B. K. MALLICK

  3. Principal, Queen Mary and Westfield College, London, E1 4NS, UK

    A. F. M. SMITH

Authors
  1. D. G. T. DENISON
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. K. MALLICK
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. F. M. SMITH
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

DENISON, D.G.T., MALLICK, B.K. & SMITH, A.F.M. Bayesian MARS. Statistics and Computing 8, 337–346 (1998). https://doi.org/10.1023/A:1008824606259

Download citation

  • Bayesian methods
  • reversible jump Markov Chain Monte Carlo
  • multiple regression
  • multivariate adaptive regression splines