Abstract
One of the fundamental steps in statistical modeling is to select the best-fitting model from a set of candidate models for given data. In this paper, based on Bayesian decision theory, we introduce a new model selection criterion, called Bregman divergence criterion (BDC). The proposed criterion improves many existing Bayesian model selection methods such as Bayes factor, intrinsic Bayes factor, pseudo-Bayes factor, etc. In addition, using a Monte Carlo approach, we develop an efficient estimator that significantly eases the computational burden associated with our approach and prove its consistency. The versatility of our methodology is demonstrated on both simulated and real data; to this end, some illustrative examples are provided for linear regression models and longitudinal data models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berger, J. O., & Pericchi, L. R. (1996a). The intrinsic Bayes factor for linear models. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. M. Smith (Eds.), Bayesian statistics (vol. 5, pp. 25–44). Oxford: Oxford University Press.
Berger, J. O., & Pericchi, L. R. (1996b). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91, 109–122.
Bernardo, J. M., & Smith, A. F. M. (2000). Bayesian theory. Chichester: Wiley.
Bregman, L. M. (1967). The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7, 200–217.
Chen, M. H. (1994). Importance-weighted marginal Bayesian posterior density estimation. Journal of the American Statistical Association, 89, 818–824.
Dawid, A. P. (1984). Statistical theory: The prequential approach. Journal of the Royal Statistical Society, Series B, 147, 278–292.
Geisser, S. (1975). The predictive sample reuse method with applications. Journal of the American Statistical Association, 70, 320–328.
Geisser, S., & Eddy, W. (1979). A predictive approach to model selection. Journal of the American Statistical Association, 74, 153–160.
Gelfand, A. E., & Dey, D. K. (1994). Bayesian model choice: Asymptotics and exact calculations. Journal of the Royal Statistical Society, Series B, 56, 501–514.
Gelfand, A. E., & Ghosh, S. K. (1998). Model choice: A minimum posterior predictive loss approach. Biometrika, 85, 1–11.
Gelfand, A. E., Hills, S. E., Racine-Poon, A., & Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association, 85, 972–985.
Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477), 359–378.
Goh, G., & Dey, D. K. (2014). Bayesian model diagnostics using functional Bregman divergence. Journal of Multivariate Analysis, 124, 371–383.
Grünwald, P. D., & Dawid, A. P. (2004). Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. Annals of Statistics, 32, 1367–1433.
Gutierrez-Pena, E., & Walker, S. G. (2001). A Bayesian predictive approach to model selection. Journal of Statistical Planning and Inference, 93, 259–276.
Hennequin, R., David, B., & Badeau, R. (2011). Beta-divergence as a subclass of Bregman divergence. IEEE Signal Processing Letters, 18, 83–86.
Ionides, E. L. (2008). Truncated importance sampling. Journal of Computational and Graphical Statistics, 17, 295–311.
Itakura, F., & Saito, S. (1970). A statistical method for estimation of speech spectral density and formant frequencies. Electronics and Communications in Japan, 53, 36–43.
Laud, P. W., & Ibrahim, J. G. (1995). Predictive model selection. Journal of the Royal Statistical Society, Series B, 57, 247–262.
Liu, J., Nordman, D. J., & Meeker, W. Q. (2016). The number of MCMC draws needed to compute Bayesian credible bounds. The American Statistician, 70(3), 275–284.
San Martini, A., & Spezzaferri, F. (1984). A predictive model selection criterion. Journal of the Royal Statistical Society, Series B, 46, 296–303.
Stone, M. (1974). Cross-validation choice and assessment of statistical predictions (with discussion). Journal of the Royal Statistical Society, Series B, 36, 111–147.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Goh, G., Dey, D.K. (2021). Bayesian Model Assessment and Selection Using Bregman Divergence. In: Ghosh, I., Balakrishnan, N., Ng, H.K.T. (eds) Advances in Statistics - Theory and Applications. Emerging Topics in Statistics and Biostatistics . Springer, Cham. https://doi.org/10.1007/978-3-030-62900-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-62900-7_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62899-4
Online ISBN: 978-3-030-62900-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)