Abstract
In Chaps. 4 and 5 we explored Maximum Likelihood estimation and Bayesian statistics and, given a particular model, used our data to estimate the unknown parameter θ. The validity of the model itself was not questioned, except for a brief detour into the Bayes factor. In this chapter we will delve a little deeper into this idea and explore common routines for selecting the most appropriate model for the data. Before we start, let us make the goal of model selection a little more explicit.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akaike, H. 1973. Information theory as an extension of the maximum likelihood principle. In Proceedings of the 2nd International Symposium on Information Theory, 267–281.
Akaike, H. 1983. Information measures and model selection. International Statistical Institute 44: 277–291.
Ando, T. 2010. Bayesian Model Selection and Statistical Modeling. Boca Raton: Chapman and Hall/CRC.
Berger, J., and L. Pericchi. 2001. Objective bayesian methods for model selection: Introduction and comparison (with discussion). In Model Selection, ed. P. Lahiri, 135–207.
Berk, R., L. Brown, A. Buja, K. Zhang, and L. Zhao. 2013. Valid post-selection inference. Annals of Statistics 41: 802–837.
Breiman, L. 1992. The little bootstrap and other methods for dimensionality selection in regression: X-fixed prediction error. Journal of the American Statistical Association 87: 738–754.
Burnham, K.P., and D.R. Anderson. 2002. Model Selection and Multimodel Inference. Berlin: Springer.
Claekens, G., and N.L. Hjort. 2008. Model Selection and Model Averaging. Cambridge: Cambridge University Press.
Cui, W., and E. George. 2008. Empirical Bayes vs. fully Bayes variable selection. Journal of Statistical Planning and Inference 138: 888–900.
Hastie, T., R. Tibshirani, and M. Wainwright. 2015. Statistical Learning with Sparsity: The Lasso and Generalizations. Boca Raton: CRC Press.
Hoeting, J.A., D. Madigan, A.E. Raftery, and C.T. Volinsky. 1999. Bayesian model averaging: A tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors). Statistical Science 14(4): 382–417.
Hurvich, C.M., and C.-L. Tsai. 1989. Regression and time series model selection in small samples. Biometrika 76(2): 297.
Kabaila, P., A.H. Welsh, and W. Abeysekera. 2016. Model-averaged confidence intervals. Scandinavian Journal of Statistics 43(1): 35–48.
Konishi, S., and G. Kitagawa. 1996. Generalised information criteria in model selection. Biometrika 83(4): 875–890.
Leeb, H., and B. Pötscher. 2005. Model selection and inference: Facts and fiction. Econometric Theory 21: 21–59.
Leeb, H., B.M. Pötscher, and K. Ewald. 2015. On various confidence intervals post-model-selection. Statistical Science 30: 216–227.
Lockhart, R., J. Taylor, and R. Tibshirani. 2014. A significance test for the Lasso (with discussion). Annals of Statistics 42: 413–468.
Park, T., and G. Casella. 2008. The Bayesian Lasso. Journal of the American Statistical Association 103(482).
Shibata, R. 1989. Statistical Aspects of Model Selection. Berlin: Springer.
Spiegelhalter, D.J., N.G. Best, B.P. Carlin, and A. van der Linde. 2002. Bayesian measures of model complexity and fit. Journal of Royal Statistical Society 64: 583–639.
Symonds, M.R.E., and A. Moussalli. 2011. A brief guide to model selection, multimodel inference and model averaging in behavioural ecology using akaike’s information criterion. Behavioral Ecology and Sociobiology 65(91): 13–21.
Takeuchi, K. 1979. Distribution of informational statistics and a criterion of model fitting. Mathematical Sciences 153: 12–18.
Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58, 267–288.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kauermann, G., Küchenhoff, H., Heumann, C. (2021). Model Selection and Model Averaging. In: Statistical Foundations, Reasoning and Inference. Springer Series in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-69827-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-69827-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69826-3
Online ISBN: 978-3-030-69827-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)