Abstract
In this chapter a number of measures of divergence are presented and the way model selection criteria are constructed via measures of divergence is discussed. The construction of the divergence information criterion based on a new family of measures of divergence is presented and the lower bound of the mean squared error of prediction is established. Some illustrative simulation results are also given.
Keywords and phrases
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. Proc. of the 2nd Intern. Symposium on Information Theory (Petrov, B. N. and Csaki, F., Eds.), Akademiai Kaid’o, Budapest.
Ali, S. M. and Silvey, S. D. (1966). A general class of coefficients of divergence of one distribution from another. J. R. Statist. Soc. B, 28, 131–142.
Basu, A., Harris, I. R., Hjort, N. L., and Jones, M. C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika, 85, 549–559.
Bengtsson, T. and Cavanaugh. J. E. (2006). An improved Akaike information criterion for state-space model selection. Comput. Statist. Data Anal., 50, 2635–2654.
Cavanaugh, J. E. (2004). Criteria for linear model selection based on Kullback’s symmetric divergence, Austral. N. Zeal. J. Statist., 46, 257–274.
Cressie, N. and Read, T. R. C. (1984). Multinomial goodness-of-fit tests. J. R. Statist. Soc., 5, 440–454.
Csiszar, I. (1963). Eine informationstheoretische ungleichung and ihre anwendung auf den beweis der ergodizitat von markoffischen ketten. Magyar Tud. Akad. Mat. Kutato Int. Kozl., 8, 85–108.
Hurvich, C. M. and Tsai, C. L. (1989). Regression and time series model selection in small samples. Biometrika, 76, 297–307.
Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. A, 186, 453–561.
Kagan, A. M. (1963). On the theory of Fisher’s amount of information (in Russian). Doklady Academii Nauk SSSR, 151, 277–278.
Konishi, S. and Kitagawa, G. (1996). Generalised information criteria in model selection. Biometrika, 83, 875–890.
Koukouvinos, C., Mylona, K., and Vonta, F. (2008). A comparative study of variable selection procedures applied in high dimensional medical problems. J. Appl. Prob. Statist., 3 (2) (195–209).
Kullback, S. and Leibler, R. (1951). On information and sufficiency. Ann. Math. Statist., 22, 79–86.
Mattheou, K., Lee, S., and Karagrigoriou, A. (2009). A model selection criterion based on the BHHJ measure of divergence. J. Statist. Plan. Infer. (139:128–135).
Renyi, A. (1961). On measures of entropy and information. Proc. 4th Berkeley Symp. on Math. Statist. Prob., 1, 547–561, University of California Press, Berkeley.
Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist., 6, 461–464.
Shang, J. (2008). Selection criteria based on Monte Carlo simulation and cross validation in mixed models. Far East J. Theor. Statist., 25, 51–72.
Shang, J. and Cavanaugh, J. E. (2008). Bootstrap variants of the Akaike information criterion for mixed model selection. Comput. Statist. Data Anal., 52, 2004–2021.
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. R. Statist. Soc. B, 64, 583–639.
Shibata, R. (1980). Asymptotically efficient selection of the order of the model for estimating parameters of linear process. Ann. Statist., 8, 147–164.
Shibata, R. (1981). An optimal selection of regression variables. Biometrika, 68, 45–54.
van der Linde, A. (2005). DIC in variable selection. Statist. Neerlandica, 59, 45–56.
Wei, C. Z. (1992). On predictive least squares principles. Ann. Statist., 20, 1–42.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Birkhäuser Boston
About this chapter
Cite this chapter
Karagrigoriou, A., Mattheou, K. (2010). Measures of Divergence in Model Selection. In: Skiadas, C. (eds) Advances in Data Analysis. Statistics for Industry and Technology. Birkhäuser Boston. https://doi.org/10.1007/978-0-8176-4799-5_6
Download citation
DOI: https://doi.org/10.1007/978-0-8176-4799-5_6
Published:
Publisher Name: Birkhäuser Boston
Print ISBN: 978-0-8176-4798-8
Online ISBN: 978-0-8176-4799-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)