Abstract
Full reality cannot be included in a model; thus we seek a good model to approximate the effects or factors supported by the empirical data. The selection of an appropriate approximating model is critical to statistical inference from many types of empirical data. This chapter introduces concepts from information theory (see Guiasu 1977), which has been a discipline only since the mid-1940s and covers a variety of theories and methods that are fundamental to many of the sciences (see Cover and Thomas 1991 for an exciting overview; Figure 2.1 is produced from their book and shows their view of the relationship of information theory to several other fields). In particular, the Kullback—Leibler “distance,” or “information,” between two models (Kull-back and Leibler 1951) is introduced, discussed, and linked to Boltzmann’s entropy in this chapter. Akaike (1973) found a simple relationship between the Kullback—Leibler distance and Fisher’s maximized log-likelihood function (see deLeeuw 1992 for a brief review). This relationship leads to a simple, effective, and very general methodology for selecting a parsimonious model for the analysis of empirical data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag New York, Inc.
About this chapter
Cite this chapter
(2002). Information and Likelihood Theory: A Basis for Model Selection and Inference. In: Burnham, K.P., Anderson, D.R. (eds) Model Selection and Multimodel Inference. Springer, New York, NY. https://doi.org/10.1007/978-0-387-22456-5_2
Download citation
DOI: https://doi.org/10.1007/978-0-387-22456-5_2
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-95364-9
Online ISBN: 978-0-387-22456-5
eBook Packages: Springer Book Archive