Skip to main content

Information and Likelihood Theory: A Basis for Model Selection and Inference

  • Chapter

Abstract

Full reality cannot be included in a model; thus we seek a good model to approximate the effects or factors supported by the empirical data. The selection of an appropriate approximating model is critical to statistical inference from many types of empirical data. This chapter introduces concepts from information theory (see Guiasu 1977), which has been a discipline only since the mid-1940s and covers a variety of theories and methods that are fundamental to many of the sciences (see Cover and Thomas 1991 for an exciting overview; Figure 2.1 is produced from their book and shows their view of the relationship of information theory to several other fields). In particular, the Kullback—Leibler “distance,” or “information,” between two models (Kull-back and Leibler 1951) is introduced, discussed, and linked to Boltzmann’s entropy in this chapter. Akaike (1973) found a simple relationship between the Kullback—Leibler distance and Fisher’s maximized log-likelihood function (see deLeeuw 1992 for a brief review). This relationship leads to a simple, effective, and very general methodology for selecting a parsimonious model for the analysis of empirical data.

Keywords

  • Model Selection
  • Candidate Model
  • Bootstrap Sample
  • Akaike Weight
  • Full Reality

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-0-387-22456-5_2
  • Chapter length: 49 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-0-387-22456-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2002 Springer-Verlag New York, Inc.

About this chapter

Cite this chapter

(2002). Information and Likelihood Theory: A Basis for Model Selection and Inference. In: Burnham, K.P., Anderson, D.R. (eds) Model Selection and Multimodel Inference. Springer, New York, NY. https://doi.org/10.1007/978-0-387-22456-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-22456-5_2

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-0-387-95364-9

  • Online ISBN: 978-0-387-22456-5

  • eBook Packages: Springer Book Archive