Skip to main content
SpringerLink
Log in

A new theorem of information theory

  • Articles
  • Published:
Journal of Statistical Physics Aims and scope Submit manuscript

Abstract

Consider a random experiment whose possible outcomes arez 1,z 2,...,z n. Let the prior probabilities be p1 0, ...,pn 0, and let the posterior probabilities bep 1,...,p n. It is shown that, subject to certain prescribed and intuitively reasonable conditions, the expressionI =k ⌆ p i In (p i/p i 0), wherek is a positive constant, is the unique expression for the information contained in a message which alters the probabilities from thep i 0 to thep i.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. C.E. Shannon,Bell System Tech. J. 27, 379 and 623 (1948). Reprinted in C. E. Shannon and W. Weaver,The Mathematical Theory of Communication (University of Illinois Press, Urbana, Illinois, 1949),

    Google Scholar 

  2. A. Feinstein,Foundations of Information Theory (McGraw-Hill Book Company, Inc., New York, 1958).

    Google Scholar 

  3. A. I. Khinchin,Mathematical Foundations of Information Theory (Dover Publications, Inc., New York, 1957).

    Google Scholar 

  4. F. M. Reza,An Introduction to Information Theory (McGraw-Hill Book Company, Inc., New York, 1961).

    Google Scholar 

  5. A. Katz,Principles of Statistical Mechanics: The Information Theory Approach (W. H. Freeman and Company, San Francisco, 1967).

    Google Scholar 

  6. S. Kullback,Information Theory and Statistics (John Wiley & Sons, Inc., New York, 1959).

    Google Scholar 

  7. A. Renyi,Wahrscheinlichkeitrechnung (VEB Deutscher Verlag der Wissenschaften, Berlin, 1966).

    Google Scholar 

  8. M. S. Pinsker,Information and Information Stability of Random Variables and Processes, translated and edited by A. Feinstein (Holden-Day, Inc., San Francisco, 1964).

    Google Scholar 

  9. F. Schlögl, inStatistical Mechanics (Proc. of the IUPAP Meeting, Copenhagen, 1966), edited by T. A. Bak (W. A. Benjamin, Inc., New York, 1967); inProc. of the International Conf. on Statistical Mechanics (Kyoto, 1968) (supplement toJ. Phys. Soc. Japan 26, 1969).

    Google Scholar 

  10. A. Hobson,Concepts in Statistical Mechanics (Gordon and Breach, Science Publishers, Inc., 1970).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Physics, University of Arkansas, Fayetteville, Arkansas

    Arthur Hobson

Authors
  1. Arthur Hobson
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hobson, A. A new theorem of information theory. J Stat Phys 1, 383–391 (1969). https://doi.org/10.1007/BF01106578

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01106578

Key words

Search

Navigation