Encyclopedia of Neuroscience

Editors: Marc D. Binder, Nobutaka Hirokawa, Uwe Windhorst

Information Theory

  • Yoav Tock
  • Yossef Steinberg
Reference work entry
DOI: https://doi.org/10.1007/978-3-540-29678-2_2440

Synonyms

A mathematical theory of communication; Shannon theory

Definition

Information theory is above all a branch of communication theory. According to Claude Shannon [1]: “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” These two points may be separated in time and/or in space. Information theory addresses this problem and tells us what the ultimate rate of communication is over a noisy channel. Information theory also tells us what the ultimate data compression rate is. In general, information theory is concerned with stating what can and cannot be done in various communication settings. The theory also suggests schemes to achieve those limits and analyses the performance of those schemes. The insight and principles embodied in the theory form the foundation for virtually all modern communication systems.

The enormous influence of information theory on manmade communication systems...

This is a preview of subscription content, log in to check access

References

  1. 1.
    Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, 623–656Google Scholar
  2. 2.
    MacKay D, McKulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135CrossRefGoogle Scholar
  3. 3.
    Rieke F, Warland D, de Ruyter van Stevenink R, Bialek W (1997) Spikes: exploring the neural code. MIT Press, Cambridge, MAGoogle Scholar
  4. 4.
    de Ruyter van Steveninck RR, Laughlin SB (1996) The rate of information transfer at graded-potential synapses. Nature 379:642–645CrossRefGoogle Scholar
  5. 5.
    London M, Schreibman A, Häusser M, Larku m ME, Segev I (2002) The information efficacy of a synapse. Nat Neurosci 5:332–340PubMedCrossRefGoogle Scholar
  6. 6.
    Strong SP, Koberle R, de Ruyter van Steveninck RR, Bialek W (1998) Entropy and information in neural spike trains. Phys Rev Lett 80:197–200CrossRefGoogle Scholar
  7. 7.
    Tock Y (2004) Information theoretic analysis of the muscle spindle system, PhD thesis. Technion – Israel Institute of Technology, Haifa, IsraelGoogle Scholar
  8. 8.
    Tock Y, Inbar GF, Steinberg Y, Ljubisavljevic M, Thunberg J, Windhorst U, Johansson H (2005) Estimation of muscle spindle information rate by pattern matching and the effect of the gamma system activity on parallel spindles. Biol Cybern:92:316–332PubMedCrossRefGoogle Scholar
  9. 9.
    Gallager RG (1968) Information theory and reliable communication. Wiley, New YorkGoogle Scholar
  10. 10.
    Cover TM, Thomas JA (1990) Elements of information theory, Wiley, New YorkGoogle Scholar

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2009

Authors and Affiliations

  • Yoav Tock
    • 1
  • Yossef Steinberg
    • 2
  1. 1.IBM Haifa Research LaboratoryUniversity CampusHaifaIsrael
  2. 2.Electrical Engineering DepartmentTechnionHaifaIsrael