Reference Work Entry

Encyclopedia of Neuroscience

pp 1964-1970

Information Theory

  • Yoav TockAffiliated withIBM Haifa Research Laboratory, University Campus
  • , Yossef SteinbergAffiliated withElectrical Engineering Department, Technion


A mathematical theory of communication; Shannon theory


Information theory is above all a branch of communication theory. According to Claude Shannon [1]: “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” These two points may be separated in time and/or in space. Information theory addresses this problem and tells us what the ultimate rate of communication is over a noisy channel. Information theory also tells us what the ultimate data compression rate is. In general, information theory is concerned with stating what can and cannot be done in various communication settings. The theory also suggests schemes to achieve those limits and analyses the performance of those schemes. The insight and principles embodied in the theory form the foundation for virtually all modern communication systems.

The enormous influence of information theory on manmade communicat ...

This is an excerpt from the content