Encyclopedia of Computational Neuroscience

2015 Edition
| Editors: Dieter Jaeger, Ranu Jung

Information Theory: Overview

  • Alexander Dimitrov
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-6675-8_751


Information Theory started with Shannon’s seminal paper “A Mathematical Theory of Communication” (Shannon 1948). Because its importance and flexibility were quickly recognized, there were numerous attempts to apply it to diverse field outside of its original scope. The entries in this section provide an overview of the current state of Information Theory in Neuroscience.

Detailed Description

When discussing a field, it is useful to review the basic concepts and their properties. That has been provided in multiple articles for Information Theory. To assist the reader, the entry “ Summary of Information-Theoretic Quantities” provides a brief summary of the main information-theoretic quantities. For a more thorough investigation, we would direct interested readers to the most excellent introduction to Information Theory by Cover and Thomas (2006), now in its second edition.

Very soon after Shannon’s initial publication (1948), a small number of manuscripts provided the...

This is a preview of subscription content, log in to check access.


  1. Abeles M, Lass Y (1975) Transmission of information by the axon: II. The channel capacity. Biol Cybern 19:121–125PubMedGoogle Scholar
  2. Attneave F (1954) Some information aspects of visual perception. Psychol Rev 61:183–193PubMedGoogle Scholar
  3. Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith WA (ed) Sensory communications. MIT Press, Cambridge, MAGoogle Scholar
  4. Cover T, Thomas J (2006) Elements of information theory, 2nd edn, Wiley series in communication and signal processing. Wiley, HobokenGoogle Scholar
  5. de Ruyter van Steveninck R, Bialek W (1988) Real-time performance of a movement- sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc Royal Soc Ser B Biol Sci 234(1277):379–414Google Scholar
  6. Dimitrov AG, Lazar AA, Victor JD (2011) Information theory in neuroscience. J Comput Neurosci 30(1):1–5PubMedCentralPubMedGoogle Scholar
  7. Eagles JP, Purple RL (1974) Afferent fibers with multiple encoding sites. Brain Res 77(2):187–193PubMedGoogle Scholar
  8. Eckhorn R, Pöpel B (1974) Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Biol Cybern 16:191–200Google Scholar
  9. Granger C (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424–438Google Scholar
  10. Harvey R (1978) Patterns of output firing generated by a many-input neuronal model for different model parameters and patterns of synaptic drive. Brain Res 150(2):259–276PubMedGoogle Scholar
  11. MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135Google Scholar
  12. Massey J (1990) Causality, feedback and directed information. In: Proceedings of the international symposium on information theory applications (ISITA-90), Yokohama National University, Yokohama, Japan, pp 303–305Google Scholar
  13. Massey J, Massey P (2005) Conservation of mutual and directed information. In: Proceedings of the international symposium on information theory (ISIT 2005), IEEE Publishing, New York, NY, pp 157–158Google Scholar
  14. Milenkovic O, Alterovitz G, Battail G, Coleman TP, Hagenauer J, Meyn SP, Price N, Ramoni MF, Shmulevich I, Szpankowski W (2010) Introduction to the special issue on information theory in molecular biology and neuroscience. IEEE Trans Inf Theory 56(2):649–652Google Scholar
  15. Miller GA (1955) Note on the bias of information estimates. In: Information theory in psychology: problems and methods. In: Henry Q (ed.), vol II-B. Free Press, Glenco, IL, pp 95–100Google Scholar
  16. Norwich K (1977) On the information received by sensory receptors. Bull Math Biol 39:453–461PubMedGoogle Scholar
  17. Poussart DJM (1971) Membrane current noise in lobster axon under voltage clamp. Biophys J 11(2):211–234PubMedCentralPubMedGoogle Scholar
  18. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:623–656Google Scholar
  19. Stark L, Negrete-Martinze J, Yankelevich G, Theodoridis G (1969) Experiments on information coding in nerve impulse trains. Math Biosci 4(3–4):451–485Google Scholar
  20. Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys J 7(6):797–826PubMedCentralPubMedGoogle Scholar
  21. Taylor RC (1975) Integration in the crayfish antennal neuropile: topographic representation and multiple-channel coding of mechanoreceptive submodalities. Dev Neurobiol 6(5):475–499Google Scholar
  22. Walloe L (1970) On the transmission of information through sensory neurons. Biophys J 10(8):745–763PubMedCentralPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of MathematicsWashington State UniversityVancouverUSA