Skip to main content
SpringerLink
Log in

Information Theory: Overview

  • Reference work entry
  • First Online:
Encyclopedia of Computational Neuroscience

  • 46 Accesses

Definition

Information Theory started with Shannon’s seminal paper “A Mathematical Theory of Communication” (Shannon 1948). Because its importance and flexibility were quickly recognized, there were numerous attempts to apply it to diverse field outside of its original scope. The entries in this section provide an overview of the current state of Information Theory in Neuroscience.

Detailed Description

When discussing a field, it is useful to review the basic concepts and their properties. That has been provided in multiple articles for Information Theory. To assist the reader, the entry “Summary of Information-Theoretic Quantities” provides a brief summary of the main information-theoretic quantities. For a more thorough investigation, we would direct interested readers to the most excellent introduction to Information Theory by Cover and Thomas (2006), now in its second edition.

Very soon after Shannon’s initial publication (1948), a small number of manuscripts provided the...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Abeles M, Lass Y (1975) Transmission of information by the axon: II. The channel capacity. Biol Cybern 19:121–125

    Article  CAS  PubMed  Google Scholar 

  • Attneave F (1954) Some information aspects of visual perception. Psychol Rev 61:183–193

    Article  CAS  PubMed  Google Scholar 

  • Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith WA (ed) Sensory communications. MIT Press, Cambridge, MA

    Google Scholar 

  • Cover T, Thomas J (2006) Elements of information theory. In: Wiley series in communication and signal processing, 2nd edn. Wiley, Hoboken

    Google Scholar 

  • de Ruyter van Steveninck R, Bialek W (1988) Real-time performance of a movement- sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc Royal Soc Ser B Biol Sci 234(1277):379–414

    Google Scholar 

  • Dimitrov AG, Lazar AA, Victor JD (2011) Information theory in neuroscience. J Comput Neurosci 30(1):1–5

    Article  PubMed  PubMed Central  Google Scholar 

  • Eagles JP, Purple RL (1974) Afferent fibers with multiple encoding sites. Brain Res 77(2):187–193

    Article  CAS  PubMed  Google Scholar 

  • Eckhorn R, Pöpel B (1974) Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Biol Cybern 16:191–200

    CAS  Google Scholar 

  • Granger C (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424–438

    Article  Google Scholar 

  • Harvey R (1978) Patterns of output firing generated by a many-input neuronal model for different model parameters and patterns of synaptic drive. Brain Res 150(2):259–276

    Article  CAS  PubMed  Google Scholar 

  • MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135

    Article  Google Scholar 

  • Massey J (1990) Causality, feedback and directed information. In: Proceedings of the international symposium on information theory applications (ISITA-90). Yokohama National University, Yokohama, pp 303–305

    Google Scholar 

  • Massey J, Massey P (2005) Conservation of mutual and directed information. In: Proceedings of the international symposium on information theory (ISIT 2005). IEEE Publishing, New York NY, pp 157–158

    Google Scholar 

  • Milenkovic O, Alterovitz G, Battail G, Coleman TP, Hagenauer J, Meyn SP, Price N, Ramoni MF, Shmulevich I, Szpankowski W (2010) Introduction to the special issue on information theory in molecular biology and neuroscience. IEEE Trans Inf Theory 56(2):649–652

    Article  Google Scholar 

  • Miller GA (1955) Note on the bias of information estimates. In: Henry Q (ed) Information theory in psychology: problems and methods, vol vol II-B. Free Press, Glenco, IL, pp 95–100

    Google Scholar 

  • Norwich K (1977) On the information received by sensory receptors. Bull Math Biol 39:453–461

    Article  CAS  PubMed  Google Scholar 

  • Poussart DJM (1971) Membrane current noise in lobster axon under voltage clamp. Biophys J 11(2):211–234

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:623–656

    Article  Google Scholar 

  • Stark L, Negrete-Martinze J, Yankelevich G, Theodoridis G (1969) Experiments on information coding in nerve impulse trains. Math Biosci 4(3–4):451–485

    Article  Google Scholar 

  • Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys J 7(6):797–826

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Taylor RC (1975) Integration in the crayfish antennal neuropile: topographic representation and multiple-channel coding of mechanoreceptive submodalities. Dev Neurobiol 6(5):475–499

    Article  CAS  Google Scholar 

  • Walloe L (1970) On the transmission of information through sensory neurons. Biophys J 10(8):745–763

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Washington State University, Vancouver, WA, USA

    Alexander Dimitrov

Authors
  1. Alexander Dimitrov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Dimitrov .

Editor information

Editors and Affiliations

  1. Emory University, Atlanta, GA, USA

    Dieter Jaeger

  2. University of Arkansas, Fayetteville, AR, USA

    Ranu Jung

Section Editor information

  1. National Research Council, Institute of Biophysics, Palermo, Italy

    Michele Migliore

  2. Computational Physiology Lab, Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA

    Christiane Linster

  3. Department of Biology, Emory University, Atlanta, GA, USA

    Francesco Cavarretta

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Science+Business Media, LLC, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Dimitrov, A. (2022). Information Theory: Overview. In: Jaeger, D., Jung, R. (eds) Encyclopedia of Computational Neuroscience. Springer, New York, NY. https://doi.org/10.1007/978-1-0716-1006-0_751

Download citation

Publish with us

Policies and ethics

Search

Navigation