Information Theory: Overview
Information Theory started with Shannon’s seminal paper “A Mathematical Theory of Communication” (Shannon 1948). Because its importance and flexibility were quickly recognized, there were numerous attempts to apply it to diverse field outside of its original scope. The entries in this section provide an overview of the current state of Information Theory in Neuroscience.
When discussing a field, it is useful to review the basic concepts and their properties. That has been provided in multiple articles for Information Theory. To assist the reader, the entry “ Summary of Information-Theoretic Quantities” provides a brief summary of the main information-theoretic quantities. For a more thorough investigation, we would direct interested readers to the most excellent introduction to Information Theory by Cover and Thomas (2006), now in its second edition.
Very soon after Shannon’s initial publication (1948), a small number of manuscripts provided the...
- Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith WA (ed) Sensory communications. MIT Press, Cambridge, MAGoogle Scholar
- Cover T, Thomas J (2006) Elements of information theory, 2nd edn, Wiley series in communication and signal processing. Wiley, HobokenGoogle Scholar
- de Ruyter van Steveninck R, Bialek W (1988) Real-time performance of a movement- sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc Royal Soc Ser B Biol Sci 234(1277):379–414Google Scholar
- Eckhorn R, Pöpel B (1974) Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Biol Cybern 16:191–200Google Scholar
- Granger C (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424–438Google Scholar
- MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135Google Scholar
- Massey J (1990) Causality, feedback and directed information. In: Proceedings of the international symposium on information theory applications (ISITA-90), Yokohama National University, Yokohama, Japan, pp 303–305Google Scholar
- Massey J, Massey P (2005) Conservation of mutual and directed information. In: Proceedings of the international symposium on information theory (ISIT 2005), IEEE Publishing, New York, NY, pp 157–158Google Scholar
- Milenkovic O, Alterovitz G, Battail G, Coleman TP, Hagenauer J, Meyn SP, Price N, Ramoni MF, Shmulevich I, Szpankowski W (2010) Introduction to the special issue on information theory in molecular biology and neuroscience. IEEE Trans Inf Theory 56(2):649–652Google Scholar
- Miller GA (1955) Note on the bias of information estimates. In: Information theory in psychology: problems and methods. In: Henry Q (ed.), vol II-B. Free Press, Glenco, IL, pp 95–100Google Scholar
- Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:623–656Google Scholar
- Stark L, Negrete-Martinze J, Yankelevich G, Theodoridis G (1969) Experiments on information coding in nerve impulse trains. Math Biosci 4(3–4):451–485Google Scholar
- Taylor RC (1975) Integration in the crayfish antennal neuropile: topographic representation and multiple-channel coding of mechanoreceptive submodalities. Dev Neurobiol 6(5):475–499Google Scholar