Information, Novelty, and Surprise in Brain Theory

  • Günther Palm
Chapter

Abstract

In biological research it is common to assume that each organ of an organism serves a definite purpose. The purpose of the brain seems to be the coordination and processing of information which the animal obtains through its sense organs about the outside world and about its own internal state (Bateson 1972). An important aspect of this is the storage of information in memory and the use of the stored information in connection with the present sensory stimuli.

Keywords

Synaptic Plasticity Brain Research Spike Train Single Neuron Neural Population 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. .
    Abbott, L. F. (1994). Decoding neuronal firing and modeling neural networks. Quarterly Reviews of Biophysics, 27, 291–331.Google Scholar
  2. .
    Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press.Google Scholar
  3. .
    Abeles, M., & Gerstein, G. L. (1988). Detecting spatiotemporal firing patterns among simultaneously recorded single neurons. Journal of Neurophysiology, 60(3), 909–924.Google Scholar
  4. .
    Abeles, M., & Lass, Y. (1975). Transmission of information by the axon: II. The channel capacity. Biological Cybernetics, 19(3), 121–125.Google Scholar
  5. .
    Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70(4), 1629–1638.Google Scholar
  6. .
    Abeles, M., Bergman, H., Gat, I., Meilijson, I., Seidemann, E., Tishby, N., & Vaadia, E. (1995). Cortical activity flips among quasi stationary states. Proceedings of the National Academy of Sciences of the United States of America, 92, 8616–8620.Google Scholar
  7. .
    Adelman, T. L., Bialek, W., & Olberg, R. M. (2003). The information content of receptive fields. Neuron, 40(13), 823–833.Google Scholar
  8. .
    Aertsen, A. M. H. J., & Johannesma, P. I. M. (1981). The spectro-temporal receptive field. A functional characteristic of auditory neurons. Biological Cybernetics, 42(2), 133–143.Google Scholar
  9. .
    Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal firing correlation: Modulation of “effective connectivity”. Journal of Neurophysiology, 61(5), 900–917.Google Scholar
  10. .
    Amari, S.-i., & Nakahara, H. (2005). Difficulty of singularity in population coding. Neural Computation, 17, 839–858.Google Scholar
  11. .
    Amari, S., & Nakahara, H. (2006). Correlation and independence in the neural code. Neural Computation, 18(6), 1259–1267.Google Scholar
  12. .
    Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. M. H. J. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science, 273(5283), 1868–1871.Google Scholar
  13. .
    Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network: Computation in Neural Systems, 3, 213–251.Google Scholar
  14. .
    Atick, J. J., & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Computation, 2(3), 308–320.Google Scholar
  15. .
    Atick, J. J., & Redlich, A. N. (1992). What does the retina know about natural scenes? Cambridge: MIT Press.Google Scholar
  16. .
    Attneave, F. (1959). Applications of information theory to psychology. New York: Holt, Rinehart and Winston.Google Scholar
  17. .
    Bach, M., & Krüger, J. (1986). Correlated neuronal variability in monkey visual cortex revealed by a multi-microelectrode. Experimental Brain Research, 61(3), 451–456.Google Scholar
  18. .
    Bair, W., & Koch, C. (1996). Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey. Neural Computation, 8(6), 1185–1202.Google Scholar
  19. .
    Baker, S. N., & Lemon, R. N. (2000). Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. Journal of Neurophysiology, 84, 1770–1780.Google Scholar
  20. .
    Bar-Hillel, Y., & Carnap, R. (1953). Semantic information. In London information theory symposium (pp. 503–512). New York: Academic.Google Scholar
  21. .
    Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. Cambridge: MIT Press.Google Scholar
  22. .
    Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295–311.Google Scholar
  23. .
    Barlow, H. B., & Földiák, P. (1989). Adaptation and decorrelation in the cortex. In C. Miall, R. M. Durbin, & G. J. Mitcheson (Eds.), The computing neuron (pp. 54–72). USA: Addison-Wesley.Google Scholar
  24. .
    Barlow, H. B., Kaushal, T. P., & Mitchison, G. J. (1989). Finding minimum entropy codes. Neural Computation, 1(3), 412–423.Google Scholar
  25. .
    Barnard, G. A. (1955). Statistical calculation of word entropies for four Western languages. IEEE Transactions on Information Theory, 1(1), 49–53.Google Scholar
  26. .
    Bateson, G. (1972). Steps to an ecology of mind. London: Intertext Books.Google Scholar
  27. .
    Bell, A. J., & Sejnowski, T. J. (1995). An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159.Google Scholar
  28. .
    Bethge, M., Rotermund, D., & Pawelzik, K. (2002). Optimal short-term population coding: When Fisher information fails. Neural Computation, 14, 2317–2351.Google Scholar
  29. .
    Bi, G.-Q., & Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. The Journal of Neuroscience, 18, 10464–10472.Google Scholar
  30. .
    Bialek, W., de Ruyter van Steveninck, R. R., & Tishby, N. (2007). Efficient representation as a design principle for neural coding and computation. Neural Computation, 19(9), 2387-2432.Google Scholar
  31. .
    Bialek, W., Reike, F., de Ruyter van Steveninck, R. R., & Warland, D. (1991). Reading a neural code. Science, 252, 1854–1857.Google Scholar
  32. .
    Bliss, T. V. P., & Collingridge, G. L. (1993). A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361, 31–39.Google Scholar
  33. .
    Borst, A., & Theunissen, F. E. (1999). Information theory and neural coding. Nature Neuroscience, 2(11), 947–957.Google Scholar
  34. .
    Brenner, N., Strong, S., Koberle, R., Bialek, W., & de Ruyter van Steveninck, R. (2000). Synergy in a neural code. Neural Computation, 12(7), 1531–1552.Google Scholar
  35. .
    Brown, E. N., Kass, R. E., & Mitra, P. P. (2004). Multiple neural spike train data analysis: State-of-the-art and future challenges. Nature Neuroscience, 7, 456–461. doi: 10.1038/nn1228.Google Scholar
  36. .
    Brunel, N., & Nadal, J.-P. (1998). Mutual information, Fisher information, and population coding. Neural Computation, 10(7), 1731–1757.Google Scholar
  37. .
    Butts, D. A. (2003). How much information is associated with a particular stimulus? Network: Computation in Neural Systems, 14(2), 177–187.Google Scholar
  38. .
    Butts, D. A., & Goldman, M. (2006). Tuning curves, neuronal variability and sensory coding. PLOS Biology, 4, 639–646.Google Scholar
  39. .
    Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., & Stanley, G. B. (2007). Temporal precision in the neural code and the timescales of natural vision. Nature, 449(7158), 92–95.Google Scholar
  40. .
    Cessac, B., Rostro-González, H., Vasquez, J.-C., & Viéville, T. (2008). To which extend is the “neural code” a metric? In Proceedings of the conference NeuroComp 2008. Informal publication.Google Scholar
  41. .
    Cherry, C. (1966). On human communication. Cambridge: MIT Press.Google Scholar
  42. .
    Christodoulou, C., & Bugmann, G. (2001). Coefficient of variation (CV) vs mean inter-spike-interval (ISI) curves: What do they tell us about the brain? Neurocomputing, 38–40, 1141–1149.Google Scholar
  43. .
    Coulter, W. K., Hillar, C. J., & Sommer, F. T. (2009). Adaptive compressed sensing—a new class of self-organizing coding models for neuroscience.Google Scholar
  44. .
    Dan, Y., & Poo, M.-M. (2006). Spike timing-dependent plasticity: From synapse to perception. Physiology Review, 86, 1033–1048.Google Scholar
  45. .
    Dan, Y., Atick, J. J., & Reid, R. C. (1996). Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. Journal of Neuroscience, 16(10), 3351–3362.Google Scholar
  46. .
    Dayhoff, J. E., & Gerstein, G. L. (1983a). Favored patterns in spike trains. I. Detection. Journal of Neurophysiology, 49(6), 1334–1348.Google Scholar
  47. .
    Dayhoff, J. E., & Gerstein, G. L. (1983b). Favored patterns in spike trains. II. Application. Journal of Neurophysiology, 49(6), 1349–1363.Google Scholar
  48. .
    Deadwyler, S. A., & Hampson, R. E. (1997). The significance of neural ensemble codes during behavior and cognition. Annual Review of Neuroscience, 20, 217–244.Google Scholar
  49. .
    Dean, I., Harper, N. S., & D. McAlpine (2005). Neural population coding of sound level adapts to stimulus statistics. Nature Neuroscience, 8(12), 1684–1689.Google Scholar
  50. .
    Denève, S. (2008). Bayesian spiking neurons I: Inference. Neural Computation, 20, 91–117.Google Scholar
  51. .
    Dong, D. W., & Atick, J. J. (1995). Statistics of natural time-varying images. Network, 6(3), 345–358.Google Scholar
  52. .
    Doob, J. L. (1953). Stochastic Processes. New York: Wiley.Google Scholar
  53. .
    Eckhorn, R. (1999). Neural mechanisms of scene segmentation: Recordings from the visual cortex suggest basic circuits for linking field models. IEEE Transactions on Neural Networks, 10(3), 464–479.Google Scholar
  54. .
    Eckhorn, R., Grüsser, O.-J., Kröller, J., Pellnitz, K., & Pöpel, B. (1976). Efficiency of different neuronal codes: Information transfer calculations for three different neuronal systems. Biological Cybernetics, 22(1), 49–60.Google Scholar
  55. .
    Edelman, G. M., & Tononi, G. (2000). A universe of consciousness: How matter becomes imagination. New York: Basic Books.Google Scholar
  56. .
    Engel, A., Fries, P., & Singer, W. (2001). Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2(10), 704–716.Google Scholar
  57. .
    Field, G. D., & Chichilnisky, E. J. (2007). Information processing in the primate retina: Circuitry and coding. Annual Review of Neuroscience, 30, 1–30.Google Scholar
  58. .
    Furber, S. B., Brown, G., Bose, J., Cumpstey, J. M., Marshall, P., & Shapiro, J. L. (2007). Sparse distributed memory using rank-order neural codes. IEEE Transactions on Neural Networks, 18, 648–659.Google Scholar
  59. .
    Gerstein, G. L., & Aertsen, A. M. (1985). Representation of cooperative firing activity among simultaneously recorded neurons. Journal of Neurophysiology, 54(6), 1513–1528.Google Scholar
  60. .
    Gerstein, G. L., & Mandelbrot, B. (1964). Random walk models for the spike activity of a single neuron. Biophysical Journal, 4(1), 41–68.Google Scholar
  61. .
    Gerstner, W., & Kistler, W. M. (2002). Spiking Neuron Models. New York: Cambridge University Press.Google Scholar
  62. .
    Gerstner, W., Kreiter, A. K., Markram, H., & Herz, A. V. M. (1997). Neural codes: Firing rates and beyond. Proceedings of the National Academy of Sciences of the United States of America, 94(24), 12740–12741.Google Scholar
  63. .
    Golomb, D., Hertz, J., Panzeri, S., Treves, A., & Richmond, B. (1997). How well can we estimate the information carried in neuronal responses from limited samples? Neural Computation, 9(3), 649–665.Google Scholar
  64. .
    Grossberg, S. (1999). How does the cerebral cortex work? Learning, attention and grouping by the laminar circuits of visual cortex. Spatial Vision, 12, 163–186.Google Scholar
  65. .
    Grün, S., Aertsen, A. M. H. J., Abeles, M., Gerstein, G., & Palm, G. (1994a). Behavior-related neuron group activity in the cortex. In Proceedings 17th Annual Meeting European Neuroscience Association. Oxford. Oxford University Press.Google Scholar
  66. .
    Grün, S., Aertsen, A. M. H. J., Abeles, M., Gerstein, G., & Palm, G. (1994b). On the significance of coincident firing in neuron group activity. In N. Elsner, & H. Breer (Eds.), Sensory transduction (p. 558). Thieme: Stuttgart.Google Scholar
  67. .
    Grün, S., Diesmann, M., & Aertsen, A. (2002a). Unitary events in multiple single-neuron spiking activity: I. Detection and significance. Neural Computation, 14(1), 43–80.Google Scholar
  68. .
    Grün, S., Diesmann, M., & Aertsen, A. (2002b). Unitary events in multiple single-neuron spiking activity: II. Nonstationary data. Neural Computation, 14(1), 81–119.Google Scholar
  69. .
    Grün, S., Diesmann, M., Grammont, F., Riehle, A., & Aertsen, A. (1999). Detecting unitary events without discretization of time. Journal of Neuroscience, 94(1), 121–154.Google Scholar
  70. .
    Grün, S., & Rotter, S. (Eds.) (2010). Analysis of spike trains. New York: Springer.Google Scholar
  71. .
    Gütig, R., Aertsen, A., & Rotter, S. (2002). Statistical significance of coincident spikes: Count-based versus rate-based statistics. Neural Computation, 14(1), 121–153.Google Scholar
  72. .
    Gutnisky, D. A., & Dragoi, V. (2008). Adaptive coding of visual information in neural populations. Nature, 452(7184), 220–224.Google Scholar
  73. .
    Guyonneau, R., VanRullen, R., & Thorpe, S. J. (2004). Temporal codes and sparse representations: A key to understanding rapid processing in the visual system. Journal of Physiology – Paris, 98, 487–497.Google Scholar
  74. .
    Haft, M., & van Hemmen, J. L. (1998). Theory and implementation of infomax filters for the retina. Network, 9, 39–71.Google Scholar
  75. .
    Hansel, D., & Sompolinsky, H. (1996). Chaos and synchrony in a model of a hypercolumn in visual cortex. Journal of Computational Neuroscience, 3(1), 7–34.Google Scholar
  76. .
    Hawkins, J., & Blakeslee, S. (2004). On intelligence. New York: Times Books, Henry Holt and Company.Google Scholar
  77. .
    Hebb, D. O. (1949). The organization of behavior: A neuropsychological theory. New York: Wiley.Google Scholar
  78. .
    Hecht-Nielsen, R. (2007). Confabulation theory. The mechanism of thought. Berlin: Springer.Google Scholar
  79. .
    Holden, A. V. (1976). Models of the stochastic activity of neurons. New York: Springer.Google Scholar
  80. .
    Hosaka, R., Araki, O., & Ikeguchi, T. (2008). STDP provides the substrate for igniting synfire chains by spatiotemporal input patterns. Neural Computation, 20(2), 415–435.Google Scholar
  81. .
    Hoyer, P. O., & Hyvärinen, A. (2002). A multi-layer sparse coding network learns contour coding from natural images. Vision Research, 42(12), 1593–1605.Google Scholar
  82. .
    Hyvärinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413–2423.Google Scholar
  83. .
    Hyvärinen, A., Hurri, J., & Hoyer, P. O. (2009). Natural Image Statistics. New York: Springer.Google Scholar
  84. .
    Hyvärinen, A., & Karhunen, J. (2001). Independent Component Analysis. New York: Wiley.Google Scholar
  85. .
    Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17, 2443–2452.Google Scholar
  86. .
    Izhikevich, E. M., & Desai, N. S. (2003). Relating STDP to BCM. Neural Computation, 15, 15111523.Google Scholar
  87. .
    Johannesma, P. I. M. (1981). Neural representation of sensory stimuli and sensory interpretation of neural activity. Advanced Physiological Science, 30, 103–125.Google Scholar
  88. .
    Kamimura, R. (2002). Information theoretic neural computation. New York: World Scientific.Google Scholar
  89. .
    Kang, K., & Sompolinsky, H. (2001). Mutual information of population codes and distance measures in probability space. Physical Review Letter, 86(21), 4958–4961.Google Scholar
  90. .
    Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Physical Review E, 59, 4498–4514.Google Scholar
  91. .
    Kjaer, T. W., Hertz, J. A., & Richmond, B. J. (1994). Decoding cortical neuronal signals: Network models, information estimation, and spatial tuning. Journal of Computational Neuroscience, 1, 109–139.Google Scholar
  92. .
    Knoblauch, A., & Palm, G. (2004). What is Signal and What is Noise in the Brain? BioSystems, 79, 83–90.Google Scholar
  93. .
    Koepsell, K., & Sommer, F. T. (2008). Information transmission in oscillatory neural activity. Biological Cybernetics, 99, 403–416.Google Scholar
  94. .
    Koepsell, K., Wang, X., Vaingankar, V., Wei, Y., Wang, Q., Rathbun, D. L., Usrey, W. M., Hirsch, J. A., & Sommer, F. T. (2009). Retinal oscillations carry visual information to cortex. Frontiers in Systems Neuroscience, 3, 1–18.Google Scholar
  95. .
    König, P., Engel, A. K., & Singer, W. (1995). Relation between oscillatory activity and long-range synchronization in cat visual cortex. In Proceedings of the National Academy of Sciences of the United States of America, 92, 290–294.Google Scholar
  96. .
    Kostal, L., Lansky, P., & Rospars, J.-P. (2007). Neuronal coding and spiking randomness. European Journal of Neuroscience, 26(10), 2693–2701.Google Scholar
  97. .
    Krone, G., Mallot, H., Palm, G., & Schüz, A. (1986). Spatiotemporal receptive fields: A dynamical model derived from cortical architectonics. Proceedings of the Royal Society of London. Series B, Biological Sciences, 226(1245), 421–444.Google Scholar
  98. .
    Krüger, J., & Bach, M. (1981). Simultaneous recording with 30 microelectrodes in monkey visual cortex. Experimental Brain Research, 41(2), 191–194.Google Scholar
  99. .
    Legéndy, C. (2009). Circuits in the brain—a model of shape processing in the primary visual cortex. New York: Springer.Google Scholar
  100. .
    Legéndy, C. R. (1975). Three principles of brain function and structure. International Journal of Neuroscience, 6, 237–254.Google Scholar
  101. .
    Legéndy, C. R., & Salcman, M. (1985). Bursts and recurrences of bursts in the spike trains of spontaneously active striate cortex neurons. Journal of Neurophysiology, 53(4), 926–939.Google Scholar
  102. .
    Letvin, J. Y., Maturana, H. R., McCulloch, W. S., & Pitts, W. H. (1959). What the frog’s eye tells the frog’s brain. Proceedings of the IRE, 47(11), 1940–1951.Google Scholar
  103. .
    Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105–117.Google Scholar
  104. .
    Linsker, R. (1989a). An application of the principle of maximum information preservation to linear systems. In D. S. Touretzky (Ed.), Advances in Neural Information Processing Systems (Vol. 1) (pp. 186–194). San Mateo: Morgan Kaufmann.Google Scholar
  105. .
    Linsker, R. (1989b). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Computation, 1(3), 402–411.Google Scholar
  106. .
    Linsker, R. (1992). Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Computation, 4, 691–702.Google Scholar
  107. .
    Linsker, R. (1997). A local learning rule that enables information maximization for arbitrary input distributions. Neural Computation, 9, 1661–1665.Google Scholar
  108. .
    Lisman, J., & Spruston, N. (2005). Postsynaptic depolarization requirements for LTP and LTD: A critique of spike timing-dependent plasticity. Nature Neuroscience, 8(7), 839–841.Google Scholar
  109. .
    Loiselle, S., Rouat, J., Pressnitzer, D., & Thorpe, S. J. (2005). Exploration of rank order coding with spiking neural networks for speech recognition. Proceedings of International Joint Conference on Neural Networks, 4, 2076–2078.Google Scholar
  110. .
    MacGregor, R. J. (1987). Neural and brain modeling. New York: Academic.Google Scholar
  111. .
    MacKay, D. M., & McCulloch, W. S. (1952). The limiting information capacity of a neuronal link. Bulletin of Mathematical Biology, 14(2), 127–135.Google Scholar
  112. .
    Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurons. Science, 268(5216), 1503–1506.Google Scholar
  113. .
    Markram, H., Luebke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215.Google Scholar
  114. .
    Martignon, L., Deco, G., Laskey, K., Diamond, M., Freiwald, W. A., & Vaadia, E. (2000). Neural coding: Higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Computation, 12(11), 2621–2653.Google Scholar
  115. .
    Martignon, L., von Hasseln, H., Grün, S., Aertsen, A. M. H. J., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics, 73(1), 69–81.Google Scholar
  116. .
    Martignon, L., von Hasseln, H., Grün, S., & Palm, G. (1994). Modelling the interaction in a set of neurons implicit in their frequency distribution: A possible approach to neural assemblies. In F. Allocati, C. Musio, & C. Taddei-Ferretti (Eds.), Biocybernetics (Cibernetica Biologica) (pp. 268–288). Torino: Rosenberg & Sellier.Google Scholar
  117. .
    Masquelier, T., Guyonneau, R., & Thorpe, S. (2009). Competitive STDP-based spike pattern learning. Neural Computation, 21(5), 1259–1276.Google Scholar
  118. .
    Massaro, D. W. (1975). Experimental psychology and human information processing. Chicago: Rand McNally & Co.Google Scholar
  119. .
    McClurkin, J. W., Gawne, T. J., Optican, L. M., & Richmond, B. J. (1991). Lateral geniculate neurons in behaving priimates II. Encoding of visual information in the temporal shape of the response. Journal of Neurophysiology, 66(3), 794–808.Google Scholar
  120. .
    Miller, J. G. (1962). Information input overload. In M. C. Yovits, G. T. Jacobi, & G. D. Goldstein (Eds.), Self-Organizing Systems (pp. 61–78). Washington DC: Spartan Books.Google Scholar
  121. .
    Morrison, A., Aertsen, A., & Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Computation, 19(6), 1437–1467.Google Scholar
  122. .
    Morrison, A., Diesmann, M., & Gerstner, W. (2008). Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics, 98, 459–478.Google Scholar
  123. .
    Nakahara, H., & Amari, S. (2002). Information geometric measure for neural spikes. Neural Computation, 14, 2269–2316.Google Scholar
  124. .
    Nakahara, H., Amari, S., & Richmond, B. J. (2006). A comparison of descriptive models of a single spike train by information geometric measure. Neural Computation, 18, 545–568.Google Scholar
  125. .
    Nemenman, I., Lewen, G. D., Bialek, W., & de Ruyter van Steveninck, R. R. (2008). Neural coding of natural stimuli: Information at sub-millisecond resolution. PLoS Computational Biology, 4(3), e1000025.Google Scholar
  126. .
    Nirenberg, S., & Latham, P. (2003). Decoding neural spike trains: How important are correlations? Proceedings of the National Academy of Science of the United States of America, 100, 7348–7353.Google Scholar
  127. .
    Nirenberg, S., & Latham, P. (2005). Synergy, redundancy and independence in population codes. Journal of Neuroscience, 25, 5195–5206.Google Scholar
  128. .
    Optican, L. M., Gawne, T. J., Richmond, B. J., & Joseph, P. J. (1991). Unbiased measures of transmitted information and channel capacity from multivariate neuronal data. Biological Cybernetics, 65(5), 305–310.Google Scholar
  129. .
    Optican, L. M., & Richmond, B. J. (1987). Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. Journal of Neurophysiology, 57(1), 162–178.Google Scholar
  130. .
    Osborne, L. C., Palmer, S. E., Lisberger, S. G., & Bialek, W. (2008). The neural basis for combinatorial coding in a cortical population response. Journal of Neuroscience, 28(50), 13522–13531.Google Scholar
  131. .
    Palm, G. (1980). On associative memory. Biological Cybernetics, 36, 167–183.Google Scholar
  132. .
    Palm, G. (1981). Evidence, information and surprise. Biological Cybernetics, 42(1), 57–68.Google Scholar
  133. .
    Palm, G. (1982). Neural assemblies, an alternative approach to artificial intelligence. New York: Springer.Google Scholar
  134. .
    Palm, G. (1985). Information und entropie. In H. Hesse (Ed.), Natur und Wissenschaft. Tubingen: Konkursbuch Tübingen.Google Scholar
  135. .
    Palm, G. (1987a). Associative memory and threshold control in neural networks. In J. L. Casti, & A. Karlqvist (Eds.), Real brains: artificial minds (pp. 165–179). New York: Elsevier.Google Scholar
  136. .
    Palm, G. (1987b). Computing with neural networks. Science, 235, 1227–1228.Google Scholar
  137. .
    Palm, G. (1992). On the information storage capacity of local learning rules. Neural Computation, 4, 703–711.Google Scholar
  138. .
    Palm, G., Aertsen, A. M. H. J., & Gerstein, G. L. (1988). On the significance of correlations among neuronal spike trains. Biological Cybernetics, 59(1), 1–11.Google Scholar
  139. .
    Palm, G., & Sommer, F. T. (1992). Information capacity in recurrent McCulloch–Pitts networks with sparsely coded memory states. Network, 3(2), 177–186.Google Scholar
  140. .
    Panzeri, S., & Schultz, S. R. (2001). A unified approach to the study of temporal, correlational, and rate coding. Neural Computation, 13(6), 1311–1349.Google Scholar
  141. .
    Panzeri, S., Schultz, S. R., Treves, A., & Rolls, E. T. (1999). Correlations and the encoding of information in the nervous system. Proceedings of the Royal Society of London Series B; Biological Science, 266(1423), 1001–1012.Google Scholar
  142. .
    Perkel, D. H., & Bullock, T. H. (1967). Neural coding. Neurosciences Research Program Bulletin, 6(3), 223–344.Google Scholar
  143. .
    Perrinet, L., Samuelides, M., & Thorpe, S. J. (2003). Coding static natural images using spike event times: Do neurons cooperate? IEEE Transactions on Neural Networks, 15, 1164–1175.Google Scholar
  144. .
    Pfaffelhuber, E. (1972). Learning and information theory. International Journal of Neuroscience, 3, 83.Google Scholar
  145. .
    Pfister, J.-P., & Gerstner, W. (2006). Triplets of spikes in a model of spike timing-dependent plasticity. The Journal of Neuroscience, 26(38), 9673–9682.Google Scholar
  146. .
    Prut, Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., & Abeles, M. (1998). Spatiotemporal structure of cortical activity: Properties and behavioral relevance. Journal of Neurophysiology, 79(6), 2857–2874.Google Scholar
  147. .
    Quastler, H. (1956a). Information theory in psychology: Problems and methods. Glencoe: Free Press.Google Scholar
  148. .
    Quastler, H. (1956b). Studies of human channel capacity. In E. Cherry (Ed.), Information theory, 3rd London symposium (p. 361). London: Butterworths.Google Scholar
  149. .
    Rieke, F., Warland, D., de Ruyter van Steveninck, R., & Bialek, W. (1997). Spikes: Exploring the neural code. Cambridge: MIT Press.Google Scholar
  150. .
    Rolls, E. T., Treves, A., & Tovee, M. J. (1997). The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex. Experimental Brain Research, 114(1), 149–162.Google Scholar
  151. .
    Schneideman, E., Bialek, W., & M. J. II. Berry (2003). Synergy, redundancy, and independence in population codes. Journal of Neuroscience, 23, 11539–11553.Google Scholar
  152. .
    Seriès, P., Latham, P., & Pouget, A. (2004). Tuning curve sharpening for orientation slectivity: Coding efficiency and the impact of correlations. Nature Neurosience, 7(10), 1129–1135.Google Scholar
  153. .
    Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4(4), 569–579.Google Scholar
  154. .
    Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. Journal of Neuroscience, 18(10), 3870–3896.Google Scholar
  155. .
    Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379–423, 623–656.Google Scholar
  156. .
    Shaw, G., & Palm, G. (Eds.) (1988). Brain Theory Reprint Volume. Singapore: World Scientific.Google Scholar
  157. .
    Softky, W., & Koch, C. (1992). Cortical cells should fire regularly, but do not. Neural Computation, 4, 643–646.Google Scholar
  158. .
    Softky, W. R. (1995). Simple codes versus efficient codes. Current Opinion in Neurobiology, 5(2), 239–247.Google Scholar
  159. .
    Softky, W. R., & Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience, 13(1), 334–350.Google Scholar
  160. .
    Song, S., Miller, K. D., & Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3, 919–926.Google Scholar
  161. .
    Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982). Predictive coding: A fresh view of inhibition in the retina. Proceedings of the Royal Society of London Series B; Biological Science, 216(1205), 427–459.Google Scholar
  162. .
    Stevens, C. F., & Zador, A. M. (1998). Input synchrony and the irregular firing of cortical neurons. Nature Neuroscience, 1(3), 210–217.Google Scholar
  163. .
    Tetko, I. V., & Villa, A. E. P. (1992). Fast combinitorial methods to estimate the probability of complex temporal patterns of spikes. Biological Cybernetics, 76, 397–407.Google Scholar
  164. .
    Thorpe, S. J., Guyonneau, R., Guilbaud, N., Allegraud, J.-M., & VanRullen, R. (2004). Spikenet: Real-time visual processing with one spike per neuron. Neurocomputing, 58–60, 857–864.Google Scholar
  165. .
    Tononi, G., Sporns, O., & Edelman, G. M. (1992). Reentry and the problem of integrating multiple cortical areas: Simulation of dynamic integration in the visual system. Cerebral Cortex, 2(4), 310–335.Google Scholar
  166. .
    Tononi, G., Sporns, O., & Edelman, G. M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system. Neurobiology, 91, 5033–5037.Google Scholar
  167. .
    Treves, A., & Panzeri, S. (1995). The upward bias in measures of information derived from limited data samples. Neural Computation, 7, 399–407.Google Scholar
  168. .
    Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter releaseprobability. Proceedings of the National Academy of Sciences of the United States of America, 94(2), 719–723.Google Scholar
  169. .
    Tsodyks, M., Uziel, A., & Markram, H. (2000). Synchrony generation in recurrent networks with frequency-dependent synapses. The Journal of Neuroscience, 20, 1–5.Google Scholar
  170. .
    Uttley, A. M. (1979). Information Transmission in the Nervous System. London: Academic.Google Scholar
  171. .
    Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. M. H. J. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518.Google Scholar
  172. .
    van Essen, D. C., Olshausen, B., Anderson, C. H., & Gallant, J. L. (1991). Pattern recognition, attention and information bottlenecks in the primate visual system. Proceedings of SPIE Conference on Visual Information Processing: From Neurons to Chips, 1473, 17–27.Google Scholar
  173. .
    van Rossum, M. C. W., Bi, G. Q., & Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. The Journal of Neuroscience, 20, 8812–8821.Google Scholar
  174. .
    Wang, X., Hirsch, J. A., & Sommer, F. T. (2010). Recoding of sensory information across the retinothalamic synapse. The Journal of Neuroscience, 30, 13567–13577.Google Scholar
  175. .
    Wenzel, F. (1961). Über die Erkennungszeit beim Lesen. Biological Cybernetics, 1(1), 32–36.Google Scholar
  176. .
    Yang, H. H., & Amari, S. (1997). Adaptive online learning algorithms for blind separation: Maximum entropy and minimum mutual information. Neural Computation, 9, 1457–1482.Google Scholar
  177. .
    Yovits, M. C., Jacobi, G. T., & Goldstein, G. D. (Eds.) (1962). Self-organizing systems. Proceedings of the Conference on Self-Organizing Systems held on May 22, 23, and 24, 1962 in Chicago, Illinois. Washington: Spartan Books.Google Scholar
  178. .
    Zemel, R. S., & Hinton, G. E. (1995). Learning population codes by minimizing description length. Neural Computation, 7, 549–564.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Günther Palm
    • 1
  1. 1.Neural Information ProcessingUniversity of UlmUlmGermany

Personalised recommendations