Biological Cybernetics

, Volume 112, Issue 6, pp 523–538 | Cite as

Improved lower bound for the mutual information between signal and neural spike count

  • Sergej O. VoronenkoEmail author
  • Benjamin Lindner
Original Article


The mutual information between a stimulus signal and the spike count of a stochastic neuron is in many cases difficult to determine. Therefore, it is often approximated by a lower bound formula that involves linear correlations between input and output only. Here, we improve the linear lower bound for the mutual information by incorporating nonlinear correlations. For the special case of a Gaussian output variable with nonlinear signal dependencies of mean and variance we also derive an exact integral formula for the full mutual information. In our numerical analysis, we first compare the linear and nonlinear lower bounds and the exact integral formula for two different Gaussian models and show under which conditions the nonlinear lower bound provides a significant improvement to the linear approximation. We then inspect two neuron models, the leaky integrate-and-fire model with white Gaussian noise and the Na–K model with channel noise. We show that for certain firing regimes and for intermediate signal strengths the nonlinear lower bound can provide a substantial improvement compared to the linear lower bound. Our results demonstrate the importance of nonlinear input–output correlations for neural information transmission and provide a simple nonlinear approximation for the mutual information that can be applied to more complicated neuron models as well as to experimental data.


Sensory coding Signal transmission Information theory Mutual information Nonlinear lower bound Nonlinear correlations 



This work was supported by the BMBF (FKZ: 01GQ1001A) and the DFG (Research Training Group GRK1589/2).


  1. Aldworth ZN, Dimitrov AG, Cummins GI, Gedeon T, Miller JP (2011) Temporal encoding in a nervous system. PLOS Comput Biol 7(5):e1002041CrossRefGoogle Scholar
  2. Bernardi D, Lindner B (2015) A frequency-resolved mutual information rate and its application to neural systems. J Neurophysiol 113(5):1342–1357CrossRefGoogle Scholar
  3. Bialek W, Rieke F, Vansteveninck RRD, Warland D (1991) Reading a neural code. Science 252:1854CrossRefGoogle Scholar
  4. Bialek W, Deweese M, Rieke F, Warland D (1993) Bits and brains—information-flow in the nervous system. Physica A 200:581CrossRefGoogle Scholar
  5. Borst A, Theunissen F (1999) Information theory and neural coding. Nat Neurosci 2:947CrossRefGoogle Scholar
  6. Brunel N, Nadal JP (1998) Mutual information, fisher information, and population coding. Neural Comput 10:1731CrossRefGoogle Scholar
  7. Burkitt AN (2006) A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biol Cybern 95:1CrossRefGoogle Scholar
  8. Chacron MJ, Doiron B, Maler L, Longtin A, Bastian J (2003) Non-classical receptive field mediates switch in a sensory neuron’s frequency tuning. Nature 423:77CrossRefGoogle Scholar
  9. Cover T, Thomas J (1991) Elements of information theory. Wiley, New YorkCrossRefGoogle Scholar
  10. Cox DR (1962) Renewal theory. Methuen, LondonGoogle Scholar
  11. Doose J, Doron G, Brecht M, Lindner B (2016) Noisy juxtacellular stimulation in vivo leads to reliable spiking and reveals high-frequency coding in single neurons. J Neurosci 36(43):11120–11132CrossRefGoogle Scholar
  12. Droste F, Schwalger T, Lindner B (2013) Interplay of two signals in a neuron with heterogeneous short-term synaptic plasticity. Front Comput Neurosci 7:1CrossRefGoogle Scholar
  13. Gabbiani F (1996) Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Netw Comput Neural Syst 7:61Google Scholar
  14. Gammaitoni L, Hänggi P, Jung P, Marchesoni F (1998) Stochastic resonance. Rev Mod Phys 70:223CrossRefGoogle Scholar
  15. Grewe J, Kruscha A, Lindner B, Benda J (2017) Synchronous spikes are necessary but not sufficient for a synchrony code in populations of spiking neurons. PNAS 114(10):1977–1985CrossRefGoogle Scholar
  16. Izhikevich EM (2007) Dynamical systems in neuroscience: the geometry of excitability and bursting. The MIT Press, CambridgeGoogle Scholar
  17. Juusola M, French AS (1997) The efficiency of sensory information coding by mechanoreceptor neurons. Neuron 18(6):959–968CrossRefGoogle Scholar
  18. Kraskov A, Stögbauer H, Grassberger P (2004) Estimating mutual information. Phys Rev E 69(6):066138CrossRefGoogle Scholar
  19. Lagarias J (2013) Euler’s constant: Euler’s work and modern developments. Bull Am Math Soc 50(4):527–628CrossRefGoogle Scholar
  20. Lindner B, Sokolov IM (2016) Giant diffusion of underdamped particles in a biased periodic potential. Phys Rev E 93(4):042106CrossRefGoogle Scholar
  21. Lindner B, Schimansky-Geier L, Longtin A (2002) Maximizing spike train coherence or incoherence in the leaky integrate-and-fire model. Phys Rev E 66:031916CrossRefGoogle Scholar
  22. Marmarelis PZ, Naka K (1972) White-noise analysis of a neuron chain: an application of the Wiener theory. Science 175(4027):1276–1278CrossRefGoogle Scholar
  23. McDonnell MD, Ward LM (2011) The benefits of noise in neural systems: bridging theory and experiment. Nat Rev Neurosci 12:415CrossRefGoogle Scholar
  24. Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. Biophys J 35:193CrossRefGoogle Scholar
  25. Neiman AB, Russell DF (2011) Sensory coding in oscillatory electroreceptors of paddlefish. Chaos 21:047505CrossRefGoogle Scholar
  26. Nemenman I, Lewen GD, Bialek W, de Ruyter van Steveninck RR (2008) Neural coding of natural stimuli: information at sub-millisecond resolution. PLOS Comput Biol 4:e1000025CrossRefGoogle Scholar
  27. Nikias CL, Petropulu AP (1993) Higher-order spectral analysis. PTR Prentice Hall, Upper Saddle RiverCrossRefGoogle Scholar
  28. Panzeri S, Schultz SR (2001) A unified approach to the study of temporal, correlational, and rate coding. Neural Comput 13(6):1311–1349CrossRefGoogle Scholar
  29. Panzeri S, Senatore R, Montemurro MA, Petersen RS (2007) Correcting for the sampling bias problem in spike train information measures. J Neurophys 98(3):1064–1072CrossRefGoogle Scholar
  30. Passaglia CL, Troy JB (2004) Information transmission rates of cat retinal ganglion cells. J Neurophysiol 91(3):1217–1229CrossRefGoogle Scholar
  31. Reimann P, Van den Broeck C, Linke H, Hänggi P, Rubi M, Perez-Madrid A (2001) Giant acceleration of free diffusion by use of tilted periodic potentials. Phys Rev Lett 87:010602CrossRefGoogle Scholar
  32. Ricciardi LM (1977) Diffusion processes and related topics on biology. Springer, BerlinCrossRefGoogle Scholar
  33. Rieke F, Warland D, Bialek W (1993) Coding efficiency and information rates in sensory neurons. Europhys Lett 22:151CrossRefGoogle Scholar
  34. Rieke F, Bodnar D, Bialek W (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc Biol Sci 262:259CrossRefGoogle Scholar
  35. Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1996) Spikes: exploring the neural code. MIT Press, CambridgeGoogle Scholar
  36. Ryzhik IM, Gradshtein IS (1963) Tables of series, products, and integrals. VEB Deutscher Verlag der Wissenschaften, BerlinGoogle Scholar
  37. Sadeghi SG, Chacron MJ, Taylor MC, Cullen KE (2007) Neural variability, detection thresholds, and information transmission in the vestibular system. J Neurosci 27(4):771–781CrossRefGoogle Scholar
  38. Shannon R (1948) The mathematical theory of communication. Bell Syst Tech J 27:379CrossRefGoogle Scholar
  39. Siegert AJF (1951) On the first passage time problem. Phys Rev 81:617CrossRefGoogle Scholar
  40. Stemmler M, Koch C (1999) How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate. Nat Neurosci 2:521CrossRefGoogle Scholar
  41. Strong SP, Koberle R, van Steveninck RRD, Bialek W (1998) Entropy and information in neural spike trains. Phys Rev Lett 80:197CrossRefGoogle Scholar
  42. Theunissen F, Miller JP (1995) Temporal encoding in nervous systems: a rigorous definition. J Comput Neurosci 2(2):149–162CrossRefGoogle Scholar
  43. Thomas PJ, Lindner B (2014) Asymptotic phase for stochastic oscillators. Phys Rev Lett 113(25):254101–5CrossRefGoogle Scholar
  44. Victor JD (2002) Binless strategies for estimation of information from neural data. Phys Rev E 66(5):051903CrossRefGoogle Scholar
  45. Victor JD (2006) Approaches to information-theoretic analysis of neural activity. Biol Theory 1(3):302–316CrossRefGoogle Scholar
  46. Vilela RD, Lindner B (2009a) Are the input parameters of white-noise-driven integrate and fire neurons uniquely determined by rate and CV? J Theor Biol 257:90CrossRefGoogle Scholar
  47. Vilela RD, Lindner B (2009b) A comparative study of three different integrate-and-fire neurons: spontaneous activity, dynamical response, and stimulus-induced correlation. Phys Rev E 80:031909CrossRefGoogle Scholar
  48. Voronenko SO (2018) Nonlinear signal processing by noisy spiking neurons. Ph.D. thesis, Humboldt-Universität zu Berlin, BerlinGoogle Scholar
  49. Voronenko SO, Lindner B (2017) Weakly nonlinear response of noisy neurons. New J Phys 19(3):033–038CrossRefGoogle Scholar
  50. Voronenko SO, Stannat W, Lindner B (2015) Shifting spike times or adding and deleting spikes—how different types of noise shape signal transmission in neural populations. JMN 5(1):1–35Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Bernstein Center for Computational Neuroscience BerlinBerlinGermany
  2. 2.Physics DepartmentHumboldt University BerlinBerlinGermany

Personalised recommendations