Biological Cybernetics

, Volume 107, Issue 3, pp 355–365 | Cite as

Metabolic cost of neuronal information in an empirical stimulus-response model

Original Paper


The limits on maximum information that can be transferred by single neurons may help us to understand how sensory and other information is being processed in the brain. According to the efficient-coding hypothesis (Barlow, Sensory Comunication, MIT press, Cambridge, 1961), neurons are adapted to the statistical properties of the signals to which they are exposed. In this paper we employ methods of information theory to calculate, both exactly (numerically) and approximately, the ultimate limits on reliable information transmission for an empirical neuronal model. We couple information transfer with the metabolic cost of neuronal activity and determine the optimal information-to-metabolic cost ratios. We find that the optimal input distribution is discrete with only six points of support, both with and without a metabolic constraint. However, we also find that many different input distributions achieve mutual information close to capacity, which implies that the precise structure of the capacity-achieving input is of lesser importance than the value of capacity.


Information capacity Metabolic cost Stimulus-response curve 



L. Kostal and P. Lansky were supported by the Institute of Physiology RVO: 67985823, Centre for Neuroscience P304/12/G069 and the Grant Agency of the Czech Republic projects P103/11/0282 and P103/12/P558. M. D. McDonnell’s contribution was supported by the Australian Research Council under ARC grant DP1093425 (including an Australian Research Fellowship).


  1. Abou-Faycal IC, Trott MD, Shamai S (2001) The capacity of discrete-time memoryless Rayleigh-fading channels. IEEE Trans Inf Theory 47(4):1290–1301CrossRefGoogle Scholar
  2. Abramowitz M, Stegun IA (1965) Handbook of mathematical functions, with formulas, graphs, and mathematical tables. Dover, New YorkGoogle Scholar
  3. Alexander RM (1996) Optima for animals. Princeton University Press, PrincetonGoogle Scholar
  4. Atick JJ (1992) Could information theory provide an ecological theory of sensory processing? Netw Comput Neural Syst 3(2):213–251CrossRefGoogle Scholar
  5. Attwell D, Laughlin SB (2001) An energy budget for signaling in the grey matter of the brain. J Cereb Blood Flow Metab 21(10):1133–1145PubMedCrossRefGoogle Scholar
  6. Baddeley R, Abbott LF, Booth MCA, Sengpiel F, Freeman T, Wakeman EA, Rolls ET (1997) Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proc Roy Soc B 264:1775–1783CrossRefGoogle Scholar
  7. Balasubramanian V, Berry MJ (2002) A test of metabolically efficient coding in the retina. Netw Comput Neural Syst 13:531–552CrossRefGoogle Scholar
  8. Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith W (ed) Sensory Communication. MIT Press, Cambridge, pp 217–234Google Scholar
  9. Bernardo JM (1979) Reference posterior distributions for Bayesian inference. J Roy Stat Soc B 41:113–147Google Scholar
  10. Blahut R (1972) Computation of channel capacity and rate-distortion functions. IEEE Trans Inf Theory 18(4):460–473CrossRefGoogle Scholar
  11. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, CambridgeGoogle Scholar
  12. Brunel N, Nadal JP (1998) Mutual information, Fisher information, and population coding. Neural Comput 10(7):1731–1757PubMedCrossRefGoogle Scholar
  13. Carandini M (2004) Amplification of trial-to-trial response variability by neurons in visual cortex. PLoS Biol 2(9):e264PubMedCrossRefGoogle Scholar
  14. Chan TH, Hranilovic S, Kschischang FR (2005) Capacity-achieving probability measure for conditionally Gaussian channels with bounded inputs. IEEE Trans Inf Theory 51:2073–2088CrossRefGoogle Scholar
  15. Chiang M, Boyd S (2004) Geometric programming duals of channel capacity and rate distortion. IEEE Trans Inf Theory 50:245–258CrossRefGoogle Scholar
  16. Clarke BS, Barron AR (1990) Information-theoretic asymptotics of Bayes methods. IEEE Trans Inf Theory 36(3):453–471CrossRefGoogle Scholar
  17. Cover TM, Thomas JA (1991) Elements of information theory. Wiley, New YorkCrossRefGoogle Scholar
  18. Dauwels J (2005) Numerical computation of the capacity of continuous memoryless channels. In: Cardinal J, Cerf N, Delgrange O (eds) Proceedings of the 26th symposium on information theory in the Benelux. WIC, Brussels, pp 221–228Google Scholar
  19. Davis M (1980) Capacity and cutoff rate for Poisson-type channels. IEEE Trans Inf Theory 26(6):710–715CrossRefGoogle Scholar
  20. Dimitrov AG, Miller JP (2001) Neural coding and decoding: communication channels and quantization. Netw Comput Neural Syst 12(4):441–472Google Scholar
  21. Fairhall AL, Lewen GD, Bialek W, de Ruyter van Steveninck RR (2001) Efficiency and ambiguity in an adaptive neural code. Nature 412:787–792PubMedCrossRefGoogle Scholar
  22. Farkhooi F, Müller E, Nawrot MP (2011) Adaptation reduces variability of the neuronal population code. Phys Rev E 83(050):905Google Scholar
  23. Gallager RG (1968) Information theory and reliable communication. Wiley, New YorkGoogle Scholar
  24. Gastpar M, Rimoldi B, Vetterli M (2003) To code, or not to code: Lossy source-channel communication revisited. IEEE Trans Inf Theory 49(5):1147–1158CrossRefGoogle Scholar
  25. Grant M, Boyd S (2009) Cvx: Matlab software for disciplined convex programming (web page and software).
  26. Greenwood PE, Lansky P (2005) Optimal signal estimation in neuronal models. Neural Comput 17(10):2240–2257PubMedCrossRefGoogle Scholar
  27. Gremiaux A, Nowotny T, Martinez D, Lucas P, Rospars JP (2012) Modelling the signal delivered by a population of first-order neurons in a moth olfactory system. Brain Res 1434:123–135PubMedCrossRefGoogle Scholar
  28. Huang J, Meyn SP (2005) Characterization and computation of optimal distributions for channel coding. IEEE Trans Inf Theory 51(7):2336–2351CrossRefGoogle Scholar
  29. Ikeda S, Manton JH (2009) Capacity of a single spiking neuron channel. Neural Comput 21(6):1714–1748PubMedCrossRefGoogle Scholar
  30. Jacobson H (1950) The informational capacity of the human ear. Science 112(2901):143PubMedCrossRefGoogle Scholar
  31. Johnson DH, Goodman IN (2008) Inferring the capacity of the vector poisson channel with a bernoulli model. Netw Comput Neural Syst 19(1):13–33CrossRefGoogle Scholar
  32. Johnson N, Kotz S, Balakrishnan N (1994) Continuous univariate distributions, vol 1. Wiley, New YorkGoogle Scholar
  33. Kelley JE (1960) The cutting-plane method for solving convex programs. J Soc Indus Appl Math 8(4):703–712CrossRefGoogle Scholar
  34. Komninakis C, Vandenberghe L, Wesel RD (2001) Capacity of the binomial channel or minimax redundancy for memoryless channels. In: Proceedings of IEEE international symposium on information theory, Washington, p 127Google Scholar
  35. Kostal L (2010) Information capacity in the weak-signal approximation. Phys Rev E 82(026):115Google Scholar
  36. Kostal L (2012) Approximate information capacity of the perfect integrate-and-fire neuron using the temporal code. Brain Res 1434:136–141PubMedCrossRefGoogle Scholar
  37. Kostal L, Lansky P (2010) Information transfer with small-amplitude signals. Phys Rev E 81:050,901(R)Google Scholar
  38. Kostal L, Lansky P (2013) Information transfer under metabolic constraints in a simple homogeneous population of olfactory neurons (manuscript submitted)Google Scholar
  39. Kostal L, Lansky P, Rospars JP (2008) Efficient olfactory coding in the pheromone receptor neuron of a moth. PLoS Comp Biol 4:e1000,053Google Scholar
  40. Lansky P, Sacerdote L (2001) The Ornstein-Uhlenbeck neuronal model with signal-dependent noise. Phys Lett A 285(3–4):132–140CrossRefGoogle Scholar
  41. Lansky P, Pokora O, Rospars JP (2008) Classification of stimuli based on stimulus-response curves and their variability. Brain Res 1225:57–66PubMedCrossRefGoogle Scholar
  42. Laughlin SB (1981) A simple coding procedure enhances a neuron’s information capacity. Z Naturforsch 36(9–10):910–912Google Scholar
  43. Laughlin SB, de Ruyter van Steveninck RR, Anderson JC (1998) The metabolic cost of neural information. Nat Neurosci 1(1):36–41PubMedCrossRefGoogle Scholar
  44. Levy WB, Baxter RA (1996) Energy efficient neural codes. Neural Comput 8(3):531–543PubMedCrossRefGoogle Scholar
  45. Levy WB, Baxter RA (2002) Energy-efficient neuronal computation via quantal synaptic failures. J Neurosci 22(11):4746–4755PubMedGoogle Scholar
  46. Machens CK, Gollisch T, Kolesnikova O, Herz AVM (2005) Testing the efficiency of sensory coding with optimal stimulus ensembles. Neuron 47(3):447–456PubMedCrossRefGoogle Scholar
  47. McDonnell MD, Flitney AP (2009) Signal acquisition via polarization modulation in single photon sources. Phys Rev E 80:060,102(R)Google Scholar
  48. McDonnell MD, Stocks NG (2008) Maximally informative stimuli and tuning curves for sigmoidal rate-coding neurons and populations. Phys Rev Lett 101(5):058,103Google Scholar
  49. McDonnell MD, Mohan A, Stricker C, Ward LM (2012) Input-rate modulation of gamma oscillations is sensitive to network topology, delays and short-term plasticity. Brain Res 1434:162–177PubMedCrossRefGoogle Scholar
  50. McEliece RJ (2002) The theory of information and coding. Cambridge University Press, CambdridgeCrossRefGoogle Scholar
  51. Moujahid A, d’Anjou A, Torrealdea FJ (2011) Energy and information in Hodgkin-Huxley neurons. Phys Rev E 83(031):912Google Scholar
  52. Mountcastle VB, Poggio GF, Werner G (1963) The relation of thalamic cell response to peripheral stimuli varied over an intensive continuum. J Neurophysiol 26(5):807–834 Google Scholar
  53. Nadal JP, Bonnasse-Gahot L (2012) Perception of categories: from coding efficiency to reaction times. Brain Res 1434:47–61Google Scholar
  54. Nikitin AP, Stocks NG, Morse RP, McDonnell MD (2009) Neural population coding is optimized by discrete tuning curves. Phys Rev Lett 103(138):101Google Scholar
  55. Pawlas Z, Klebanov LB, Prokop M, Lansky P (2008) Parameters of spike trains observed in a short time window. Neural Comput 20(5):1325–1343Google Scholar
  56. Quastler H (1953) Essays on the use of information theory in biology. University of Illinois Press, ChampaignGoogle Scholar
  57. Quiroga RQ, Panzeri S (2009) Extracting information from neuronal populations: information theory and decoding approaches. Nat Rev Neurosci 10:173–185CrossRefGoogle Scholar
  58. Rieke F, de Ruyter van Steveninck RR, Warland D, Bialek W (1997) Spikes: exploring the neural code. MIT Press, CambridgeGoogle Scholar
  59. Rissanen JJ (1996) Fisher information and stochastic complexity. IEEE Trans Inf Theory 42(1):40–47CrossRefGoogle Scholar
  60. Sadeghi P, Vontobel PO, Shams R (2009) Optimization of information rate upper and lower bounds for channels with memory. IEEE Trans Inf Theory 55(2):663–688CrossRefGoogle Scholar
  61. Schreiber S, Machens CK, Herz AVM, Laughlin SB (2002) Energy-efficient coding with discrete stochastic events. Neural Comput 14:1323–1346PubMedCrossRefGoogle Scholar
  62. Schroeder DJ (1999) Astronomical optics. Academic Press, San DiegoGoogle Scholar
  63. Shadlen MN, Newsome WT (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J Neurosci 18(10):3870–3896PubMedGoogle Scholar
  64. Smith JG (1971) The information capacity of amplitude-and variance-constrained sclar gaussian channels. Inform Control 18(3):203–219CrossRefGoogle Scholar
  65. Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys J 7(6):797–826PubMedCrossRefGoogle Scholar
  66. Stein RB, Gossen ER, Jones KE (2005) Neuronal variability: noise or part of the signal? Nat Rev Neurosci 6(5):389–397PubMedCrossRefGoogle Scholar
  67. de Ruyter van Steveninck RR, Laughlin SB (1996) The rate of information transfer at graded-potential synapses. Nature 379(6566):642–644CrossRefGoogle Scholar
  68. Suksompong P, Berger T (2010) Capacity analysis for integrate-and-fire neurons with descending action potential thresholds. IEEE Trans Inf Theory 56(2):838–851CrossRefGoogle Scholar
  69. Tchamkerten A (2004) On the discreteness of capacity-achieving distributions. IEEE Trans Inf Theory 50(11):2773–2778CrossRefGoogle Scholar
  70. Tuckwell HC (1988) Introduction to theoretical neurobiology, vol 2. Cambridge University Press, New YorkCrossRefGoogle Scholar
  71. Verdu S (1990) On channel capacity per unit cost. IEEE Trans Inf Theory 36(5):1019–1030CrossRefGoogle Scholar
  72. Wainrib G, Thieullen M, Pakdaman K (2010) Intrinsic variability of latency to first-spike. Biol Cyb 103:43–56CrossRefGoogle Scholar
  73. Weinstock R (1974) Calculus of variations. Dover, New YorkGoogle Scholar
  74. Wiener MC, Richmond BJ (1999) Using response models to estimate channel capacity for neuronal classification of stationary visual stimuli using temporal coding. J Neurophysiol 82(6):2861–2875PubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Lubomir Kostal
    • 1
  • Petr Lansky
    • 1
  • Mark D. McDonnell
    • 2
  1. 1.Institute of Physiology AS CR, v.v.i.VidenskaCzech Republic
  2. 2.Institute for Telecommunications ResearchUniversity of South AustraliaMawson LakesAustralia

Personalised recommendations