Biological Cybernetics

, Volume 104, Issue 4–5, pp 251–262 | Cite as

Normalization for probabilistic inference with neurons

  • Chris EliasmithEmail author
  • James Martens
Original Paper


Recently, there have been a number of proposals regarding how biologically plausible neural networks might perform probabilistic inference (Rao, Neural Computation, 16(1):1–38, 2004; Eliasmith and Anderson, Neural engineering: computation, representation and dynamics in neurobiological systems, 2003; Ma et al., Nature Neuroscience, 9(11):1432–1438, 2006; Sahani and Dayan, Neural Computation, 15(10):2255–2279, 2003). To be able to repeatedly perform such inference, it is essential that the represented distributions be appropriately normalized. Past approaches have considered normalization mechanisms independently of inference, often leaving them unexplored, or appealing to a notion of divisive normalization that requires pooling across many neurons. Here, we demonstrate how normalization and inference can be combined into an appropriate connection matrix, eliminating the need for pooling or a division-like operation. We algebraically demonstrate that such a solution is available regardless of the inference being performed. We show that such a solution is relevant to neural computation by implementing it in a recurrent spiking neural network.


Neural computation Probabilistic inference Spiking network Attractor network Normalization NEF 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material (1.7 mb)
ESM 1 (ZIP 1785 kb)


  1. Conklin J, Eliasmith C (2005) An attractor network model of path integration in the rat. J Comput Neurosci 18: 183–203PubMedCrossRefGoogle Scholar
  2. DeAngelis GC, Robson JG, Ohzawa I, Freeman RD (1992) Organization of suppression in receptive fields of neurons in cat visual cortex. J Neurophysiol 68: 144–163PubMedGoogle Scholar
  3. Eliasmith C, Anderson CH (2003) Neural engineering: computation, representation and dynamics in neurobiological systems. MIT Press, CambridgeGoogle Scholar
  4. Fischer BJ (2005) A model of the computations leading to a representation of auditory space in the midbrain of the barn owl. PhD thesis, Washington University, St. LouisGoogle Scholar
  5. Kuo D, Eliasmith C (2005) Integrating behavioral and neural data in a model of zebrafish network interaction. Biol Cybern 93(3): 178–187PubMedCrossRefGoogle Scholar
  6. Ma WJ, Beck JM, Latham PE, Pouget A (2006) Bayesian inference with probabilistic population codes. Nat Neurosci 9(11): 1432–1438PubMedCrossRefGoogle Scholar
  7. Rao R (2004) Bayesian computation in recurrent neural circuits. Neural Comput 16(1): 1–38PubMedCrossRefGoogle Scholar
  8. Reynolds JH, Chelazzi L, Desimone R (1999) Competitive mechanisms sub serve attention in macaque areas V2 and V4. J Neurosci 19: 1736–1753PubMedGoogle Scholar
  9. Rieke F, Warland D, deRuytervan Steveninick R, Bialek W (1997) Spikes: exploring the neural code. MIT Press, Cambridge, MAGoogle Scholar
  10. Ringach DL (2010) Population coding under normalization. Vis Res 50(22): 2223–2232PubMedCrossRefGoogle Scholar
  11. Sahani M, Dayan P (2003) Doubly distributional population codes: Simultaneous representation of uncertainty and multiplicity. Neural Comput 15(10): 2255–2279PubMedCrossRefGoogle Scholar
  12. Salinas E, Abbott LF (1994) Vector reconstruction from firing rates. J Comput Neurosci 1: 89–107PubMedCrossRefGoogle Scholar
  13. Shi J, Wielaard J, Sajda P (2006) Analysis of a gain control model of V1: is the goal redundancy reduction? Conference proceedings: ... Annual international conference of the IEEE engineering in medicine and biology society. IEEE Eng Med Biol Soc 1:4991–4994.Google Scholar
  14. Singh R, Eliasmith C (2006) Higher-dimensional neurons explain the tuning and dynamics of working memory cells. J Neurosci 26: 3667–3678PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Centre for Theoretical NeuroscienceUniversity of WaterlooWaterlooCanada
  2. 2.Department of Computer ScienceUniversity of TorontoTorontoCanada

Personalised recommendations