Journal of Computational Neuroscience

, Volume 30, Issue 3, pp 699–720

Capacity analysis in multi-state synaptic models: a retrieval probability perspective

Article

Abstract

We define the memory capacity of networks of binary neurons with finite-state synapses in terms of retrieval probabilities of learned patterns under standard asynchronous dynamics with a predetermined threshold. The threshold is set to control the proportion of non-selective neurons that fire. An optimal inhibition level is chosen to stabilize network behavior. For any local learning rule we provide a computationally efficient and highly accurate approximation to the retrieval probability of a pattern as a function of its age. The method is applied to the sequential models (Fusi and Abbott, Nat Neurosci 10:485–493, 2007) and meta-plasticity models (Fusi et al., Neuron 45(4):599–611, 2005; Leibold and Kempter, Cereb Cortex 18:67–77, 2008). We show that as the number of synaptic states increases, the capacity, as defined here, either plateaus or decreases. In the few cases where multi-state models exceed the capacity of binary synapse models the improvement is small.

Keywords

Hebbian learning Network dynamics Inhibition Sparse coding 

Supplementary material

10827_2010_287_MOESM1_ESM.pdf (132 kb)
(PDF 131 KB)

References

  1. Abraham, W. C., & Bear, M. F. (1996). Metaplasticity: The plasticity of synaptic plasticity. Trends in Neurosciences, 19(4), 126–130. doi:10.1016/S0166-2236(96)80018-X.PubMedCrossRefGoogle Scholar
  2. Amit, D. J., & Brunel, N. (1997a). Dynamics of recurrent network of spiking neurons before and following learning. Network, 8, 373–404.CrossRefGoogle Scholar
  3. Amit, D. J., & Brunel, N. (1997b). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex, 7, 237–252.PubMedCrossRefGoogle Scholar
  4. Amit, D. J., & Fusi, S. (1994). Learning in neural networks with material synapses. Neural Computation, 6, 957–982.CrossRefGoogle Scholar
  5. Amit, D. J., & Mongillo, G. (2003). Selective delay activity in the cortex: Phenomena and interpretation. Cerebral Cortex, 13, 1139–1150.PubMedCrossRefGoogle Scholar
  6. Amit, Y., & Huang, Y. (2010). Precise capacity analysis in binary networks with multiple coding level inputs. Neural Computation, 22(3), 660–688. doi:10.1162/neco.2009.02-09-967.PubMedCrossRefGoogle Scholar
  7. Attwell, D., & Laughlin, S. B. (2001). An energy budget for signaling in the grey matter of the brain. Journal of Cerebral Blood Flow and Metabolism, 21(10), 1133–1145.PubMedGoogle Scholar
  8. Barnes, C. A., McNaughton, B. L., Mizumori, S. J., Leonard, B. W., & Lin, L. H. (1990). Comparison of spatial and temporal characteristics of neuronal activity in sequential stages of hippocampal processing. Progress in Brain Research, 83, 287–300.PubMedCrossRefGoogle Scholar
  9. Barret, A. B., & van Rossum, M. C. (2008). Optimal learning rules for discrete synapses. PLoS Computational Biology, 4, 1–7.CrossRefGoogle Scholar
  10. Ben Dayan Rubin, D. D., & Fusi, S. (2007). Long memory lifetimes require complex synapses and limited sparseness. Frontiers in Computational Neuroscience, 1, 1–14.CrossRefGoogle Scholar
  11. Brunel, N. (2003). Dynamics and plasticity of stimulus-selective persistent activity in cortical network models. Cerebral Cortex, 13, 1151–1161.PubMedCrossRefGoogle Scholar
  12. Curti, E., Mongillo, G., La Camera, G., & Amit, D. J. (2004). Mean-field and capacity in realistic networks of spiking neurons storing sparsely coded random memories. Neural Computation, 16, 2597–2637.PubMedCrossRefGoogle Scholar
  13. Del Giudice, P., Fusi, S., & Mattia, M. (2003). Modelling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses. Journal of Physiology Paris, 97(4–6), 659–681. doi:10.1016/j.jphysparis.2004.01.021.CrossRefGoogle Scholar
  14. Fusi, S., & Abbott, L. F. (2007). Limits on the memory storage capacity of bounded synapses. Nature Neuroscience, 10, 485–493.PubMedGoogle Scholar
  15. Fusi, S., Drew, P. J., & Abbott, L. (2005). Cascade models of synaptically stored memories. Neuron, 45(4), 599–611. doi:10.1016/j.neuron.2005.02.001.PubMedCrossRefGoogle Scholar
  16. Fuster, J. (1995). Memory in the cerebral cortex: An empirical approach to neural networks in the human and nonhuman primate. Cambridge, MA: MIT Press.Google Scholar
  17. Jung, M. W., & McNaughton, B. L. (1993). Spatial selectivity of unit activity in the hippocampal granular layer. Hippocampus, 3(0), 165–182.PubMedCrossRefGoogle Scholar
  18. Leibold, C., & Kempter, R. (2008). Sparseness constrains the prolongation of memory lifetime via synaptic metaplasticity. Cerebral Cortex 18, 67–77.PubMedCrossRefGoogle Scholar
  19. Lennie P (2003) The cost of cortical computation. Current Biology, 13(6), 493–497. doi:10.1016/S0960-9822(03)00135-0.PubMedCrossRefGoogle Scholar
  20. Miyashita, Y., & Hayashi, T. (2000). Neural representation of visual objects: Encoding and top-down activation. Current Opinion in Neurobiology, 10(2), 187–194.PubMedCrossRefGoogle Scholar
  21. Olshausen, B. A., & Field, D. J. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4), 481–487. doi:10.1016/j.conb.2004.07.007.PubMedCrossRefGoogle Scholar
  22. Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature, 435, 1102–1107.PubMedCrossRefGoogle Scholar
  23. Robertson, T., Wright, F. T., & Dykstra, R. L. (1988). Order restricted statistical inference. Wiley.Google Scholar
  24. Rolls, E. T., & Tovee, M. J. (1995). Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. Journal of Physiology, 73(2), 713–726.Google Scholar
  25. Romani, S., Amit, D., & Amit, Y. (2008). Optimizing one-shot learning with binary synapses. Neural Computation, 20, 1928–1950.PubMedCrossRefGoogle Scholar
  26. Sato, T., Uchida, G., & Tanifuji, M. (2007). The nature of neuronal clustering in inferotemporal cortex of macaque monkey revealed by optical imaging and extracellular recording. In 34th Ann. meet. of soc. for neuroscience. San Diego, USA.Google Scholar
  27. Wang, X. J. (2001). Synaptic reverberation underlying mnemonic persistent activity. Trends in Neurosciences, 24(8), 455–463. doi:10.1016/S0166-2236(00)01868-3.PubMedCrossRefGoogle Scholar
  28. Willshaw, D., Buneman, O. P., & Longuet-Higgins, H. (1969). Non-holographic associative memory. Nature (London), 222, 960–962.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Department of StatisticsUniversity of ChicagoChicagoUSA
  2. 2.Departments of Statistics and Computer ScienceUniversity of ChicagoChicagoUSA

Personalised recommendations