Advertisement

Biological Cybernetics

, Volume 54, Issue 1, pp 53–63 | Cite as

Long term memory storage capacity of multiconnected neural networks

  • P. Peretto
  • J. J. Niez
Article

Abstract

Quantitative expressions of long-term memory storage capacities of complex neural network are derived. The networks are made of neurons connected by synapses of any order, of the axono-axonal type considered by Kandel et al. for example. The effect of link deletion possibly related to aging, is also considered. The central result of this study is that, within the framework of Hebb's laws, the number of stored bits is proportional to the number of synapses. The proportionality factor however, decreases when the order of involved synaptic contact increases. This tends to favor neural architectures with low-order synaptic connectivities. It is finally shown that the memory storage capacities can be optimized by a partition of the network into neuron clusters with size comparable with that observed for cortical microcolumns.

Keywords

Neural Network Storage Capacity Term Memory Memory Storage Proportionality Factor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alberts B, Bray D, Lewis J, Raff M, Roberts K, Watson JD (1983) Molecular biology of the cell. Garland Publishing Co, New YorkGoogle Scholar
  2. Carew TJ, Castellucci VF, Kandel ER (1971) An analysis of dishabituation and sensitization of the gill-withdrawal reflex in Aplysia. Int J Neurosci 2:79–98Google Scholar
  3. Changeux JP, Heidmann T (1986) Allosteric receptors and molecular models of learning. In: Edelman G, Gall WE, Cowman WSM (eds.) New insights into synaptic function. Wiley, New York (in press)Google Scholar
  4. Delacour J (1981) Conditionnement et biologie. Masson, ParisGoogle Scholar
  5. Eccles JC (1981) The modular operation of the cerebral neocortex considered as the material basis of neural events. Neurosciences 6:1839–1856Google Scholar
  6. Hebb DO (1949) The organization of behavior. Wiley, New YorkGoogle Scholar
  7. Hopfield JJ (1982) Neural networks and physical system with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558Google Scholar
  8. Hopfield JJ, Computation in simple collective “neural” systems. To appear in proceedings of Les Houches Winter session (26 February–8 March 1985)Google Scholar
  9. Hubel D, Wiesel TN, Stryker MP (1978) Anatomical demonstration of orientation columns in macaque monkey. J Comparative Neurol 177:361–379Google Scholar
  10. Kandel ER (1976) Cellular basis of behavior. Freeman, San FranciscoGoogle Scholar
  11. Kandel ER, Tauc L (1965) Heterosynaptic facilitation in neuron of the abdominal ganglion of Aplysia depilans. J Physiol 181:1–27Google Scholar
  12. Kohonen T (1978) Associative memory. A system theoretical approach. Springer, Berlin Heidelberg New YorkGoogle Scholar
  13. Kohonen T (1984) Self-organization and associative memory. Springer, Berlin Heidelberg New York TokyoGoogle Scholar
  14. Little WA, Shaw GL (1978) Analytic study of the memory storage capacity of neural networks. Math Biosci 39:281–290Google Scholar
  15. Lynch G, Baudry M (1984) The biochemistry of memory: a new and specific hypothesis. Science 224:1057–1063Google Scholar
  16. McCulloch WW, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133Google Scholar
  17. Palm G (1981) On the storage capacity of an associative memory with randomly distributed elements. Biol Cybern 39:125–135Google Scholar
  18. Palm G (1982) Neural assemblies: an alternative approach of artificial intelligence: Springer, Berlin Heidelberg New YorkGoogle Scholar
  19. Peretto P (1984) Collective properties of neural networks. A statistical physics approach. Biol Cybern 50:51–62Google Scholar
  20. Peretto P, Niez JJ (1986) Stochastic dynamics of neural networks. IEEE Trans Syst Man Cabern 16:1Google Scholar
  21. Personnaz L, Guyon I, Dreyfus G (1985) Information storage and retrieval in spinglass like neural networks. J Physique Lett 46:359–365Google Scholar
  22. Roney KJ, Scheibel AB, Shaw GL (1979) Dendritic bundles: survey of anatomical experiments and physiological theories. Brain Res Rev 1:225–271Google Scholar
  23. Shaw GL, Harth E, Scheibel AB (1982) Cooperativity in brain function: assemblies of approximately 30 neurons. Exp Neurol 77:324–3587Google Scholar
  24. Stent GS (1973) A physiological mechanism for Hebb's postulate of learning. Proc Natl Acad Sci USA 70:997–1001Google Scholar
  25. Szentagothai J (1975) The “module concept” in cerebral corterx architecture. Brain Res 95:475–496Google Scholar
  26. Szentagothai J (1983) The modular architectonic principle of neural centers. Rev Physiol Biochem Pharmacol 98:11–61Google Scholar
  27. Thomson RS, Gibson WG (1981) Neural model with probabilistic behavior II. One and two-neuron networks. Math Biosci 56:255–285Google Scholar
  28. Weisbuch G, Fogelman-Soulié F (1985) Scaling laws for the attractors of Hopfield networks. J Phys Lett 46:623–630Google Scholar
  29. Woody CD (1982) Memory learning and higher function: a cellular view. Springer, Berlin Heidelberg New YorkGoogle Scholar

Copyright information

© Springer-Verlag 1986

Authors and Affiliations

  • P. Peretto
    • 1
  • J. J. Niez
    • 2
  1. 1.Centre d'Etudes Nucléaires de Grenoble DRF/PSCGrenobleFrance
  2. 2.Centre d'Etudes Nucléaires de Grenoble LETI/MSCGrenobleFrance

Personalised recommendations