Abstract
Quantitative expressions of long-term memory storage capacities of complex neural network are derived. The networks are made of neurons connected by synapses of any order, of the axono-axonal type considered by Kandel et al. for example. The effect of link deletion possibly related to aging, is also considered. The central result of this study is that, within the framework of Hebb's laws, the number of stored bits is proportional to the number of synapses. The proportionality factor however, decreases when the order of involved synaptic contact increases. This tends to favor neural architectures with low-order synaptic connectivities. It is finally shown that the memory storage capacities can be optimized by a partition of the network into neuron clusters with size comparable with that observed for cortical microcolumns.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alberts B, Bray D, Lewis J, Raff M, Roberts K, Watson JD (1983) Molecular biology of the cell. Garland Publishing Co, New York
Carew TJ, Castellucci VF, Kandel ER (1971) An analysis of dishabituation and sensitization of the gill-withdrawal reflex in Aplysia. Int J Neurosci 2:79–98
Changeux JP, Heidmann T (1986) Allosteric receptors and molecular models of learning. In: Edelman G, Gall WE, Cowman WSM (eds.) New insights into synaptic function. Wiley, New York (in press)
Delacour J (1981) Conditionnement et biologie. Masson, Paris
Eccles JC (1981) The modular operation of the cerebral neocortex considered as the material basis of neural events. Neurosciences 6:1839–1856
Hebb DO (1949) The organization of behavior. Wiley, New York
Hopfield JJ (1982) Neural networks and physical system with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558
Hopfield JJ, Computation in simple collective “neural” systems. To appear in proceedings of Les Houches Winter session (26 February–8 March 1985)
Hubel D, Wiesel TN, Stryker MP (1978) Anatomical demonstration of orientation columns in macaque monkey. J Comparative Neurol 177:361–379
Kandel ER (1976) Cellular basis of behavior. Freeman, San Francisco
Kandel ER, Tauc L (1965) Heterosynaptic facilitation in neuron of the abdominal ganglion of Aplysia depilans. J Physiol 181:1–27
Kohonen T (1978) Associative memory. A system theoretical approach. Springer, Berlin Heidelberg New York
Kohonen T (1984) Self-organization and associative memory. Springer, Berlin Heidelberg New York Tokyo
Little WA, Shaw GL (1978) Analytic study of the memory storage capacity of neural networks. Math Biosci 39:281–290
Lynch G, Baudry M (1984) The biochemistry of memory: a new and specific hypothesis. Science 224:1057–1063
McCulloch WW, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133
Palm G (1981) On the storage capacity of an associative memory with randomly distributed elements. Biol Cybern 39:125–135
Palm G (1982) Neural assemblies: an alternative approach of artificial intelligence: Springer, Berlin Heidelberg New York
Peretto P (1984) Collective properties of neural networks. A statistical physics approach. Biol Cybern 50:51–62
Peretto P, Niez JJ (1986) Stochastic dynamics of neural networks. IEEE Trans Syst Man Cabern 16:1
Personnaz L, Guyon I, Dreyfus G (1985) Information storage and retrieval in spinglass like neural networks. J Physique Lett 46:359–365
Roney KJ, Scheibel AB, Shaw GL (1979) Dendritic bundles: survey of anatomical experiments and physiological theories. Brain Res Rev 1:225–271
Shaw GL, Harth E, Scheibel AB (1982) Cooperativity in brain function: assemblies of approximately 30 neurons. Exp Neurol 77:324–3587
Stent GS (1973) A physiological mechanism for Hebb's postulate of learning. Proc Natl Acad Sci USA 70:997–1001
Szentagothai J (1975) The “module concept” in cerebral corterx architecture. Brain Res 95:475–496
Szentagothai J (1983) The modular architectonic principle of neural centers. Rev Physiol Biochem Pharmacol 98:11–61
Thomson RS, Gibson WG (1981) Neural model with probabilistic behavior II. One and two-neuron networks. Math Biosci 56:255–285
Weisbuch G, Fogelman-Soulié F (1985) Scaling laws for the attractors of Hopfield networks. J Phys Lett 46:623–630
Woody CD (1982) Memory learning and higher function: a cellular view. Springer, Berlin Heidelberg New York
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Peretto, P., Niez, J.J. Long term memory storage capacity of multiconnected neural networks. Biol. Cybern. 54, 53–63 (1986). https://doi.org/10.1007/BF00337115
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00337115