Abstract
How to judge the performance of associative memories in applications? Using information theory, we examine the static structure of memory states and spurious states of a recurrent associative memory after learning. In this framework we consider the critical pattern capacity often used in the literature and introduce the information capacity as a more relevant performance measure for pattern completion. For two types of local learning rule, the Hebb and the clipped Hebb rule our method yields new asymptotic estimates for the information capacity.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
G. Palm, Computing with neural networks, Science 235, 1227–1228 (1987)
J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Sci. 79, 2554–2558 (1982)
G. Palm, On the asymptotic storage capacity on neural networks, Neural Computers, Ed. R. Eckmiller, Chr. v.d. Malsburg, Springer 1988
J.-P. Nadal, G. Toulouse, Information storage in sparsely coded memory nets, Network 1, 61–74 (1990)
J.-P. Nadal, Associative memory: on the (puzzling) sparse coding limit, J. Phys A 24, 1093–1101 (1991)
M. V. Tsodyks, M. V. Feigelman, The enhanced storage capacity in neural networks with low activity level, Europhys. Lett. 6 (2), 101–105 (1988)
E. Gardner, The space of interactions in neural network models, J. Phys. A 21, 257–270 (1988)
H. Gutfreund, Y. Stein, Capacity of neural networks with discrete synaptic couplings, J. Phys A 23, 2613–2630 (1990)
G. Palm, Local learning rules and sparse coding in neural networks in: Advanced Neural Computers Ed: R. Eckmiller Elsevier Science Publishers B.V. (North Holland) (1990)
D.J. Willshaw, O.P. Buneman, H.C. Longuet-Higgins, Nonholographic associative memory, Nature (London) 222, 960–962 (1969)
G. Palm, On associative memory, Biol. Cybern. 36, 19–31 (1980)
S. I. Amari, Statistical neurodynamics of associative memory, Neural Networks 1 63–73 (1989)
D. J. Amit, H. Gutfreund, Statistical mechanics of neural networks near saturation, Ann. Phys. 173, 30–67 (1987)
E. Gardner, H. Gutfreund, I. Yekutieli, The phase space of interactions in neural networks with definite symmetry, J. Phys. A 22, 1995–2008 (1989)
H. Horner, Neural networks with low levels of activity: Ising vs. McCulloch-Pitts neurons, Z. Phys. B 75, 133–136 (1989)
G. Palm, Information capacities of simple storage and retrieval procedures in neural networks, submitted to: J. of stat. Physics (1991)
G. Palm, On the information storage capacity of local learning rules, to be published in Neural Computation (1991)
F. Schwenker, F. Sommer, G. Palm, Simulations of recurrent networks of threshold neurons with sparse activity, in preparation
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1992 Springer-Verlag London Limited
About this paper
Cite this paper
Palm, G., Sommer, F.T. (1992). Information and pattern capacities in neural associative memories with feedback for sparse memory patterns. In: Taylor, J.G., Caianiello, E.R., Cotterill, R.M.J., Clark, J.W. (eds) Neural Network Dynamics. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-2001-8_1
Download citation
DOI: https://doi.org/10.1007/978-1-4471-2001-8_1
Publisher Name: Springer, London
Print ISBN: 978-3-540-19771-3
Online ISBN: 978-1-4471-2001-8
eBook Packages: Springer Book Archive