Abstract
A neural network model of associative memory is presented which unifies the two historically more relevant enhancements to the basic Little-Hopfield discrete model: the graded response units approach and the stochastic, Glauber-inspired model with a random field representing thermal fluctuations. This is done by casting the retrieval process of the model with graded response neurons, into the framework of a diffusive process governed by the Fokker-Plank equation, which leads to a Langevin system describing the process at a microscopic level, while the time evolution of the probability density function is governed by a multivariate Fokker Planck equation operating over the space of all possible activation patterns. The present unified approach has two notable features: (i) greater biological plausibility and (ii) ability to escape local minima of energy (associated with spurious memories), which makes it a potential tool for those complex optimization problems for which the previous models failed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amit, D. J.: Modeling Brain Function, Cambridge University Press, Cambridge, 1989.
Glauber, R. J.: Time-dependent Statistics of the Ising Model, Journal of Mathematical Physics 4 (1963), 294-307.
Hartman, E. J., Keeler, J. D. and Kowalsky, J. M.: Layered Neural Networks with Gaussian Hidden Units as Universal Approximations, Neural Computation 2, 210-215.
Hinton, G. E., Sejnowsky, T. J.: Optimal Perceptual Inference. In: Proc. IEEE Conf. Comp. Vision and Patt. Recognition (Washington, 1983) New York, IEEE. 448-453, 1983.
Hopfield, J. J.: Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. 79 (1982), 2554-2558.
Hopfield, J. J.: Neurons with graded response have collective computational properties like those of two-state neurons, Proc. Natl. Acad. Sci. 81 (1984), 3088-3092.
Hopfield, J. J. and Tank, D. W.: 'Neural' Computation of Decisions in Optimization Problems, Biological Cybernetics 52 (1985), 141-152.
Kampen, N. G. van: Stochastic Processes in Physics and Chemistry, Elsevier, Amsterdam, 1997.
Little, W. A.: The Existence of Persistent States in the Brain, Mathematical Biosciences 19 (1974), 101-120.
Little, W. A.: Analytic Study of the Memory Storage Capacity of a Neural Network, Mathematical Biosciences 39 (1978), 281-290.
Peretto, P.: Collective Properties of Neural Networks: A Statistical Physics Approach, Biological Cybernetics 50 (1984), 51-62.
Segura, E. C. and Perazzo, R. P. J.: Associative memories in infinite dimensional spaces, Neural Processing Letters 12 (2000), 129-144.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Segura Meccia, E.C., Perazzo, R.P.J. Biologically Plausible Associative Memory: Continuous Unit Response + Stochastic Dynamics. Neural Processing Letters 16, 243–257 (2002). https://doi.org/10.1023/A:1021742025239
Issue Date:
DOI: https://doi.org/10.1023/A:1021742025239