A Bayesian Framework for Associative Memories
We develop a Bayesian model of associative memory and demonstrate how some well-known associative memory networks can be regarded as particular instances of ours, thereby providing an empirical understanding of their behavior. Specifically, these are the exponential correlation associative memory (ECAM) of Chiueh and Goodman, and the Hopfield model. A connection with the Hamming net and the Aleksander Boolean model is also established. The framework for our study is a novel relaxation method which involves direct probabilistic modelling of the pattern corruption mechanism. The parameter of this model is the memoryless probability of error on nodes of the network. This bit-error probability is not only important for the interpretation of the ECAM and the Hopfield models, but allows also us to understand some more general properties of Bayesian pattern reconstruction by relaxation.
KeywordsAssociative Memory Boolean Network Node Variable Hopfield Network Gradient Ascent
Unable to display preview. Download preview PDF.
- R. P. Lippmann, “An introduction to computing with neural nets ”IEEE ASSP Magazinepp. 4–22, April 1987.Google Scholar
- I. Aleksander, “The logic of connectionist systems,” in I. Aleksander, Ed., Neural Computing Architectures: The Design of Brain-Like Machines London, UK North Oxford Academic Publishers, 1989, pp. 133–155.Google Scholar
- P. A. Chou, “The capacity of the Kanerva associative memory is exponential,” in D. Z. Anderson, Ed., Neural Information Processing Systems New York, NY: American Institute of Physics, 1988, pp. 184–191.Google Scholar
- E. R. Hancock and M. Pelillo, “An analysis of the exponential correlation associative memory,” Proc. 13th Int. ConJ. Pattern Recognition, Vienna, Austria, 1996.Google Scholar