A Bayesian Framework for Associative Memories

  • Edwin R. Hancock
  • Marcello Pelillo
Conference paper
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)


We develop a Bayesian model of associative memory and demonstrate how some well-known associative memory networks can be regarded as particular instances of ours, thereby providing an empirical understanding of their behavior. Specifically, these are the exponential correlation associative memory (ECAM) of Chiueh and Goodman, and the Hopfield model. A connection with the Hamming net and the Aleksander Boolean model is also established. The framework for our study is a novel relaxation method which involves direct probabilistic modelling of the pattern corruption mechanism. The parameter of this model is the memoryless probability of error on nodes of the network. This bit-error probability is not only important for the interpretation of the ECAM and the Hopfield models, but allows also us to understand some more general properties of Bayesian pattern reconstruction by relaxation.


Associative Memory Boolean Network Node Variable Hopfield Network Gradient Ascent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl Acad. Set. USA, vol. 79, pp. 2554–2558, 1982.MathSciNetCrossRefGoogle Scholar
  2. [2]
    T. D. Chiueh and R. M. Goodman, “Recurrent correlation associative memories,” IEEE Trans. Neural Networks, vol. 2, pp. 275–284, 1991.CrossRefGoogle Scholar
  3. [3]
    S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Machine Intell., vol. 6, pp. 721–741, 1984.zbMATHCrossRefGoogle Scholar
  4. [4]
    E. R. Hancock and J. Kittler, “Discrete relaxation,” Pattern Recognition, vol. 23, pp. 711–733, 1990.CrossRefGoogle Scholar
  5. [5]
    R. P. Lippmann, “An introduction to computing with neural nets ”IEEE ASSP Magazinepp. 4–22, April 1987.Google Scholar
  6. [6]
    I. Aleksander, “The logic of connectionist systems,” in I. Aleksander, Ed., Neural Computing Architectures: The Design of Brain-Like Machines London, UK North Oxford Academic Publishers, 1989, pp. 133–155.Google Scholar
  7. [7]
    K. Y. M. Wong and D. Sherrington, “Theory of associative memory in randomly connected Boolean neural networks,” J. Phys. A vol. 22, pp. 2233–2264, 1989.MathSciNetCrossRefGoogle Scholar
  8. [8]
    P. A. Chou, “The capacity of the Kanerva associative memory is exponential,” in D. Z. Anderson, Ed., Neural Information Processing Systems New York, NY: American Institute of Physics, 1988, pp. 184–191.Google Scholar
  9. E. R. Hancock and M. Pelillo, “An analysis of the exponential correlation associative memory,” Proc. 13th Int. ConJ. Pattern Recognition, Vienna, Austria, 1996.Google Scholar

Copyright information

© Springer-Verlag London Limited 1997

Authors and Affiliations

  • Edwin R. Hancock
    • 1
  • Marcello Pelillo
    • 2
  1. 1.Department of Computer ScienceUniversity of YorkYorkEngland
  2. 2.Dipartimento di Matematica Applicata e InformaticaUniversità di VeneziaVenezia MestreItaly

Personalised recommendations