A New Learning Algorithm for Mean Field Boltzmann Machines

  • Max Welling
  • Geoffrey E. Hinton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2415)


We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution, we maximize the divergence between one-step reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to approximate the multimodal probability distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits.


Independent Component Analysis Hide Unit Coordinate Descent Contrastive Divergence Boltzmann Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G.E. Hinton. Training products of experts by minimizing contrastive divergence. Technical Report GCNU TR 2000-004, Gatsby Computational Neuroscience Unit, University College London, 2000.Google Scholar
  2. 2.
    G.E. Hinton and T.J. Sejnowski. Learning and relearning in Boltzmann machines, volume Volume 1: Foundations. MIT Press, 1986.Google Scholar
  3. 3.
    G.E. Hinton, M. Welling, Y.W. Teh, and K. Osindero. A new view of ICA. In Int. Conf. on Independent Component Analysis and Blind Source Separation, 2001.Google Scholar
  4. 4.
    J.J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. In Proceedings of the National Academy of Sciences, volume 79, pages 2554–2558, 1982.CrossRefMathSciNetGoogle Scholar
  5. 5.
    J.R. Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist Models, Proceedings of the 1990 Summer School, pages 10–17, 1991.Google Scholar
  6. 6.
    C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995–1019, 1987.zbMATHGoogle Scholar
  7. 7.
    I. Stoianov, M. Zorzi, S. Becker, and C. Umilta. Associative arithmetic with Boltz-mann machines: the role of number representations. In International Conference on Artificial Neural Networks, 2002.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Max Welling
    • 1
  • Geoffrey E. Hinton
    • 1
  1. 1.Depart. of Computer ScienceUniv. of TorontoTorontoCanada

Personalised recommendations