A New Learning Algorithm for Mean Field Boltzmann Machines
We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution, we maximize the divergence between one-step reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to approximate the multimodal probability distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits.
KeywordsIndependent Component Analysis Hide Unit Coordinate Descent Contrastive Divergence Boltzmann Machine
Unable to display preview. Download preview PDF.
- 1.G.E. Hinton. Training products of experts by minimizing contrastive divergence. Technical Report GCNU TR 2000-004, Gatsby Computational Neuroscience Unit, University College London, 2000.Google Scholar
- 2.G.E. Hinton and T.J. Sejnowski. Learning and relearning in Boltzmann machines, volume Volume 1: Foundations. MIT Press, 1986.Google Scholar
- 3.G.E. Hinton, M. Welling, Y.W. Teh, and K. Osindero. A new view of ICA. In Int. Conf. on Independent Component Analysis and Blind Source Separation, 2001.Google Scholar
- 5.J.R. Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist Models, Proceedings of the 1990 Summer School, pages 10–17, 1991.Google Scholar
- 7.I. Stoianov, M. Zorzi, S. Becker, and C. Umilta. Associative arithmetic with Boltz-mann machines: the role of number representations. In International Conference on Artificial Neural Networks, 2002.Google Scholar