Chapter

Neural Networks: Tricks of the Trade

Volume 7700 of the series Lecture Notes in Computer Science pp 621-637

Deep Boltzmann Machines and the Centering Trick

  • Grégoire MontavonAffiliated withMachine Learning Group, Technische Universität Berlin
  • , Klaus-Robert MüllerAffiliated withMachine Learning Group, Technische Universität BerlinDepartment of Brain and Cognitive Engineering, Korea University

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Deep Boltzmann machines are in theory capable of learning efficient representations of seemingly complex data. Designing an algorithm that effectively learns the data representation can be subject to multiple difficulties. In this chapter, we present the “centering trick” that consists of rewriting the energy of the system as a function of centered states. The centering trick improves the conditioning of the underlying optimization problem and makes learning more stable, leading to models with better generative and discriminative properties.

Keywords

Deep Boltzmann machine centering reparameterization unsupervised learning optimization representations