Weakly Supervised Learning of Foreground-Background Segmentation Using Masked RBMs

  • Nicolas Heess
  • Nicolas Le Roux
  • John Winn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6792)


We propose an extension of the Restricted Boltzmann Machine (RBM) that allows the joint shape and appearance of foreground objects in cluttered images to be modeled independently of the background. We present a learning scheme that learns this representation directly from cluttered images with only very weak supervision. The model generates plausible samples and performs foreground-background segmentation. We demonstrate that representing foreground objects independently of the background can be beneficial in recognition tasks.


RBM segmentation weakly supervised learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hinton, G.E.: Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation 14(8), 1771–1800 (2002)CrossRefGoogle Scholar
  2. 2.
    Hinton, G.E., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 18, 1527–1554 (2006)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Lee, H., Gross, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML (2009)Google Scholar
  4. 4.
    Le Roux, N., Heess, N., Shotton, J., Winn, J.: Learning a Generative Model of Images by Factoring Appearance and Shape. Neural Computation 23(3), 593–650 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Huang, G.B., Rames, M., Berg, T., Learned-Miller, E.: Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. TR 07-49; Univ. of Mass., Amherst (2007)Google Scholar
  6. 6.
    Ranzato, M., Hinton, G.: Modeling Pixel Means and Covariances Using Factorized Third-Order Boltzmann Machines. In: CVPR (2010)Google Scholar
  7. 7.
    Salakhutdinov, R., Hinton, G.E.: Deep Boltzmann Machines. In: AISTATS (2009)Google Scholar
  8. 8.
    Tang, Y.: Gated Boltzmann Machine for Recognition under Occlusion. In: NIPS Workshop on Transfer Learning by Learning Rich Generative Models (2010)Google Scholar
  9. 9.
    Tieleman, T.: Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. In: ICML (2008)Google Scholar
  10. 10.
    Wang, J.Y.A., Adelson, E.H.: Representing moving images with layers. IEEE Transactions on Image Processing 3(5), 625 (1994)CrossRefGoogle Scholar
  11. 11.
    Williams, C.K.I., Titsias, M.K.: Greedy Learning of Multiple Objects in Images using Robust Statistics and Factorial Learning. Neural Comp. 16(5), 1039–1062 (2004)CrossRefGoogle Scholar
  12. 12.
    Wolf, L., Hassner, T., Taigman, Y.: Similarity scores based on background samples. In: Zha, H., Taniguchi, R.-i., Maybank, S. (eds.) ACCV 2009. LNCS, vol. 5995, pp. 88–97. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Nicolas Heess
    • 1
  • Nicolas Le Roux
    • 2
  • John Winn
    • 3
  1. 1.IANCUniversity of EdinburghEdinburghUK
  2. 2.Sierra TeamINRIAParisFrance
  3. 3.Microsoft ResearchCambridgeUK

Personalised recommendations