Discriminative Learning with Latent Variables for Cluttered Indoor Scene Understanding

  • Huayan Wang
  • Stephen Gould
  • Daphne Koller
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)


We address the problem of understanding an indoor scene from a single image in terms of recovering the layouts of the faces (floor, ceiling, walls) and furniture. A major challenge of this task arises from the fact that most indoor scenes are cluttered by furniture and decorations, whose appearances vary drastically across scenes, and can hardly be modeled (or even hand-labeled) consistently. In this paper we tackle this problem by introducing latent variables to account for clutters, so that the observed image is jointly explained by the face and clutter layouts. Model parameters are learned in the maximum margin formulation, which is constrained by extra prior energy terms that define the role of the latent variables. Our approach enables taking into account and inferring indoor clutter layouts without hand-labeling of the clutters in the training set. Yet it outperforms the state-of-the-art method of Hedau et al. [4] that requires clutter labels.


Latent Variable Prior Constraint Inference Method Indoor Scene Discriminative Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on PAMI 24(5) (2002)Google Scholar
  2. 2.
    Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Transactions on PAMI (2010) (to appear)Google Scholar
  3. 3.
    Gould, S., Fulton, R., Koller, D.: Decomposing a scene into geometric and semantically consistent regions. In: ICCV (2009)Google Scholar
  4. 4.
    Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered room. In: ICCV (2009)Google Scholar
  5. 5.
    Heitz, G., Koller, D.: Learning spatial context: Using stuff to find things. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 30–43. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Hoiem, D., Efros, A., Hebert, M.: Recovering surface layout from an image. IJCV 75(1) (2007)Google Scholar
  7. 7.
    Joachims, T., Finley, T., Yu, C.-N.: Cutting-Plane Training of Structural SVMs. Machine Learning 77(1), 27–59 (2009)CrossRefGoogle Scholar
  8. 8.
    Rother, C.: A new approach to vanishing point detection in architectural environments. In: IVC, vol. 20 (2002)Google Scholar
  9. 9.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV (2007)Google Scholar
  10. 10.
    Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y., Singer, Y.: Large margin methods for structured and interdependent output variables. JMLR 6, 1453–1484 (2005)Google Scholar
  11. 11.
    Vedaldi, A., Zisserman, A.: Structured output regression for detection with partial occlusion. In: NIPS (2009)Google Scholar
  12. 12.
    Yu, C.-N., Joachims, T.: Learning structural SVMs with latent variable. In: ICML (2009)Google Scholar
  13. 13.
    Besag, J.: On the statistical analysis of dirty pictures (with discussions). Journal of the Royal Statistical Society, Series B 48, 259–302 (1986)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Huayan Wang
    • 1
  • Stephen Gould
    • 2
  • Daphne Koller
    • 1
  1. 1.Computer Science DepartmentStanford UniversityUSA
  2. 2.Electrical Engineering DepartmentStanford UniveristyUSA

Personalised recommendations