Indoor Scene Classification Using Combined 3D and Gist Features

  • Agnes Swadzba
  • Sven Wachsmuth
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6493)


Scene categorization is an important mechanism for providing high-level context which can guide methods for a more detailed analysis of scenes. State-of-the-art techniques like Torralba’s Gist features show a good performance on categorizing outdoor scenes but have problems in categorizing indoor scenes. In contrast to object based approaches, we propose a 3D feature vector capturing general properties of the spatial layout of indoor scenes like shape and size of extracted planar patches and their orientation to each other. This idea is supported by psychological experiments which give evidence for the special role of 3D geometry in categorizing indoor scenes. In order to study the influence of the 3D geometry we introduce in this paper a novel 3D indoor database and a method for defining 3D features on planar surfaces extracted in 3D data. Additionally, we propose a voting technique to fuse 3D features and 2D Gist features and show in our experiments a significant contribution of the 3D features to the indoor scene categorization task.


Feature Vector Amplitude Image Outdoor Scene Indoor Scene Spatial Pyramid Match 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 145–175 (2001)CrossRefzbMATHGoogle Scholar
  2. 2.
    Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: CVPR (2009)Google Scholar
  3. 3.
    Henderson, J.M., Larson, C.L., Zhu, D.C.: Full scenes produce more activation than close-up scenes and scene-diagnostic objects in parahippocampal and retrosplenial cortex: An fMRI study. Brain and Cognition 66, 40–49 (2008)CrossRefGoogle Scholar
  4. 4.
    Henderson, J.M., Larson, C.L., Zhu, D.C.: Cortical activation to indoor versus outdoor scenes: An fMRI study. Experimental Brain Research 179, 75–84 (2007)CrossRefGoogle Scholar
  5. 5.
    Topp, E., Christensen, H.: Dectecting structural ambiguities and transistions during a guided tour. In: ICRA, pp. 2564–2570 (2008)Google Scholar
  6. 6.
    Ullah, M.M., Pronobis, A., Caputo, B., Luo, J., Jensfelt, P., Christensen, H.I.: Towards Robust Place Recognition for Robot Localization. In: ICRA (2008)Google Scholar
  7. 7.
    Ranganathan, A., Dellaert, F.: Semantic modeling of places using objects. In: Robotics: Science and Systems (2007)Google Scholar
  8. 8.
    Mozos, Ó.M., Stachniss, C., Burgard, W.: Supervised learning of places from range data using adaboost. In: ICRA, pp. 1730–1735 (2005)Google Scholar
  9. 9.
    Zender, H., Mozos, O., Jensfelt, P., Kruijff, G.J.M., Burgard, W.: Conceptual spatial representations for indoor mobile robots. RAS 56, 493–502 (2008)Google Scholar
  10. 10.
    Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernández-Madrigal, J.A., González, J.: Multi-hierarchical semantic maps for mobile robotics. In: IROS, pp. 3492–3497 (2005)Google Scholar
  11. 11.
    Vasudevan, S., Gächter, S., Nguyen, V., Siegwart, R.: Cognitive maps for mobile robots - an object based approach. RAS 55, 359–371 (2007)Google Scholar
  12. 12.
    Kim, S., Kweon, I.S.: Scene interpretation: Unified modeling of visual context by particle-based belief propagation in hierarchical graphical model. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 963–972. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Pirri, F.: Indoor environment classification and perceptual matching. In: Intl. Conf. on Knowledge Representation, pp. 30–41 (2004)Google Scholar
  14. 14.
    Torralba, A., Murphy, K.P., Freeman, W.T., Rubin, M.A.: Context-based vision system for place and object recognition. In: ICCV (2003)Google Scholar
  15. 15.
    Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: CVPR (2005)Google Scholar
  16. 16.
    Posner, I., Schröter, D., Newman, P.M.: Using scene similarity for place labelling. In: Intl. Symp. on Experimental Robotics, pp. 85–98 (2006)Google Scholar
  17. 17.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene classification using a hybrid generative/discriminative approach. PAMI 30, 712–727 (2008)CrossRefGoogle Scholar
  18. 18.
    Vogel, J., Schiele, B.: A semantic typicality measure for natural scene categorization. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 195–203. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  19. 19.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  20. 20.
    Huber, D.F., Kapuria, A., Donamukkala, R., Hebert, M.: Parts-based 3D object classification. In: CVPR, pp. 82–89 (2004)Google Scholar
  21. 21.
    Csákány, P., Wallace, A.M.: Representation and classification of 3D objects. Systems, Man, and Cybernetics 33, 638–647 (2003)CrossRefGoogle Scholar
  22. 22.
    Johnson, A.E., Herbert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. PAMI 21, 433–449 (1999)CrossRefGoogle Scholar
  23. 23.
    Munoz, D., Bagnell, J.A., Vandapel, N., Hebert, M.: Contextual classification with functional max-margin markov networks. In: CVPR (2009)Google Scholar
  24. 24.
    Triebel, R., Schmidt, R., Mozos, Ó.M., Burgard, W.: Instance-based amn classification for improved object recognition in 2D and 3D laser range data. In: IJCAI, pp. 2225–2230 (2007)Google Scholar
  25. 25.
    Rusu, R.B., Marton, Z.C., Blodow, N., Beetz, M.: Learning informative point classes for the acquisition of object model maps. In: Intl. Conf. on Control, Automation, Robotics and Vision, Hanoi, Vietnam (2008)Google Scholar
  26. 26.
    Lee, D., Hebert, M., Kanade, T.: Geometric reasoning for single image structure recovery. In: CVPR (2009)Google Scholar
  27. 27.
    Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: ICCV (2009)Google Scholar
  28. 28.
    Yu, S.X., Zhang, H., Malik, J.: Inferring spatial layout from a single image via depth-ordered grouping. In: CVPR Workshops (2008)Google Scholar
  29. 29.
    Weingarten, J., Gruener, G., Siegwart, R.: A state-of-the-art 3D sensor for robot navigation. In: IROS, vol. 3, pp. 2155–2160 (2004)Google Scholar
  30. 30.
    Hois, J., Wünstel, M., Bateman, J., Röfer, T.: Dialog-based 3D-image recognition using a domain ontology. In: Barkowsky, T., Knauff, M., Ligozat, G., Montello, D.R. (eds.) Spatial Cognition 2007. LNCS (LNAI), vol. 4387, pp. 107–126. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  31. 31.
    Stamos, I., Allen, P.K.: Geometry and texture recovery of scenes of large scale. CVIU 88, 94–118 (2002)zbMATHGoogle Scholar
  32. 32.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Kuncheva, L.I.: Combining Pattern Classifiers. John Wiley & Sons, Chichester (2004)CrossRefzbMATHGoogle Scholar
  34. 34.
    Vapnik, V.N.: The Nature of Statistical Learning Theory (1995)Google Scholar
  35. 35.
    Joachims, T.: Learning to Classify Text Using Support Vector Machines. PhD thesis, Cornell University (2002)Google Scholar
  36. 36.
    Möller, T., Kraft, H., Frey, J., Albrecht, M., Lange, R.: Robust 3D measurement with PMD sensors. In: 1st Range Imaging Research Day at ETH (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Agnes Swadzba
    • 1
  • Sven Wachsmuth
    • 1
  1. 1.Applied InformaticsBielefeld UniversityGermany

Personalised recommendations