Semantic Segmentation with Millions of Features: Integrating Multiple Cues in a Combined Random Forest Approach

  • Björn Fröhlich
  • Erik Rodner
  • Joachim Denzler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7724)


In this paper, we present a new combined approach for feature extraction, classification, and context modeling in an iterative framework based on random decision trees and a huge amount of features. A major focus of this paper is to integrate different kinds of feature types like color, geometric context, and auto context features in a joint, flexible and fast manner. Furthermore, we perform an in-depth analysis of multiple feature extraction methods and different feature types. Extensive experiments are performed on challenging facade recognition datasets, where we show that our approach significantly outperforms previous approaches with a performance gain of more than 15% on the most difficult dataset.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arbelaez, P., Hariharan, B., Gu, C., Gupta, S., Bourdev, L., Malik, J.: Semantic segmentation using regions and parts. In: CVPR (2012)Google Scholar
  2. 2.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. PAMI 24(5), 603–619 (2002)CrossRefGoogle Scholar
  4. 4.
    Csurka, G., Perronnin, F.: An efficient approach to semantic segmentation. IJCV 95(2), 198–212 (2011)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004)CrossRefGoogle Scholar
  6. 6.
    Fink, M., Perona, P.: Mutual boosting for contextual inference. In: NIPS, vol. 16, pp. 1515–1522 (2003)Google Scholar
  7. 7.
    Fröhlich, B., Rodner, E., Denzler, J.: A fast approach for pixelwise labeling of facade images. In: ICPR, pp. 3029–3032 (2010)Google Scholar
  8. 8.
    Fröhlich, B., Rodner, E., Denzler, J.: As Time Goes by—Anytime Semantic Segmentation with Iterative Context Forests. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 1–10. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Gool, L.J.V., Zeng, G., den Borre, F.V., Müller, P.: Towards mass-produced building models. In: Photogrammetric Image Analysis, pp. 209–220 (2007)Google Scholar
  10. 10.
    Hoiem, D., Efros, A.A., Hebert, M.: Geometric context from a single image. In: ICCV, vol. 1, pp. 654–661. IEEE (October 2005)Google Scholar
  11. 11.
    Kohli, P., Ladicky, L., Torr, P.: Robust higher order potentials for enforcing label consistency. In: CVPR, pp. 1–8 (2008)Google Scholar
  12. 12.
    Korč, F., Förstner, W.: eTRIMS image database for interpreting images of man-made scenes. Tech. Rep. TR-IGG-P-2009-01, University of Bonn (2009)Google Scholar
  13. 13.
    Ladický, Ľ., Russell, C., Kohli, P., Torr, P.H.S.: Associative hierarchical crfs for object class image segmentation. In: ICCV, pp. 739–746 (2009)Google Scholar
  14. 14.
    Martinović, A., Mathias, M., Weissenberg, J., Van Gool, L.: A Three-Layered Approach to Facade Parsing. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 416–429. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Ripperda, N., Brenner, C.: Evaluation of Structure Recognition Using Labelled Facade Images. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 532–541. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  16. 16.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: CVPR, pp. 1–8 (2008)Google Scholar
  17. 17.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Teboul, O., Simon, L., Koutsourakis, P., Paragios, N.: Segmentation of building facades using procedural shape priors. In: CVPR, pp. 3105–3112 (2010)Google Scholar
  19. 19.
    Tu, Z., Bai, X.: Auto-context and its application to high-level vision tasks and 3d brain image segmentation. PAMI 32(10), 1744–1757 (2010)CrossRefGoogle Scholar
  20. 20.
    Viola, P., Jones, M.: Robust real-time object detection. IJCV 57, 137–154 (2002)CrossRefGoogle Scholar
  21. 21.
    van de Weijer, J., Schmid, C.: Applying color names to image description. In: ICIP, vol. 3, pp. 493–496 (2007)Google Scholar
  22. 22.
    Yang, M.Y., Förstner, W.: A hierarchical conditional random field model for labeling and classifying images of man-made scenes. In: ICCV Workshops, pp. 196–203 (2011)Google Scholar
  23. 23.
    Yang, M.Y., Förstner, W.: Regionwise Classification of Building Facade Images. In: Stilla, U., Rottensteiner, F., Mayer, H., Jutzi, B., Butenuth, M. (eds.) PIA 2011. LNCS, vol. 6952, pp. 209–220. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Björn Fröhlich
    • 1
  • Erik Rodner
    • 1
  • Joachim Denzler
    • 1
  1. 1.Computer Vision GroupFriedrich Schiller University JenaGermany

Personalised recommendations