As Time Goes by—Anytime Semantic Segmentation with Iterative Context Forests

  • Björn Fröhlich
  • Erik Rodner
  • Joachim Denzler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7476)


We present a new approach for contextual semantic segmentation and introduce a new tree-based framework, which combines local information and context knowledge in a single model. The method itself is also suitable for anytime classification scenarios, where the challenge is to estimate a label for each pixel in an image while allowing an interruption of the estimation at any time. This offers the application of the introduced method in time-critical tasks, like automotive applications, with limited computational resources unknown in advance. Label estimation is done in an iterative manner and includes spatial context right from the beginning. Our approach is evaluated in extensive experiments showing its state-of-the-art performance on challenging street scene datasets with anytime classification abilities.


Random Forest Context Feature Average Recognition Rate Street Scene Semantic Segmentation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Csurka, G., Perronnin, F.: An efficient approach to semantic segmentation. IJCV 95(2), 198–212 (2011)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Dahlkamp, H., Kaehler, A., Stavens, D., Thrun, S., Bradski, G.: Self-supervised monocular road detection in desert terrain. In: Robotics: Science and Systems (2006)Google Scholar
  4. 4.
    DeCoste, D.: Anytime interval-valued outputs for kernel machines: Fast support vector machine classification via distance geometry. In: Proceedings of the International Conference on Machine Learning (ICML 2002), pp. 99–106 (2002)Google Scholar
  5. 5.
    Esmeir, S., Markovitch, S.: Anytime learning of anycost classifiers. Machine Learning 82(3), 445–473 (2011)CrossRefGoogle Scholar
  6. 6.
    Fink, M., Perona, P.: Mutual boosting for contextual inference. In: Advances in Neural Information Processing Systems (NIPS 2003), vol. 16, pp. 1515–1522 (2003)Google Scholar
  7. 7.
    Fröhlich, B., Rodner, E., Denzler, J.: A fast approach for pixelwise labeling of facade images. In: ICPR, pp. 3029–3032 (2010)Google Scholar
  8. 8.
    Hui, B., Yang, Y., Webb, G.: Anytime classification for a pool of instances. Machine Learning 77(1), 61–102 (2009)CrossRefGoogle Scholar
  9. 9.
    Korč, F., Förstner, W.: eTRIMS image database for interpreting images of man-made scenes. Tech. Rep. TR-IGG-P-2009-01, University of Bonn (2009)Google Scholar
  10. 10.
    Ladický, Ľ., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W., Torr, P.: Joint optimisation for object class segmentation and dense stereo reconstruction. In: BMVC, pp. 104.1–104.11 (2010)Google Scholar
  11. 11.
    Rusu, R.B., Holzbach, A., Bradski, G., Beetz, M.: Detecting and segmenting objects for mobile manipulation. In: Proceedings of IEEE Workshop on Search in 3D and Video (S3DV), pp. 47–54 (2009)Google Scholar
  12. 12.
    Seidl, T., Assent, I., Kranen, P., Krieger, R., Herrmann, J.: Indexing density models for incremental learning and anytime classification on data streams. In: Proceedings of the 12th Int. Conference on Extending Database Technology, pp. 311–322 (2009)Google Scholar
  13. 13.
    Shieh, J., Keogh, E.J.: Polishing the right apple: Anytime classification also benefits data streams with constant arrival times. In: Proceedings of the International Conference on Data Mining, pp. 461–470 (2010)Google Scholar
  14. 14.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: CVPR, pp. 1–8 (2008)Google Scholar
  15. 15.
    Viola, P., Jones, M.: Robust real-time object detection. IJCV 57, 137–154 (2002)CrossRefGoogle Scholar
  16. 16.
    Yang, M.Y., Förstner, W.: A hierarchical conditional random field model for labeling and classifying images of man-made scenes. In: Proceedings of the IEEE Computer Vision Workshops (ICCV Workshops), pp. 196–203 (2011)Google Scholar
  17. 17.
    Yang, M.Y., Förstner, W.: Regionwise Classification of Building Facade Images. In: Stilla, U., Rottensteiner, F., Mayer, H., Jutzi, B., Butenuth, M. (eds.) PIA 2011. LNCS, vol. 6952, pp. 209–220. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Björn Fröhlich
    • 1
  • Erik Rodner
    • 1
  • Joachim Denzler
    • 1
  1. 1.Computer Vision GroupFriedrich Schiller University of JenaGermany

Personalised recommendations