Optimizing the Relevance-Redundancy Tradeoff for Efficient Semantic Segmentation

  • Caner HazırbaşEmail author
  • Julia Diebold
  • Daniel Cremers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9087)


Semantic segmentation aims at jointly computing a segmentation and a semantic labeling of the image plane. The main ingredient is an efficient feature selection strategy. In this work we perform a systematic information-theoretic evaluation of existing features in order to address the question which and how many features are appropriate for an efficient semantic segmentation. To this end, we discuss the tradeoff between relevance and redundancy and present an information-theoretic feature evaluation strategy. Subsequently, we perform a systematic experimental validation which shows that the proposed feature selection strategy provides state-of-the-art semantic segmentations on five semantic segmentation datasets at significantly reduced runtimes. Moreover, it provides a systematic overview of which features are the most relevant for various benchmarks.


Feature analysis Feature selection Image segmentation Semantic scene understanding 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bergbauer, J., Nieuwenhuis, C., Souiai, M., Cremers, D.: Proximity priors for variational semantic segmentation and recognition. In: ICCV Workshop (2013)Google Scholar
  2. 2.
    Chambolle, A., Cremers, D., Pock, T.: A convex approach to minimal partitions. SIAM Journal on Imaging Sciences 5(4), 1113–1158 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Couprie, C., Farabet, C., Najman, L., LeCun, Y.: Indoor semantic segmentation using depth information. In: Int. Conf. on Learning Representations (2013)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Int. Conf. on Comp. Vision and Pattern Recog. (2005)Google Scholar
  5. 5.
    Fröhlich, B., Rodner, E., Denzler, J.: Semantic segmentation with millions of features: integrating multiple cues in a combined random forest approach. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part I. LNCS, vol. 7724, pp. 218–231. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  6. 6.
    He, X., Zemel, R.S., Carreira-Perpindn, M.A.: Multiscale conditional random fields for image labeling. In: Int. Conf. on Comp. Vision and Pattern Recog. (2004)Google Scholar
  7. 7.
    Hermans, A., Floros, G., Leibe, B.: Dense 3D semantic mapping of indoor scenes from RGB-D images. In: Int. Conf. on Robotics and Automation (2014)Google Scholar
  8. 8.
    Korč, F., Förstner, W.: eTRIMS Image Database for Interpreting Images of Man-Made Scenes. Technical report, Department of Photogrammetry, University of Bonn (2009)Google Scholar
  9. 9.
    Ladicky, L., Russell, C., Kohli, P., Torr, P.H.S.: Graph cut based inference with co-occurrence statistics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 239–253. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  10. 10.
    Lowe, D.: Object recognition from local scale-invariant features. In: Int. Conf. on Comp. Vision (1999)Google Scholar
  11. 11.
    Nieuwenhuis, C., Strekalovskiy, E., Cremers, D.: Proportion priors for image sequence segmentation. In: Int. Conf. on Comp. Vision (2013)Google Scholar
  12. 12.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. Trans. on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  13. 13.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  14. 14.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context. Int. Journal of Comp. Vision 81(1), 2–23 (2009)CrossRefGoogle Scholar
  15. 15.
    Silberman, N., Fergus, R.: Indoor Scene segmentation using a structured Light Sensor. In: ICCV Workshop on 3D Representation and Recognition (2011)Google Scholar
  16. 16.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  17. 17.
    Souiai, M., Nieuwenhuis, C., Strekalovskiy, E., Cremers, D.: Convex optimization for scene understanding. In: ICCV Workshop (2013)Google Scholar
  18. 18.
    Souiai, M., Strekalovskiy, E., Nieuwenhuis, C., Cremers, D.: A co-occurrence prior for continuous multi-label optimization. In: Int. Conf. on Energy Minimization Methods for Comp. Vision and Pattern Recog. (2013)Google Scholar
  19. 19.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Int. Conf. on Comp. Vision and Pattern Recog. (2001)Google Scholar
  20. 20.
    Yao, J., Fidler, S., Urtasun, R.: Describing the scene as a whole: joint object detection, scene classification and semantic segmentation. In: Int. Conf. on Comp. Vision and Pattern Recog. (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Caner Hazırbaş
    • 1
    Email author
  • Julia Diebold
    • 1
  • Daniel Cremers
    • 1
  1. 1.Technical University of MunichMunichGermany

Personalised recommendations