Real-Time Semantic Segmentation with Label Propagation

  • Rasha Sheikh
  • Martin Garbade
  • Juergen Gall
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)


Despite of the success of convolutional neural networks for semantic image segmentation, CNNs cannot be used for many applications due to limited computational resources. Even efficient approaches based on random forests are not efficient enough for real-time performance in some cases. In this work, we propose an approach based on superpixels and label propagation that reduces the runtime of a random forest approach by factor 192 while increasing the segmentation accuracy.


Random Forest Training Image Convolutional Neural Network Conditional Random Field Weak Classifier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The work has been financially supported by the DFG project GA 1927/2-2 as part of the DFG Research Unit FOR 1505 Mapping on Demand (MoD).


  1. 1.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  2. 2.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: International Conference on Learning Representations (2015)Google Scholar
  3. 3.
    Badrinarayanan, V., Handa, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. CoRR abs/1505.07293 (2015)Google Scholar
  4. 4.
    Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 44–57. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88682-2_5 CrossRefGoogle Scholar
  5. 5.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: IEEE Computer Vision and Pattern Recognition (2008)Google Scholar
  6. 6.
    Sturgess, P., Alahari, K., Ladicky, L., Torr, P.H.: Combining appearance and structure from motion features for road scene understanding. In: British Machine Vision Conference (2009)Google Scholar
  7. 7.
    Ladický, Ľ., Sturgess, P., Alahari, K., Russell, C., Torr, P.H.S.: What, where and how many? combining object detectors and CRFs. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 424–437. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_31 CrossRefGoogle Scholar
  8. 8.
    Kontschieder, P., Rota Bulò, S., Bischof, H., Pelillo, M.: Structured class-labels in random forests for semantic image labelling. In: IEEE International Conference on Computer Vision, pp. 2190–2197 (2011)Google Scholar
  9. 9.
    Schulz, H., Waldvogel, B., Sheikh, R., Behnke, S.: CURFIL: random forests for image labeling on GPU. In: Proceedings of the International Conference on Computer Vision Theory and Applications (2015)Google Scholar
  10. 10.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2016)Google Scholar
  11. 11.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: IEEE Computer Vision and Pattern Recognition, pp. 2141–2148 (2010)Google Scholar
  12. 12.
    Budvytis, I., Badrinarayanan, V., Cipolla, R.: Label propagation in complex video sequences using semi-supervised learning. In: British Machine Vision Conference, vol. 2257, pp. 2258–2259 (2010)Google Scholar
  13. 13.
    Reso, M., Jachalsky, J., Rosenhahn, B., Ostermann, J.: Fast label propagation for real-time superpixels for video content. In: IEEE International Conference on Image Processing (2015)Google Scholar
  14. 14.
    Criminisi, A., Shotton, J.: Decision Forests for Computer Vision and Medical Image Analysis. Springer, London (2013)CrossRefGoogle Scholar
  15. 15.
    Yang, B., Yan, J., Lei, Z., Li, S.Z.: Convolutional channel features. In: IEEE International Conference on Computer Vision, pp. 82–90 (2015)Google Scholar
  16. 16.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  17. 17.
    Iqbal, U., Garbade, M., Gall, J.: Pose for action - action for pose. CoRR abs/1603.04037 (2016)Google Scholar
  18. 18.
    Tighe, J., Lazebnik, S.: Superparsing. Int. J. Comput. Vision 101(2), 329–349 (2013)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Bulo, S., Kontschieder, P.: Neural decision forests for semantic image labelling. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 81–88 (2014)Google Scholar
  20. 20.
    Zhang, C., Wang, L., Yang, R.: Semantic segmentation of urban scenes using dense depth maps. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 708–721. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15561-1_51 CrossRefGoogle Scholar
  21. 21.
    Yang, Y., Li, Z., Zhang, L., Murphy, C., Hoeve, J., Jiang, H.: Local label descriptor for example based semantic image labeling. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 361–375. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33786-4_27 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Computer Science Institute IIIUniversity of BonnBonnGermany

Personalised recommendations