Advertisement

Supervised Geodesic Propagation for Semantic Label Transfer

  • Xiaowu Chen
  • Qing Li
  • Yafei Song
  • Xin Jin
  • Qinping Zhao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7574)

Abstract

In this paper we propose a novel semantic label transfer method using supervised geodesic propagation (SGP). We use supervised learning to guide the seed selection and the label propagation. Given an input image, we first retrieve its similar image set from annotated databases. A Joint Boost model is learned on the similar image set of the input image. Then the recognition proposal map of the input image is inferred by this learned model. The initial distance map is defined by the proposal map: the higher probability, the smaller distance. In each iteration step of the geodesic propagation, the seed is selected as the one with the smallest distance from the undetermined superpixels. We learn a classifier as an indicator to indicate whether to propagate labels between two neighboring superpixels. The training samples of the indicator are annotated neighboring pairs from the similar image set. The geodesic distances of its neighbors are updated according to the combination of the texture and boundary features and the indication value. Experiments on three datasets show that our method outperforms the traditional learning based methods and the previous label transfer method for the semantic segmentation work.

Keywords

Input Image Geodesic Distance Similar Image Propagation Indicator Seed Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Tu, Z.: Auto-context and its application to high-level vision tasks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Press, Piscataway (2008)Google Scholar
  2. 2.
    Gould, S., Fulton, R., Koller, D.: Decomposing a Scene into Geometric and Semantically Consistent Regions. In: IEEE International Conference on Computer Vision, pp. 1–8. IEEE Press, Piscataway (2009)CrossRefGoogle Scholar
  3. 3.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Press, Piscataway (2008)Google Scholar
  4. 4.
    Han, F., Zhu, S.-C.: Bottom-up/Top-Down Image Parsing by Attribute Graph Grammar. In: IEEE International Conference on Computer Vision, pp. 1778–1785. IEEE Press, Piscataway (2005)Google Scholar
  5. 5.
    Zhao, P., Fang, T., Xiao, J., Zhang, H., Zhao, Q., Quan, L.: Rectilinear parsing of architecture in urban environment. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 342–349. IEEE Press, Piscataway (2010)CrossRefGoogle Scholar
  6. 6.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context. Int. J. Comput. Vis. 81, 2–23 (2009)CrossRefGoogle Scholar
  7. 7.
    Russell, B., Torralba, A., Murphy, K., Freeman, W.: Labelme: A database and web-based tool for image annotation. MIT AI Lab Memo (2005)Google Scholar
  8. 8.
    Torralba, A., Fergus, R., Freeman, W.: 80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1958–1970 (2008)CrossRefGoogle Scholar
  9. 9.
    Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing: Label transfer via dense scene alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1972–1979. IEEE Press, Piscataway (2009)CrossRefGoogle Scholar
  10. 10.
    Tighe, J., Lazebnik, S.: SuperParsing: Scalable Nonparametric Image Parsing with Superpixels. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 352–365. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Zhang, H., Xiao, J., Quan, L.: Supervised Label Transfer for Semantic Segmentation of Street Scenes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 561–574. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Zhang, H., Fang, T., Chen, X., Zhao, Q., Quan, L.: Partial similarity based nonparametric scene parsing in certain environment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2241–2248. IEEE Press, Piscataway (2011)Google Scholar
  13. 13.
    Oliva, A., Torralba, A.: Building the gist of a scene: the role of global image features in recognition. Prog. Brain Res. 155, 23–36 (2006)CrossRefGoogle Scholar
  14. 14.
    Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  15. 15.
    Price, B., Morse, B., Cohen, S.: Geodesic Graph Cut for Interactive Image Segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3161–3168. IEEE Press, Piscataway (2010)Google Scholar
  16. 16.
    Gulshany, V., Rotherz, C., Criminisiz, A., Blakez, A., Zisserman, A.: Geodesic Star Convexity for Interactive Image Segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3129–3136. IEEE Press, Piscataway (2010)Google Scholar
  17. 17.
    Chen, X., Zhao, D., Zhao, Y., Lin, L.: Accurate semantic image labeling by fast geodesic propagation. In: IEEE International Conference on Image processing, pp. 4021–4024. IEEE Press, Piscataway (2009)Google Scholar
  18. 18.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)zbMATHCrossRefGoogle Scholar
  19. 19.
    Xiao, J., Quan, L.: Multiple view semantic segmentation for street view images. In: Proceedings of 12th IEEE International Conference on Computer Vision, pp. 686–693. IEEE Press, Piscataway (2009)CrossRefGoogle Scholar
  20. 20.
    Xiao, J., Fang, T., Zhao, P., Lhuillier, M., Quan, L.: Image-based street-side city modeling. ACM Trans. Graph. 28, 1–12 (2009)CrossRefGoogle Scholar
  21. 21.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour Detection and Hierarchical Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)CrossRefGoogle Scholar
  22. 22.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)zbMATHCrossRefGoogle Scholar
  24. 24.
    Liaw, A., Wiener, M.: Classification and Regression by randomForest. R News 2, 18–22 (2002)Google Scholar
  25. 25.
    Jaiantilal, A.: Classification and regression by randomforest-matlab (2009), http://code.google.com/p/randomforest-matlab
  26. 26.
    Yatziv, L., Bartesaghi, A., Sapiro, G.: O(n) implementation of the fast marching algorithm. J. Comput. Phys. 212, 393–399 (2006)zbMATHCrossRefGoogle Scholar
  27. 27.
    Brostow, G., Fauqueur, J., Cipolla, R.: Semantic Object Classes in Video: A High-Definition Ground Truth Database. Pattern Recognit. Lett. 30, 88–97 (2009)CrossRefGoogle Scholar
  28. 28.
    Bileschi, S.: CBCL streetscenes challenge framework (2007), http://cbcl.mit.edu/software-datasets/streetscenes/

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xiaowu Chen
    • 1
  • Qing Li
    • 1
  • Yafei Song
    • 1
  • Xin Jin
    • 1
  • Qinping Zhao
    • 1
  1. 1.State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and EngineeringBeihang UniversityBeijingChina

Personalised recommendations