Advertisement

Computational Visual Media

, Volume 1, Issue 3, pp 183–195 | Cite as

Robust interactive image segmentation via graph-based manifold ranking

  • Hong LiEmail author
  • Wen Wu
  • Enhua Wu
Open Access
Research Article

Abstract

Interactive image segmentation aims at classifying the image pixels into foreground and background classes given some foreground and background markers. In this paper, we propose a novel framework for interactive image segmentation that builds upon graph-based manifold ranking model, a graph-based semi-supervised learning technique which can learn very smooth functions with respect to the intrinsic structure revealed by the input data. The final segmentation results are improved by overcoming two core problems of graph construction in traditional models: graph structure and graph edge weights. The user provided scribbles are treated as the must-link and must-not-link constraints. Then we model the graph as an approximatively k-regular sparse graph by integrating these constraints and our extended neighboring spatial relationships into graph structure modeling. The content and labels driven locally adaptive kernel parameter is proposed to tackle the insufficiency of previous models which usually employ a unified kernel parameter. After the graph construction, a novel three-stage strategy is proposed to get the final segmentation results. Due to the sparsity and extended neighboring relationships of our constructed graph and usage of superpixels, our model can provide nearly real-time, user scribble insensitive segmentations which are two core demands in interactive image segmentation. Last but not least, our framework is very easy to be extended to multi-label segmentation, and for some less complicated scenarios, it can even get the segmented object through single line interaction. Experimental results and comparisons with other state-of-the-art methods demonstrate that our framework can efficiently and accurately extract foreground objects from background.

Keywords

interactive image segmentation graph structure graph edge weights manifold ranking relevance inference 

References

  1. [1]
    Hu, S.-M.; Chen, T.; Xu, K.; Cheng, M.-M.; Martin R. R. Internet visual media processing: A survey with graphics and vision applications. The Visual Computer Vol. 29, No. 5,' 393–405, 2013.CrossRefGoogle Scholar
  2. [2]
    Comaniciu, D.; Meer P. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 5, 603–619, 2002.CrossRefGoogle Scholar
  3. [3]
    Xiao, C.; Liu M. Efficient mean-shift clustering using Gaussian KD-tree. Computer Graphics Forum Vol. 29, No. 7, 2065–2073, 2010.MathSciNetCrossRefGoogle Scholar
  4. [4]
    Vese L. A.; Chan T. F. A multiphase level set framework for image segmentation using the Mumford and Shah model. International Journal of Computer Vision Vol. 50, No. 3, 271–293, 2002.zbMATHCrossRefGoogle Scholar
  5. [5]
    Li, C.; Xu, C.; Gui, C.; Fox M. D. Distance regularized level set evolution and its application to image segmentation. IEEE Transactions on Image Processing Vol. 19, No. 12, 3243–3254, 2010.MathSciNetCrossRefGoogle Scholar
  6. [6]
    Liu, Y.; Yu Y. Interactive image segmentation based on level sets of probabilities. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 2, 202–213, 2012.CrossRefGoogle Scholar
  7. [7]
    Shi, J.; Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 22, No. 8, 888–905, 2000.CrossRefGoogle Scholar
  8. [8]
    Huang, H.; Zhang, L.; Zhang H.-C. RepSnapping: Efficient image cutout for repeated scene elements. Computer Graphics Forum Vol. 30, No. 7, 2059–2066, 2011.CrossRefGoogle Scholar
  9. [9]
    Eigen, D.; Fergus R. Nonparametric image parsing using adaptive neighbor sets. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2799–2806, 2012.CrossRefGoogle Scholar
  10. [10]
    Boykov Y. Y.; Jolly M.-P. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings of the Eighth IEEE International Conference on Computer Vision, Vol. 1, 105–112, 2001.Google Scholar
  11. [11]
    Rother, C.; Kolmogorov, V.; Blake A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309–314, 2004.CrossRefGoogle Scholar
  12. [12]
    Li, Y.; Sun, J.; Tang, C.-K.; Shum H.-Y. Lazy snapping. ACM Transactions on Graphics Vol. 23, No. 3, 303–308, 2004.CrossRefGoogle Scholar
  13. [13]
    Price B. L.; Morse, B.; Cohen S. Geodesic graph cut for interactive image segmentation. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3161–3168, 2010.CrossRefGoogle Scholar
  14. [14]
    Gulshan, V.; Rother, C.; Criminisi, A.; Blake, A.; Zisserman A. Geodesic star convexity for interactive image segmentation. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3129–3136, 2010.CrossRefGoogle Scholar
  15. [15]
    Mortensen E. N.; Barrett W. A. Intelligent scissors for image composition. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, 191–198, 1995.Google Scholar
  16. [16]
    Sundaramoorthi, G.; Yezzi, A.; Mennucci A. C. Coarse-to-fine segmentation and tracking using Sobolev active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 5, 851–864, 2008.CrossRefGoogle Scholar
  17. [17]
    Nguyen T. N. A. N.; Cai, J.; Zhang, J.; Zheng J. Robust interactive image segmentation using convex active contours. IEEE Transactions on Image Processing Vol. 21, No. 8, 3734–3743, 2012.MathSciNetCrossRefGoogle Scholar
  18. [18]
    Grady L. Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 28, No. 11, 1768–1783, 2006.CrossRefGoogle Scholar
  19. [19]
    Kim T. H.; Lee K. M.; Lee S. U. Generative image segmentation using random walks with restart. In: Lecture Notes in Computer Science, Vol. 5304. Berlin Heidelberg: Springer, 264–275, 2008.Google Scholar
  20. [20]
    Yang, W.; Cai, J.; Zheng, J.; Luo J. Userfriendly interactive image segmentation through unified combinatorial user inputs. IEEE Transactions on Image Processing Vol. 19, No. 9, 2470–2479, 2010.MathSciNetCrossRefGoogle Scholar
  21. [21]
    Adams, R.; Bischof L. Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 16, No. 6, 641–647, 1994.CrossRefGoogle Scholar
  22. [22]
    Ning, J.; Zhang, L.; Zhang, D.; Wu C. Interactive image segmentation by maximal similarity based region merging. Pattern Recognition Vol. 43, No. 2, 445–456, 2010.zbMATHCrossRefGoogle Scholar
  23. [23]
    Noma, A.; Graciano A. B. V.; Cesar Jr., R. M.; Consularo L. A.; Bloch I. Interactive image segmentation by matching attributed relational graphs. Pattern Recognition Vol. 45, No. 3, 1159–1179, 2012.CrossRefGoogle Scholar
  24. [24]
    Greig D. M.; Porteous B. T.; Seheult A. H. Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society. Series B (Methodological) Vol. 51, No. 2, 271–279, 1989.Google Scholar
  25. [25]
    Zhou, D.; Bousquet, O.; Lal T. N.; Weston, J.; Schölkopf, B. Learning with local and global consistency. In: Proceedings of Advances in Neural Information Processing Systems, 321–328, 2003.Google Scholar
  26. [26]
    Zhou, D.; Weston, J.; Gretton, A.; Bousquest, O.; Schölkopf, B. Ranking on data manifolds. In: Proceedings of Advances in Neural Information Processing Systems, 2004. Available at http://papersnipscc/paper/2447-ranking-ondata-manifoldspdf.Google Scholar
  27. [27]
    Shen, J.; Du, Y.; Wang, W.; Li X. Lazy random walks for superpixel segmentation. IEEE Transactions on Image Processing Vol. 23, No. 4, 1451–1462, 2014.MathSciNetCrossRefGoogle Scholar
  28. [28]
    Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Susstrunk S. SLIC superpixels compared to stateof–the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274–2282, 2012.CrossRefGoogle Scholar
  29. [29]
    Liu J. Y.; Sun, J.; Shum H. Y. Paint selection. In: Proceedings of ACM SIGGRAPH 2009 papers, Article No. 69, 2009.Google Scholar
  30. [30]
    Kim T. H.; Lee K. M.; Lee S. U. Nonparametric higher-order learning for interactive segmentation. In: Proceedings of 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3201–3208, 2010.CrossRefGoogle Scholar
  31. [31]
    Sinop A. K.; Grady L. A seeded image segmentation framework unifying graph cuts and randomwalker which yields a new algorithm. In: Proceedings of IEEE 11th International Conference on Computer Vision, 1–8, 2007.Google Scholar

Copyright information

© The Author(s) 2015

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Computer and Information ScienceUniversity of MacauMacauChina
  2. 2.Chinese Academy of SciencesBeijingChina

Personalised recommendations