Abstract
Interactive image segmentation aims at classifying the image pixels into foreground and background classes given some foreground and background markers. In this paper, we propose a novel framework for interactive image segmentation that builds upon graph-based manifold ranking model, a graph-based semi-supervised learning technique which can learn very smooth functions with respect to the intrinsic structure revealed by the input data. The final segmentation results are improved by overcoming two core problems of graph construction in traditional models: graph structure and graph edge weights. The user provided scribbles are treated as the must-link and must-not-link constraints. Then we model the graph as an approximatively k-regular sparse graph by integrating these constraints and our extended neighboring spatial relationships into graph structure modeling. The content and labels driven locally adaptive kernel parameter is proposed to tackle the insufficiency of previous models which usually employ a unified kernel parameter. After the graph construction, a novel three-stage strategy is proposed to get the final segmentation results. Due to the sparsity and extended neighboring relationships of our constructed graph and usage of superpixels, our model can provide nearly real-time, user scribble insensitive segmentations which are two core demands in interactive image segmentation. Last but not least, our framework is very easy to be extended to multi-label segmentation, and for some less complicated scenarios, it can even get the segmented object through single line interaction. Experimental results and comparisons with other state-of-the-art methods demonstrate that our framework can efficiently and accurately extract foreground objects from background.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Hu, S.-M.; Chen, T.; Xu, K.; Cheng, M.-M.; Martin R. R. Internet visual media processing: A survey with graphics and vision applications. The Visual Computer Vol. 29, No. 5,' 393–405, 2013.
Comaniciu, D.; Meer P. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 5, 603–619, 2002.
Xiao, C.; Liu M. Efficient mean-shift clustering using Gaussian KD-tree. Computer Graphics Forum Vol. 29, No. 7, 2065–2073, 2010.
Vese L. A.; Chan T. F. A multiphase level set framework for image segmentation using the Mumford and Shah model. International Journal of Computer Vision Vol. 50, No. 3, 271–293, 2002.
Li, C.; Xu, C.; Gui, C.; Fox M. D. Distance regularized level set evolution and its application to image segmentation. IEEE Transactions on Image Processing Vol. 19, No. 12, 3243–3254, 2010.
Liu, Y.; Yu Y. Interactive image segmentation based on level sets of probabilities. IEEE Transactions on Visualization and Computer Graphics Vol. 18, No. 2, 202–213, 2012.
Shi, J.; Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 22, No. 8, 888–905, 2000.
Huang, H.; Zhang, L.; Zhang H.-C. RepSnapping: Efficient image cutout for repeated scene elements. Computer Graphics Forum Vol. 30, No. 7, 2059–2066, 2011.
Eigen, D.; Fergus R. Nonparametric image parsing using adaptive neighbor sets. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2799–2806, 2012.
Boykov Y. Y.; Jolly M.-P. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings of the Eighth IEEE International Conference on Computer Vision, Vol. 1, 105–112, 2001.
Rother, C.; Kolmogorov, V.; Blake A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309–314, 2004.
Li, Y.; Sun, J.; Tang, C.-K.; Shum H.-Y. Lazy snapping. ACM Transactions on Graphics Vol. 23, No. 3, 303–308, 2004.
Price B. L.; Morse, B.; Cohen S. Geodesic graph cut for interactive image segmentation. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3161–3168, 2010.
Gulshan, V.; Rother, C.; Criminisi, A.; Blake, A.; Zisserman A. Geodesic star convexity for interactive image segmentation. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3129–3136, 2010.
Mortensen E. N.; Barrett W. A. Intelligent scissors for image composition. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, 191–198, 1995.
Sundaramoorthi, G.; Yezzi, A.; Mennucci A. C. Coarse-to-fine segmentation and tracking using Sobolev active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 5, 851–864, 2008.
Nguyen T. N. A. N.; Cai, J.; Zhang, J.; Zheng J. Robust interactive image segmentation using convex active contours. IEEE Transactions on Image Processing Vol. 21, No. 8, 3734–3743, 2012.
Grady L. Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 28, No. 11, 1768–1783, 2006.
Kim T. H.; Lee K. M.; Lee S. U. Generative image segmentation using random walks with restart. In: Lecture Notes in Computer Science, Vol. 5304. Berlin Heidelberg: Springer, 264–275, 2008.
Yang, W.; Cai, J.; Zheng, J.; Luo J. Userfriendly interactive image segmentation through unified combinatorial user inputs. IEEE Transactions on Image Processing Vol. 19, No. 9, 2470–2479, 2010.
Adams, R.; Bischof L. Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 16, No. 6, 641–647, 1994.
Ning, J.; Zhang, L.; Zhang, D.; Wu C. Interactive image segmentation by maximal similarity based region merging. Pattern Recognition Vol. 43, No. 2, 445–456, 2010.
Noma, A.; Graciano A. B. V.; Cesar Jr., R. M.; Consularo L. A.; Bloch I. Interactive image segmentation by matching attributed relational graphs. Pattern Recognition Vol. 45, No. 3, 1159–1179, 2012.
Greig D. M.; Porteous B. T.; Seheult A. H. Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society. Series B (Methodological) Vol. 51, No. 2, 271–279, 1989.
Zhou, D.; Bousquet, O.; Lal T. N.; Weston, J.; Schölkopf, B. Learning with local and global consistency. In: Proceedings of Advances in Neural Information Processing Systems, 321–328, 2003.
Zhou, D.; Weston, J.; Gretton, A.; Bousquest, O.; Schölkopf, B. Ranking on data manifolds. In: Proceedings of Advances in Neural Information Processing Systems, 2004. Available at http://papersnipscc/paper/2447-ranking-ondata-manifoldspdf.
Shen, J.; Du, Y.; Wang, W.; Li X. Lazy random walks for superpixel segmentation. IEEE Transactions on Image Processing Vol. 23, No. 4, 1451–1462, 2014.
Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Susstrunk S. SLIC superpixels compared to stateof–the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274–2282, 2012.
Liu J. Y.; Sun, J.; Shum H. Y. Paint selection. In: Proceedings of ACM SIGGRAPH 2009 papers, Article No. 69, 2009.
Kim T. H.; Lee K. M.; Lee S. U. Nonparametric higher-order learning for interactive segmentation. In: Proceedings of 2010 IEEE Conference on Computer Vision and Pattern Recognition, 3201–3208, 2010.
Sinop A. K.; Grady L. A seeded image segmentation framework unifying graph cuts and randomwalker which yields a new algorithm. In: Proceedings of IEEE 11th International Conference on Computer Vision, 1–8, 2007.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Hong Li is currently a third-year Ph.D. candidate at Macau University (Macao, China), Computer Graphics Laboratory. His researches focus on image processing, semi-supervised learning, computer vision, and so on.
Wen Wu received her Ph.D. degree in computer science and engineering from The Chinese University of Hong Kong. She is currently an assistant professor in Department of Computer and Information Science at University of Macau. Her research interests include medical simulation, virtual reality, and physically-based animation.
Enhua Wu is a professor of the State Key Laboratory of Computer Science at the Chinese Academy of Sciences Institute of Software and the University of Macau. His interests include VR, realistic-image synthesis, and data visualization. He received his Ph.D. in computer science from University of Manchester.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Li, H., Wu, W. & Wu, E. Robust interactive image segmentation via graph-based manifold ranking. Comp. Visual Media 1, 183–195 (2015). https://doi.org/10.1007/s41095-015-0024-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-015-0024-2