Interactive Image Segmentation Using Constrained Dominant Sets

  • Eyasu ZemeneEmail author
  • Marcello Pelillo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9912)


We propose a new approach to interactive image segmentation based on some properties of a family of quadratic optimization problems related to dominant sets, a well-known graph-theoretic notion of a cluster which generalizes the concept of a maximal clique to edge-weighted graphs. In particular, we show that by properly controlling a regularization parameter which determines the structure and the scale of the underlying problem, we are in a position to extract groups of dominant-set clusters which are constrained to contain user-selected elements. The resulting algorithm can deal naturally with any type of input modality, including scribbles, sloppy contours, and bounding boxes, and is able to robustly handle noisy annotations on the part of the user. Experiments on standard benchmark datasets show the effectiveness of our approach as compared to state-of-the-art algorithms on a variety of natural images under several input conditions.


Interactive segmentation Dominant sets Quadratic optimization 



This work has been partly supported by Samsung Global Research Outreach Program.

Supplementary material

419983_1_En_17_MOESM1_ESM.pdf (1.8 mb)
Supplementary material 1 (pdf 1878 KB)


  1. 1.
    Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRefGoogle Scholar
  2. 2.
    Lempitsky, V.S., Kohli, P., Rother, C., Sharp, T.: Image segmentation with a bounding box prior. In: ICCV, pp. 277–284 (2009)Google Scholar
  3. 3.
    Wu, J., Zhao, Y., Zhu, J., Luo, S., Tu, Z.: Milcut: a sweeping line multiple instance learning paradigm for interactive image segmentation. In: CVPR, pp. 256–263 (2014)Google Scholar
  4. 4.
    Bai, X., Sapiro, G.: Geodesic matting: a framework for fast interactive image and video segmentation and matting. Int. J. Comput. Vis. 82(2), 113–132 (2009)CrossRefGoogle Scholar
  5. 5.
    Li, Y., Sun, J., Tang, C., Shum, H.: Lazy snapping. ACM Trans. Graph. 23(3), 303–308 (2004)CrossRefGoogle Scholar
  6. 6.
    Protiere, A., Sapiro, G.: Interactive image segmentation via adaptive weighted distances. IEEE Trans. Image Process. 16(4), 1046–1057 (2007)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: ICCV, pp. 105–112 (2001)Google Scholar
  8. 8.
    Mortensen, E.N., Barrett, W.A.: Interactive segmentation with intelligent scissors. Graph. Models Image Process. 60(5), 349–384 (1998)CrossRefzbMATHGoogle Scholar
  9. 9.
    Yu, H., Zhou, Y., Qian, H., Xian, M., Lin, Y., Guo, D., Zheng, K., Abdelfatah, K., Wang, S.: Loosecut: interactive image segmentation with loosely bounded boxes. CoRR abs/1507.03060 (2015)Google Scholar
  10. 10.
    Xian, M., Zhang, Y., Cheng, H.D., Xu, F., Ding, J.: Neutro-connectedness cut. CoRR abs/1512.06285Google Scholar
  11. 11.
    Bai, J., Wu, X.: Error-tolerant scribbles based interactive image segmentation. In: CVPR, pp. 392–399 (2014)Google Scholar
  12. 12.
    Jain, S.D., Grauman, K.: Predicting sufficient annotation strength for interactive foreground segmentation. In: ICCV, pp. 1313–1320 (2013)Google Scholar
  13. 13.
    Pavan, M., Pelillo, M.: A new graph-theoretic approach to clustering and segmentation. In: CVPR, pp. 145–152 (2003)Google Scholar
  14. 14.
    Pavan, M., Pelillo, M.: Dominant sets and pairwise clustering. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 167–172 (2007)CrossRefGoogle Scholar
  15. 15.
    Rota Bulò, S., Pelillo, M.: A game-theoretic approach to hypergraph clustering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1312–1327 (2013)CrossRefGoogle Scholar
  16. 16.
    Pavan, M., Pelillo, M.: Dominant sets and hierarchical clustering. In: ICCV, pp. 362–369 (2003)Google Scholar
  17. 17.
    Luenberger, D.G., Ye, Y.: Linear and Nonlinear Programming. Springer, New York (2008)zbMATHGoogle Scholar
  18. 18.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, New York (1985)CrossRefzbMATHGoogle Scholar
  19. 19.
    Hoiem, D., Efros, A.A., Hebert, M.: Geometric context from a single image. In: ICCV, pp. 654–661 (2005)Google Scholar
  20. 20.
    Wang, J., Jia, Y., Hua, X., Zhang, C., Quan, L.: Normalized tree partitioning for image segmentation. In: CVPR (2008)Google Scholar
  21. 21.
    Xiao, J., Quan, L.: Multiple view semantic segmentation for street view images. In: ICCV, pp. 686–693 (2009)Google Scholar
  22. 22.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  23. 23.
    Zhou, Y., Ju, L., Wang, S.: Multiscale superpixels and supervoxels based on hierarchical edge-weighted centroidal voronoi tessellation. In: WACV, pp. 1076–1083 (2015)Google Scholar
  24. 24.
    Arbelaez, P., Maire, M., Fowlkes, C.C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)CrossRefGoogle Scholar
  25. 25.
    Leung, T.K., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vis. 43(1), 29–44 (2001)CrossRefzbMATHGoogle Scholar
  26. 26.
    Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: NIPS, pp. 1601–1608 (2004)Google Scholar
  27. 27.
    Li, H., Meng, F., Ngan, K.N.: Co-salient object detection from multiple images. IEEE Trans. Multimedia 15(8), 1896–1909 (2013)CrossRefGoogle Scholar
  28. 28.
    Tang, M., Ben Ayed, I., Boykov, Y.: Pseudo-bound optimization for binary energies. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part V. LNCS, vol. 8693, pp. 691–707. Springer, Heidelberg (2014)Google Scholar
  29. 29.
    Tang, M., Gorelick, L., Veksler, O., Boykov, Y.: Grabcut in one cut. In: ICCV, pp. 1769–1776 (2013)Google Scholar
  30. 30.
    Price, B.L., Morse, B.S., Cohen, S.: Geodesic graph cut for interactive image segmentation. In: CVPR, pp. 3161–3168 (2010)Google Scholar
  31. 31.
    Yang, W., Cai, J., Zheng, J., Luo, J.: User-friendly interactive image segmentation through unified combinatorial user inputs. IEEE Trans. Image Process. 19(9), 2470–2479 (2010)CrossRefMathSciNetGoogle Scholar
  32. 32.
    McGuinness, K., O’Connor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recogn. 43(2), 434–444 (2010)CrossRefzbMATHGoogle Scholar
  33. 33.
    Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)CrossRefGoogle Scholar
  34. 34.
    Duchenne, O., Audibert, J., Keriven, R., Ponce, J., Ségonne, F.: Segmentation by transduction. In: CVPR (2008)Google Scholar
  35. 35.
    Salembier, P., Garrido, L.: Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans. Image Process. 9(4), 561–576 (2000)CrossRefGoogle Scholar
  36. 36.
    Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)CrossRefGoogle Scholar
  37. 37.
    Friedland, G., Jantz, K., Rojas, R.: SIOX: simple interactive object extraction in still images. In: ISM, pp. 253–260 (2005)Google Scholar
  38. 38.
    Liu, J., Sun, J., Shum, H.: Paint selection. ACM Trans. Graph. 28(3), 303–308 (2009)Google Scholar
  39. 39.
    Sener, O., Ugur, K., Alatan, A.A.: Error-tolerant interactive image segmentation using dynamic and iterated graph-cuts. In: IMMPD@ACM Multimedia, pp. 9–16 (2012)Google Scholar
  40. 40.
    Subr, K., Paris, S., Soler, C., Kautz, J.: Accurate binary image selection from inaccurate user input. Comput. Graph. Forum 32(2), 41–50 (2013)CrossRefGoogle Scholar
  41. 41.
    Rota Bulò, S., Bomze, L.M.: Infection and immunization: a new class of evolutionary game dynamics. Games Econ. Behav. 71(1), 193–211 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  42. 42.
    Catanzaro, B.C., Su, B., Sundaram, N., Lee, Y., Murphy, M., Keutzer, K.: Efficient, high-quality image contour detection. In: ICCV, pp. 2381–2388 (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Ca’ Foscari University of VeniceVeniceItaly

Personalised recommendations