Interactive Image Segmentation Techniques

  • Jia He
  • Chang-Su Kim
  • C.-C. Jay Kuo
Part of the SpringerBriefs in Electrical and Computer Engineering book series (BRIEFSELECTRIC)


Interactive image segmentation techniques are semiautomatic image processing approaches. They are used to track object boundaries and/or propagate labels to other regions by following user guidance so that heterogeneous regions in one image can be separated. User interactions provide the high-level information indicating the “object” and “background” regions. Then, various features such as locations, color intensities, local gradients can be extracted and used to provide the information to separate desired objects from the background. We introduce several interactive image segmentation methods according to different models and used image features.


Graph-cut Random walks Active contour Matching attributed relational graph Region merging Matting 


  1. 1.
    Boykov Y, Jolly M (2001) Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: Proceeding of 8th IEEE international conference on computer vision, ICCV 2001, IEEE, vol 1, pp 105–112Google Scholar
  2. 2.
    Greig D, Porteous B, Seheult A (1989) Exact maximum a posteriori estimation for binary images. J Roy Stat Soc 51(2):271–279Google Scholar
  3. 3.
    McGuinness K, O’Connor N (2010) A comparative evaluation of interactive segmentation algorithms. Pattern Recogn 43(2):434–444MATHCrossRefGoogle Scholar
  4. 4.
    Rother C, Kolmogorov V, Blake A (2004) “Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314CrossRefGoogle Scholar
  5. 5.
    Li Y, Sun J, Tang CK, Shum HY (2004) Lazy snapping. ACM Trans Graph 23(3):303–308CrossRefGoogle Scholar
  6. 6.
    Boykov Y, Funka-Lea G (2006) Graph cuts and efficient n-d image segmentation. Int J Comput Vision 70(2):109–131CrossRefGoogle Scholar
  7. 7.
    Boykov Y, Veksler O (2006) Graph cuts in vision and graphics: theories and applications. Handbook of mathematical models in computer vision, pp. 79–96Google Scholar
  8. 8.
    Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137CrossRefGoogle Scholar
  9. 9.
    Ford LR, Fulkerson DR (1962) Flows in networks. Princeton University Press, PrincetonGoogle Scholar
  10. 10.
    Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Trans Pattern Anal Mach Intell 15(11):1101–1113CrossRefGoogle Scholar
  11. 11.
    Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell 13(6):583–598CrossRefGoogle Scholar
  12. 12.
    Price B, Morse B, Cohen S (2010) Geodesic graph cut for interactive image segmentation. In: 2010 IEEE conference on computer vision and pattern recognition, IEEE, pp 3161–3168Google Scholar
  13. 13.
    Bai X, Sapiro G (2007) A geodesic framework for fast interactive image and video segmentation and matting. In: IEEE 11th international conference on computer vision, ICCV 2007, IEEE, pp 1–8Google Scholar
  14. 14.
    Criminisi A, Sharp T, Blake A (2008) Geos: geodesic image segmentation. Comput Vis ECCV 2008:99–112Google Scholar
  15. 15.
    Freedman D, Zhang T (2005) Interactive graph cut based segmentation with shape priors. In: IEEE computer society conference on computer vision and pattern recognition, IEEE, vol 1, pp 755–762Google Scholar
  16. 16.
    Veksler O (2008) Star shape prior for graph-cut image segmentation. Comput Vis ECCV 2008:454–467Google Scholar
  17. 17.
    Gulshan V, Rother C, Criminisi A, Blake A, Zisserman A (2010) Geodesic star convexity for interactive image segmentation. In: 2010 IEEE conference on computer vision and pattern recognition(CVPR), IEEE, pp 3129–3136Google Scholar
  18. 18.
    Vicente S, Kolmogorov V, Rother C (2008) Graph cut based image segmentation with connectivity priors. In: IEEE Conference on Computer vision and pattern recognition, CVPR 2008, IEEE, pp 1–8Google Scholar
  19. 19.
    Wang J, Bhat P, Colburn R, Agrawala M, Cohen M (2005) Interactive video cutout. In: ACM transactions on graphics, ACM, pp 585–594Google Scholar
  20. 20.
    Lombaert H, Sun Y, Grady L, Xu C (2005) A multilevel banded graph cuts method for fast image segmentation. In: 10th IEEE international conference on computer vision, ICCV 2005, IEEE, vol 1, pp 259–265Google Scholar
  21. 21.
    Pratt W (2007) Digital image processing: PIKS scientific inside. Wiley-Interscience Publication, HobokenGoogle Scholar
  22. 22.
    Lakshmi S, Sankaranarayanan D (2010) A study of edge detection techniques for segmentation computing approaches. Int J Comput Appl IJCA 1:7–10Google Scholar
  23. 23.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679–698CrossRefGoogle Scholar
  24. 24.
    Bao P, Zhang L, Wu X (2005) Canny edge detection enhancement by scale multiplication. IEEE Trans Pattern Anal Mach Intell 27(9):1485–1490CrossRefGoogle Scholar
  25. 25.
    Demigny D (2002) On optimal linear filtering for edge detection. IEEE Trans Image Process 11(7):728–737CrossRefGoogle Scholar
  26. 26.
    Petrou M, Kittler J (1991) Optimal edge detectors for ramp edges. IEEE Trans Pattern Anal Mach Intell 13(5):483–491CrossRefGoogle Scholar
  27. 27.
    Torre V, Poggio T (1986) On edge detection. IEEE Trans Pattern Anal Mach Intell 2:147–163CrossRefGoogle Scholar
  28. 28.
    Marr D, Hildreth E (1980) Theory of edge detection. Proc Roy Soc Lond Ser B Biol Sci 207(1167):187–217CrossRefGoogle Scholar
  29. 29.
    Haralick R (1984) Digital step edges from zero crossing of second directional derivatives. IEEE Trans Pattern Anal Mach Intell 1:58–68CrossRefGoogle Scholar
  30. 30.
    Gauch J, Hsia C (1992) Comparison of three-color image segmentation algorithms in four color spaces. In: Applications in optical science and engineering. International Society for Optics and Photonics, Bellingham, pp 1168–1181Google Scholar
  31. 31.
    Koschan A, Abidi M (2005) Detection and classification of edges in color images. IEEE Signal Process Mag 22(1):64–73CrossRefGoogle Scholar
  32. 32.
    Mortensen E, Morse B, Barrett W, Udupa J (1992) Adaptive boundary detection using ‘live-wire’ two-dimensional dynamic programming. In: Proceedings of computers in cardiology, IEEE, pp 635–638Google Scholar
  33. 33.
    Udupa J, Samarasekera S, Barrett W (1992) Boundary detection via dynamic programming. In: Proceedings of SPIE-the international society for optical engineering, pp 33–33Google Scholar
  34. 34.
    Barrett W, Mortensen E (1996) Fast, accurate, and reproducible live-wire boundary extraction. In: Visualization in biomedical computing. Springer, New York, pp 183–192Google Scholar
  35. 35.
    Barrett W, Mortensen E (1997) Interactive live-wire boundary extraction. Med Image Anal 1(4):331–341CrossRefGoogle Scholar
  36. 36.
    Falcão A, Udupa J, Miyazawa F (2000) An ultra-fast user-steered image segmentation paradigm: live wire on the fly. IEEE Trans Med Imaging 19(1):55–62CrossRefGoogle Scholar
  37. 37.
    Falcão A, Udupa J, Samarasekera S, Hirsch B (1996) User-steered image boundary segmentation. In: Proceedings of SPIE on medical imaging, vol 2710, pp 278–288Google Scholar
  38. 38.
    Falcão A, Udupa J, Samarasekera S, Sharma S, Hirsch B, Lotufo R (1998) User-steered image segmentation paradigms: live wire and live lane. Graph Models Image Process 60(4):233–260CrossRefGoogle Scholar
  39. 39.
    Kang H, Shin S (2002) Enhanced lane: interactive image segmentation by incremental path map construction. Graph Models 64(5):282–303MATHCrossRefGoogle Scholar
  40. 40.
    Mortensen E, Barrett W (1998) Interactive segmentation with intelligent scissors. Graph Models Image Process 60(5):349–384MATHCrossRefGoogle Scholar
  41. 41.
    Mortensen EN, Barrett WA (1995) Intelligent scissors for image composition. In: Proceedings of the 22nd annual conference on computer graphics and interactive techniques, SIGGRAPH ’95. ACM, New York, pp 191–198Google Scholar
  42. 42.
    Gimp G (2008) Image manipulation program. User manual, edge-detect filters, sobel. The GIMP Documentation TeamGoogle Scholar
  43. 43.
    Dijkstra E (1959) A note on two problems in connexion with graphs. Numer math 1(1):269–271MathSciNetMATHCrossRefGoogle Scholar
  44. 44.
    Mortensen E, Barrett W (1999) Toboggan-based intelligent scissors with a four-parameter edge model. In: IEEE computer society conference on computer vision and pattern recognition, IEEE, vol 2Google Scholar
  45. 45.
    Blake A, Isard M et al (1998) Active contours, vol 1. Springer, LondonGoogle Scholar
  46. 46.
    McInerney T, Terzopoulos D (1996) Deformable models in medical image analysis: a survey. In: Proceedings of the workshop on mathematical methods in biomedical image analysis, IEEE, pp 171–180Google Scholar
  47. 47.
    Singh A, Terzopoulos D, Goldgof D (1998) Deformable models in medical image analysis. IEEE Computer Society Press, New YorkGoogle Scholar
  48. 48.
    Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis 1(4):321–331CrossRefGoogle Scholar
  49. 49.
    Papari G, Petkov N (2011) Edge and line oriented contour detection: state of the art. Image Vis Comput 29(2):79–103CrossRefGoogle Scholar
  50. 50.
    Brigger P, Hoeg J, Unser M (2000) B-spline snakes: a flexible tool for parametric contour detection. IEEE Trans Image Process 9(9):1484–1496MathSciNetMATHCrossRefGoogle Scholar
  51. 51.
    Cohen L, Cohen I (1993) Finite-element methods for active contour models and balloons for 2-d and 3-d images. IEEE Trans Pattern Anal Mach Intell 15(11):1131–1147CrossRefGoogle Scholar
  52. 52.
    Gunn S, Nixon M (1995) Improving snake performance via a dual active contour. In: Computer analysis of images and patterns. Springer, Berlin, pp 600–605Google Scholar
  53. 53.
    Gunn S, Nixon M (1997) A robust snake implementation; a dual active contour. IEEE Trans Pattern Anal Mach Intell 19(1):63–68CrossRefGoogle Scholar
  54. 54.
    Leymarie F, Levine M (1993) Tracking deformable objects in the plane using an active contour model. IEEE Trans Pattern Anal Mach Intell 15(6):617–634CrossRefGoogle Scholar
  55. 55.
    Li B, Acton S (2007) Active contour external force using vector field convolution for image segmentation. IEEE Trans Image Process 16(8):2096–2106MathSciNetCrossRefGoogle Scholar
  56. 56.
    Mishra A, Fieguth P, Clausi D (2011) Decoupled active contour (dac) for boundary detection. IEEE Trans Pattern Anal Mach Intell 33(2):310–324CrossRefGoogle Scholar
  57. 57.
    Shih F, Zhang K (2007) Locating object contours in complex background using improved snakes. Comput Vis Image Underst 105(2):93–98MathSciNetCrossRefGoogle Scholar
  58. 58.
    Sundaramoorthi G, Yezzi A, Mennucci A (2008) Coarse-to-fine segmentation and tracking using sobolev active contours. IEEE Trans Pattern Anal Mach Intell 30(5):851–864CrossRefGoogle Scholar
  59. 59.
    Wong Y, Yuen P, Tong C (1998) Segmented snake for contour detection. Pattern Recogn 31(11):1669–1679CrossRefGoogle Scholar
  60. 60.
    Xu C, Prince J (1998) Snakes, shapes, and gradient vector flow. IEEE Trans Image Process 7(3):359–369MathSciNetMATHCrossRefGoogle Scholar
  61. 61.
    Bresson X, Esedoglu S, Vandergheynst P, Thiran J, Osher S (2007) Fast global minimization of the active contour/snake model. J Math Imaging Vis 28(2):151–167MathSciNetCrossRefGoogle Scholar
  62. 62.
    Caselles V, Catté F, Coll T, Dibos F (1993) A geometric model for active contours in image processing. Numer Math 66(1):1–31MathSciNetMATHCrossRefGoogle Scholar
  63. 63.
    Chan T, Vese L (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277MATHCrossRefGoogle Scholar
  64. 64.
    Malladi R, Sethian J, Vemuri B (1995) Shape modeling with front propagation: a level set approach. IEEE Transa Pattern Anal Mach Intell 17(2):158–175CrossRefGoogle Scholar
  65. 65.
    Zhu S, Yuille A (1996) Region competition: Unifying snakes, region growing, and bayes/mdl for multiband image segmentation. IEEE Trans Pattern Anal Mach Intell 18(9):884–900CrossRefGoogle Scholar
  66. 66.
    Jalba A, Wilkinson M, Roerdink J (2004) Cpm: a deformable model for shape recovery and segmentation based on charged particles. IEEE Trans Pattern Anal Mach Intell 26(10):1320–1335CrossRefGoogle Scholar
  67. 67.
    Xie X, Mirmehdi M (2008) Mac: magnetostatic active contour model. IEEE Trans Pattern Anal Mach Intell 30(4):632–646CrossRefGoogle Scholar
  68. 68.
    Liang J, McInerney T, Terzopoulos D et al (2006) United snakes. Med Image Anal 10(2):215–233CrossRefGoogle Scholar
  69. 69.
    Zienkiewicz O, Taylor R, Zhu J (2005) The finite element method: its basis and fundamentals, vol 1. Butterworth-Heinemann, OxfordGoogle Scholar
  70. 70.
    Ronfard R (1994) Region-based strategies for active contour models. Int J Comput Vis 13(2):229–251CrossRefGoogle Scholar
  71. 71.
    Paragios N, Deriche R (2002) Geodesic active regions and level set methods for supervised texture segmentation. Int J Comput Vis 46(3):223–247MATHCrossRefGoogle Scholar
  72. 72.
    Leventon M, Grimson W, Faugeras O (2000) Statistical shape influence in geodesic active contours. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, vol 1, pp 316–323Google Scholar
  73. 73.
    Nguyen TNA, Cai J, Zhang J, Zheng J (2012) Robust interactive image segmentation using convex active contours. IEEE Transactions on Image Processing 21(8):3734–3743MathSciNetCrossRefGoogle Scholar
  74. 74.
    Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT press, CambridgeGoogle Scholar
  75. 75.
    TT II C, EE C, Image segmentation and techniques: a reviewGoogle Scholar
  76. 76.
    Grady L (2006) Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell 28(11):1768–1783CrossRefGoogle Scholar
  77. 77.
    Grady L, Sinop A (2008) Fast approximate random walker segmentation using eigenvector precomputation. In: IEEE conference on computer vision and pattern recognition, CVPR 2008, IEEE, pp 1–8Google Scholar
  78. 78.
    Kim T, Lee K, Lee S (2008) Generative image segmentation using random walks with restart. Comput Vis ECCV 2008:264–275Google Scholar
  79. 79.
    Courant R, Hilbert D (1944) Methods of mathematical physics. Interscience Publishers Inc., New YorkGoogle Scholar
  80. 80.
    Dodziuk J (1984) Difference equations, isoperimetric inequality and transience of certain random walks. Trans Am Math Soc 284:787–794Google Scholar
  81. 81.
    Yang W, Cai J, Zheng J, Luo J (2010) User-friendly interactive image segmentation through unified combinatorial user inputs. IEEE Trans Image Process 19(9):2470–2479MathSciNetCrossRefGoogle Scholar
  82. 82.
    Pan J, Yang H, Faloutsos C, Duygulu P (2004) Automatic multimedia cross-modal correlation discovery. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, pp 653–658Google Scholar
  83. 83.
    Tong H, Faloutsos C, Pan J (2008) Random walk with restart: fast solutions and applications. Knowl Inf Syst 14(3):327–346MATHCrossRefGoogle Scholar
  84. 84.
    Jeh G, Widom J (2003) Scaling personalized web search. In: Proceedings of the 12th international conference on World Wide Web, ACM, pp 271–279Google Scholar
  85. 85.
    Kamvar S, Haveliwala T, Manning C, Golub G (2003) Exploiting the block structure of the web for computing pagerank. Stanford University Technical ReportGoogle Scholar
  86. 86.
    Fogaras D, Rácz B (2004) Towards scaling fully personalized pagerank. In: Algorithms and Models for the Web-Graph, pp 105–117Google Scholar
  87. 87.
    Beucher S et al (1992) The watershed transformation applied to image segmentation. Scanning Microsc Suppl 6:299–314Google Scholar
  88. 88.
    Adams R, Bischof L (1994) Seeded region growing. IEEE Trans Pattern Anal Mach Intell 16(6):641–647CrossRefGoogle Scholar
  89. 89.
    Horowitz SL, Pavlidis T (1974) Picture segmentation by a directed split-and-merge procedure. In: Proceedings of the 2nd international joint conference on pattern recognition, vol 424, p 433Google Scholar
  90. 90.
    Raja D, Khadir A, Ahamed D (2009) Moving toward region-based image segmentation techniques: a study. J Theor Appl Inf Technol 81–87Google Scholar
  91. 91.
    Haris K, Efstratiadis S, Maglaveras N, Katsaggelos A (1998) Hybrid image segmentation using watersheds and fast region merging. IEEE Trans Image Process 7(12):1684–1699CrossRefGoogle Scholar
  92. 92.
    Mohana Rao K, Dempster A (2002) Modification on distance transform to avoid over-segmentation and under-segmentation. In: Video/Image processing and multimedia communications 4th EURASIP-IEEE region 8 international symposium on VIPromCom, IEEE, pp 295–301Google Scholar
  93. 93.
    Edge detection and image segmentation (edison) system (2009).
  94. 94.
    Mehnert A, Jackway P (1997) An improved seeded region growing algorithm. Pattern Recogn Lett 18(10):1065–1071CrossRefGoogle Scholar
  95. 95.
    Fan J, Zeng G, Body M, Hacid M (2005) Seeded region growing: an extensive and comparative study. Pattern Recogn Lett 26(8):1139–1156CrossRefGoogle Scholar
  96. 96.
    Vezhnevets V, Konouchine V (2005) Growcut: interactive multi-label nd image segmentation by cellular automata. In: Proceedings of graphicon, pp 150–156Google Scholar
  97. 97.
    Von Neumann J, Burks A (1966) Theory of self-reproducing automata. University of Illinois Press, UrbanaGoogle Scholar
  98. 98.
    Das D (2012) A survey on cellular automata and its applications. In: Global trends in computing and communication systems, pp 753–762Google Scholar
  99. 99.
    Kari J (2005) Theory of cellular automata: a survey. Theor Comput Sci 334(1):3–33MathSciNetMATHCrossRefGoogle Scholar
  100. 100.
    Kauffmann C, Piché N (2010) Seeded and medical image segmentation by cellular automaton on GPU. Int J Comput Assist Radiol Surg 5(3):251–262CrossRefGoogle Scholar
  101. 101.
    Grady L, Funka-Lea G (2004) Multi-label image segmentation for medical applications based on graph-theoretic electrical potentials. In: Computer vision and mathematical methods in medical and biomedical image, analysis, pp 230–245Google Scholar
  102. 102.
    Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recogn 43(2):445–456MATHCrossRefGoogle Scholar
  103. 103.
    Noma A, Graciano A, Consularo L, Bloch I (2012) Interactive image segmentation by matching attributed relational graphs. Pattern Recogn 45(3):1159–1179CrossRefGoogle Scholar
  104. 104.
    Consularo L, Cesar R, Bloch I (2007) Structural image segmentation with interactive model generation. In: IEEE international conference on image processing, vol 6, IEEE, pp VI-45Google Scholar
  105. 105.
    Bunke H (2000) Recent developments in graph matching. In: 15th international conference on pattern recognition, vol 2, IEEE, pp 117–124Google Scholar
  106. 106.
    Noma A, Pardo A (2011) Structural matching of 2d electrophoresis gels using deformed graphs. Pattern Recogn Lett 32(1):3–11CrossRefGoogle Scholar
  107. 107.
    Lin H, Tai Y, Brown M (2011) Motion regularization for matting motion blurred objects. IEEE Trans Pattern Anal Mach Intell 33(11):2329–2336CrossRefGoogle Scholar
  108. 108.
    Porter T, Duff T (1984) Compositing digital images. ACM Siggraph Comput Graph 18(3):253–259CrossRefGoogle Scholar
  109. 109.
    Wang J, Cohen MF (2005) An iterative optimization approach for unified image segmentation and matting. In: 10th IEEE international conference on computer vision, ICCV 2005, vol 2, IEEE, pp 936–943Google Scholar
  110. 110.
    Wang J, Cohen MF (2008) Image and video matting, vol 3. Now Publishers, HanoverGoogle Scholar
  111. 111.
    Wang J, Agrawala M, Cohen MF (2007) Soft scissors: an interactive tool for realtime high quality matting. In: ACM Transactions on Graphics (TOG), vol 26, p 9, ACMGoogle Scholar
  112. 112.
    Anh NTN, Cai J, Zhang J, Zheng J (2012) Constrained active contours for boundary refinement in interactive image segmentation. In: 2012 IEEE international symposium on circuits and systems (ISCAS), IEEE, pp 870–873Google Scholar

Copyright information

© The Author(s) 2014

Authors and Affiliations

  1. 1.Department of Electrical EngineeringUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.School of Electrical EngineeringKorea UniversitySeoulRepublic of South Korea
  3. 3.Department of Electrical EngineeringUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations