International Journal of Computer Vision

, Volume 70, Issue 2, pp 109–131

Graph Cuts and Efficient N-D Image Segmentation

Article

Abstract

Combinatorial graph cut algorithms have been successfully applied to a wide range of problems in vision and graphics. This paper focusses on possibly the simplest application of graph-cuts: segmentation of objects in image data. Despite its simplicity, this application epitomizes the best features of combinatorial graph cuts methods in vision: global optima, practical efficiency, numerical robustness, ability to fuse a wide range of visual cues and constraints, unrestricted topological properties of segments, and applicability to N-D problems. Graph cuts based approaches to object extraction have also been shown to have interesting connections with earlier segmentation methods such as snakes, geodesic active contours, and level-sets. The segmentation energies optimized by graph cuts combine boundary regularization with region-based properties in the same fashion as Mumford-Shah style functionals. We present motivation and detailed technical description of the basic combinatorial optimization framework for image segmentation via s/t graph cuts. After the general concept of using binary graph cut algorithms for object segmentation was first proposed and tested in Boykov and Jolly (2001), this idea was widely studied in computer vision and graphics communities. We provide links to a large number of known extensions based on iterative parameter re-estimation and learning, multi-scale or hierarchical approaches, narrow bands, and other techniques for demanding photo, video, and medical applications.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amini, A.A., Weymouth, T.E., and Jain, R.C. 1990. Using dynamic programming for solving variational problems in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(9):855–867.CrossRefGoogle Scholar
  2. Appleton, B. and Talbot, H. 2006. Globally minimal surfaces by continuous maximal flows. IEEE transactions on Pattern Analysis and Pattern Recognition (PAMI), 28(1):106–118.CrossRefGoogle Scholar
  3. Blake, A., Rother, C., Brown, M., Perez, P., and Torr, P. 2004. Interactive image segmentation using an adaptive gmmrf model. In European Conference on Computer Vision (ECCV), Prague, Chech Republic.Google Scholar
  4. Boykov, Y. and Kolmogorov, V. 2003. Computing geodesics and minimal surfaces via graph cuts. In International Conference on Computer Vision, vol. I, pp. 26–33.Google Scholar
  5. Boykov, Y., Veksler, O., and Zabih, R. 1998. Markov random fields with efficient approximations. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 648–655.Google Scholar
  6. Boykov, Y. and Jolly, M.-P. 2001. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In International Conference on Computer Vision, vol. I, pp. 105–112, July 2001.Google Scholar
  7. Boykov, Y. and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9):1124–1137.CrossRefGoogle Scholar
  8. Boykov, Y., Kolmogorov, V., Cremers, D., and Delong, A. 2006. An integral solution to surface evolution PDEs via geo-cuts. In European Conference on Computer Vision, LNCS 3953, Graz, Austria, vol. III, pp. 409–422.Google Scholar
  9. Boykov, Y. and Veksler, O. 2006. Graph cuts in vision and graphics: Theories and applications. In: N. Paragios, Y. Chen, and O. Faugeras, (Eds.), Handbook of Mathematical Models in Computer Vision, Springer-Verlag, pp. 79–96.Google Scholar
  10. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222–1239.CrossRefGoogle Scholar
  11. Bray, M., Kohli, P., and Torr, P.H.S. 2006. Posecut: Simultaneous segmentation and 3D pose estimation of humans using dynamic graph-cuts. In European Conference on Computer Vision, Graz, Austria, May 2006, (to appear).Google Scholar
  12. Caselles, V., Kimmel, R., and Sapiro, G. 1997. Geodesic active contours. International Journal of Computer Vision, 22(1):61–79.MATHCrossRefGoogle Scholar
  13. Cohen, L.D. 1991. On active contour models and ballons. Computer Vision, Graphics, and Image Processing: Image Understanding, 53(2):211–218.MATHGoogle Scholar
  14. Cohen, L.D. and Kimmel, R. 1997. Global minimum for active contour models: A minimal path approach. International Journal of Computer Vision, 24(1):57–78.CrossRefGoogle Scholar
  15. Cook, W.J., Cunningham, W.H., Pulleyblank, W.R., and Schrijver, A. 1998. Combinatorial Optimization. John Wiley & Sons.Google Scholar
  16. Cox, I.J., Rao, S.B., and Zhong, Y. 1996. “Ratio regions”: a technique for image segmentation. In International Conference on Pattern Recognition, vol. II, pp. 557–564.Google Scholar
  17. Cremers, D. 2006. Dynamical statistical shape priors for level set based tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence, (to appear).Google Scholar
  18. Cremers, D., Osher, S.J., and Soatto, S. 2006. Kernel density estimation and intrinsic alignment for shape priors in level set segmentation. International Journal of Computer Vision, (to appear).Google Scholar
  19. FalcÃo, A.X., Udupa, J.K., Samarasekera, S., and Sharma, S. 1998. User-steered image segmentation paradigms: Live wire and live lane. Graphical Models and Image Processing, 60:233–260.CrossRefGoogle Scholar
  20. Felzenszwalb, P. and Huttenlocher, D. 2004. Efficient graph-based image segmentation. International Journal of Computer Vision, 59(2):167–181.CrossRefGoogle Scholar
  21. Ford, L. and Fulkerson, D. 1962. Flows in Networks. Princeton University Press.Google Scholar
  22. Funka-Lea, G., Boykov, Y., Florin, C., Jolly, M.-P., Moreau-Gobard, R., Ramaraj, R., and Rinck, D. 2006. Automatic heart isolation for CT coronary visualization using graph-cuts. In IEEE International Symposium on Biomedical Imaging, Arlington, VA, April 2006.Google Scholar
  23. Geiger, D., Gupta, A., Costa, L.A., and Vlontzos, J. 1995. Dynamic programming for detecting, tracking, and matching deformable contours. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(3):294–402.CrossRefGoogle Scholar
  24. Goldberg, A.V. and Tarjan, R.E. 1988. A new approach to the maximum-flow problem. Journal of the Association for Computing Machinery, 35(4):921–940.MATHMathSciNetGoogle Scholar
  25. Grady, L. 2005. Multilabel random walker segmentation using prior models. In IEEE Conference of Computer Vision and Pattern Recognition, San Diego, CA, June 2005, vol. 1, pp. 763–770.Google Scholar
  26. Greig, D., Porteous, B., and Seheult, A. 1989. Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society, Series B, 51(2):271–279.Google Scholar
  27. Griffin, L.D., Colchester, A.C.F., Röll, S.A., and Studholme, C.S. 1994. Hierarchical segmentation satisfying constraints. In British Machine Vision Conference, pp. 135–144.Google Scholar
  28. Haralick, R.M. and Shapiro, L.G. 1992. Computer and Robot Vision. Addison-Wesley Publishing Company.Google Scholar
  29. Hochbaum, D.S. 1998. The pseudoflow algorithm for the maximum flow problem. Manuscript, UC Berkeley, revised 2003, Extended abstract in: The pseudoflow algorithm and the pseudoflow-based simplex for the maximum flow problem. Proceedings of IPCO98, June 1998. Lecture Notes in Computer Science, Bixby, Boyd and Rios-Mercado (Eds.) 1412, Springer, pp. 325–337.Google Scholar
  30. Isard, M. and Blake, A. 1998. Active contours. Springer-Verlag.Google Scholar
  31. Ishikawa, H. and Geiger, D. 1998. Occlusions, discontinuities, and epipolar lines in stereo. In 5th European Conference on Computer Vision, pp. 232–248.Google Scholar
  32. Ishikawa, H. and Geiger, D. 1998. Segmentation by grouping junctions. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 125–131.Google Scholar
  33. Ishikawa, H. 2003. Exact optimization for Markov Random Fields with convex priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1333–1336.CrossRefGoogle Scholar
  34. Jermyn, I.H. and Ishikawa, H. 1999. Globally optimal regions and boundaries. In International Conference on Computer Vision, vol. II, pp. 904–910.Google Scholar
  35. Juan, O. and Boykov, Y. 2006. Active Graph Cuts. In IEEE Conference of Computer Vision and Pattern Recognition, 2006 (to appear). Also in Tech.Rep. #655, UWO, CS Dept., Nov. 18, 2005 (ISBN-13: 978-0-7714-2558-9).Google Scholar
  36. Kass, M., Witkin, A., and Terzolpoulos, D. 1988. Snakes: Active contour models. International Journal of Computer Vision, 1(4):321–331.CrossRefGoogle Scholar
  37. Kimmel, R. and Bruckstein, A.M. 2003. Regularized Laplacian zero crossings as optimal edge integrators. International Journal of Computer Vision, 53(3):225–243.CrossRefGoogle Scholar
  38. Kirsanov, D. and Gortler, S.J. 2004. A discrete global minimization algorithm for continuous variational problems. Harvard Computer Science Technical Report, TR-14-04, July 2004, (also submitted to a journal).Google Scholar
  39. Kleinberg, J. 2002. An impossibility theorem for clustering. In The 16th conference on Neural Information Processing Systems (NIPS).Google Scholar
  40. Kohli, P. and Torr, P.H.S. 2005. Efficiently solving dynamic markov random fields using graph cuts. In International Conference on Computer Vision.Google Scholar
  41. Kohli, P. and Torr, P.H.S. 2006. Measuring uncertainty in graph cut solutions—efficiently computing min-marginal energies using dynamic graph cuts. In European Conference on Computer Vision, Graz, Austria, May 2006 (to appear).Google Scholar
  42. Kolmogorov, V. and Boykov, Y. 2005. What metrics can be approximated by geo-cuts, or global optimization of length/area and flux. In International Conference on Computer Vision, Beijing, China, vol. I, pp. 564–571.Google Scholar
  43. Kolmogorov, V., Criminisi, A., Blake, A., Cross, G., and Rother, C. 2005. Bi-layer segmentation of binocular stereo video. In IEEE Conference of Computer Vision and Pattern Recognition, San Diego, CA.Google Scholar
  44. Kolmogorov, V. and Zabih, R. 2002. Multi-camera scene reconstruction via graph cuts. In 7th European Conference on Computer Vision, volume III of LNCS 2352, pp. 82–96, Copenhagen, Denmark, May 2002. Springer-Verlag.Google Scholar
  45. Kolmogorov, V. and Zabih, R. 2004. What energy functions can be minimized via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):147–159.CrossRefGoogle Scholar
  46. Kumar, M.P., Torr, P.H.S., and Objcut, A.Z. 2005. In IEEE Conference of Computer Vision and Pattern Recognition, pp. 18–25.Google Scholar
  47. Kwatra, V., Schodl, A., Essa, I., and Bobick, A. 2003. GraphCut textures: image and video synthesis using graph cuts. In ACM Transactions on Graphics (SIGGRAPH), vol. 22, July 2003.Google Scholar
  48. Lempitsky, V., Boykov, Y., and Ivanov, D. 2006. Oriented visibility for multiview reconstruction. In European Conference on Computer Vision, Graz, Austria, May 2006 (to appear).Google Scholar
  49. Li, K., Wu, X., Chen, D.Z., and Sonka, M. 2006. Optimal surface segmentation in volumetric images-a graph-theoretic approach. IEEE transactions on Pattern Analysis and Pattern Recognition (PAMI), 28(1):119–134.Google Scholar
  50. Li, Y., Sun, J., and Shum, H.-Y. 2005. Video object cut and paste. In SIGGRAPH (ACM Transaction on Graphics) Google Scholar
  51. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. In SIGGRAPH (ACM Transaction on Graphics).Google Scholar
  52. Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In International Conference on Computer Vision, October 2005.Google Scholar
  53. Mortensen, E.N. and Barrett, W.A. 1998. Interactive segmentation with intelligent scissors. Graphical Models and Image Processing, 60:349–384.MATHCrossRefGoogle Scholar
  54. Mumford, D. and Shah, J. 1989. Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math., 42:577–685.MATHMathSciNetGoogle Scholar
  55. Murota, K. 2003. Discrete Convex Analysis. SIAM Monographs on Discrete Mathematics and Applications.Google Scholar
  56. Osher, S. and Paragios, N. 2003. Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer Verlag.Google Scholar
  57. Osher, S.J. and Fedkiw, R.P. 2002. Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag.Google Scholar
  58. Reese, L.J. 1999. Intelligent paint: Region-based interactive image segmentation. Master’s thesis, Brigham Young University.Google Scholar
  59. Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut—interactive foreground extraction using iterated graph cuts. In ACM Transactions on Graphics (SIGGRAPH).Google Scholar
  60. Rother, C., Kumar, S., Kolmogorov, V., and Blake, A. 2005. Digital tapestry. In IEEE Conference of Computer Vision and Pattern Recognition, San Diego, CA.Google Scholar
  61. Roy, S. and Cox, I. 1998. A maximum-flow formulation of the n-camera stereo correspondence problem. In IEEE Proc. of Int. Conference on Computer Vision, pp. 492–499.Google Scholar
  62. Sapiro, G. 2001. Geometric Partial Differential Equations and Image Analysis. Cambridge University Press.Google Scholar
  63. Scharstein, D. and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1/3):7–42.MATHCrossRefGoogle Scholar
  64. Sethian, J.A. 1999. Level Set Methods and Fast Marching Methods. Cambridge University Press.Google Scholar
  65. Shi, J. and Malik, J. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905.CrossRefGoogle Scholar
  66. Szeliski, R. and Zabih, R. 1999. An experimental comparison of stereo algorithms. In Vision Algorithms: Theory and Practice, number 1883 in LNCS, pp. 1–19, Springer-Verlag, Corfu, Greece, September 1999.Google Scholar
  67. Vasilevskiy, A. and Siddiqi, K. 2002. Flux maximizing geometric flows. PAMI, 24(12):1565–1578.Google Scholar
  68. Veksler, O. 2000. Image segmentation by nested cuts. In IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 339–344.Google Scholar
  69. Vogiatzis, G., Torr, P.H.S., and Cipolla, R. 2005. Multi-view stereo via volumetric graph-cuts. In IEEE Conference of Computer Vision and Pattern Recognition, pp. 391–398.Google Scholar
  70. Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., and Cohen, M.F. 2005. Interactive video cutout. In SIGGRAPH (ACM Transaction on Graphics).Google Scholar
  71. Wu, Z. and Leahy, R. 1993. An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11):1101–1113.CrossRefGoogle Scholar
  72. Xu, N., Bansal, R., and Ahuja, N. 2003. Object segmentation using graph cuts based active contours. In IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 46–53.Google Scholar
  73. Yezzi, A., Jr., Kichenassamy, S., Kumar, A., Olver, P., and Tannenbaum, A. 1997. A geometric snake model for segmentation of medical imagery. IEEE Transactions on Medical Imaging, 16(2):199–209.CrossRefGoogle Scholar
  74. Zhu, S.C. and Yuille, A. 1996. Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9):884–900.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Computer ScienceUniversity of Western OntarioLondonCanada
  2. 2.Imaging & VisualizationSiemens Corp. ResearchPrincetonUSA

Personalised recommendations