Advertisement

International Journal of Computer Vision

, Volume 82, Issue 2, pp 113–132 | Cite as

Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting

  • Xue Bai
  • Guillermo Sapiro
Article

Abstract

An interactive framework for soft segmentation and matting of natural images and videos is presented in this paper. The proposed technique is based on the optimal, linear time, computation of weighted geodesic distances to user-provided scribbles, from which the whole data is automatically segmented. The weights are based on spatial and/or temporal gradients, considering the statistics of the pixels scribbled by the user, without explicit optical flow or any advanced and often computationally expensive feature detectors. These could be naturally added to the proposed framework as well if desired, in the form of weights in the geodesic distances. An automatic localized refinement step follows this fast segmentation in order to further improve the results and accurately compute the corresponding matte function. Additional constraints into the distance definition permit to efficiently handle occlusions such as people or objects crossing each other in a video sequence. The presentation of the framework is complemented with numerous and diverse examples, including extraction of moving foreground from dynamic background in video, natural and 3D medical images, and comparisons with the recent literature.

Keywords

Interactive image and video segmentation Matting Geodesic computations Weighted distance functions Fast algorithms User-provided scribbles 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. ADOBE SYSTEMS INCORP. (2002). Adobe photoshop user guide. Google Scholar
  2. ADOBE SYSTEMS INCORP. (2007). Adobe photoshop CS3 new features. http://www.adobe.com/products/photoshop/photoshop.
  3. Agarwala, A., Hertzmann, A., Salesin, D., & Seitz, S. (2004). Keyframe-based tracking for rotoscoping and animation. In Proceedings of SIGGRAPH’04. Google Scholar
  4. Bai, X., & Sapiro, G. (2007). A geodesic framework for fast interactive image and video segmentation and matting. In Proc. international conference computer vision, Rio de Janeiro, Brazil, 16–19 October 2007. Google Scholar
  5. Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104. CrossRefGoogle Scholar
  6. Boykov, Y. Y., & Jolly, M.-P. (2001). Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In IEEE ICCV 2001 (Vol. 01, pp. 105). Google Scholar
  7. Caselles, V., Morel, J.-M., & Sbert, C. (1998). An axiomatic approach to image interpolation. IEEE Transactions on Image Processing, 7(3), 376–386. zbMATHCrossRefMathSciNetGoogle Scholar
  8. Chuang, Y.-Y., Curless, B., Salesin, D. H., & Szeliski, R. (2001). A Bayesian approach to digital matting. In Proceedings of IEEE CVPR 2001 (Vol. 2, pp. 264–271). December 2001. Google Scholar
  9. Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., & Szeliski, R. (2002). Video matting of complex scenes. In SIGGRAPH ’02 (pp. 243–248). Google Scholar
  10. COREL CORPORATION. (2002). Knockout user guide. Google Scholar
  11. Criminisi, A., Cross, G., Blake, A., & Kolmogorov, V. (2006). Bilayer segmentation of live video. In Proceedings of IEEE CVPR 2006 (pp. 53–60). Google Scholar
  12. Dial, R. (1969). Shortest path forest with topological ordering. Communications of the ACM, 12, 632–633. CrossRefGoogle Scholar
  13. Falcao, A. X., Stolfi, J., & de Alencar Lotufo, R. (2004). The image foresting transform: Theory, algorithms, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 19–29. CrossRefGoogle Scholar
  14. Grady, L. (2006). Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1768–1783. CrossRefGoogle Scholar
  15. Grady, L., Schiwietz, T., Aharon, S., & Westermann, R. (2005). Random walks for interactive alpha-matting. In Proceedings of VIIP 2005 (pp. 423–429). Spain, September 2005. ACTA Press. Google Scholar
  16. Levin, A., Lischinski, D., & Weiss, Y. (2004). Colorization using optimization. SIGGRAPH’04, 23(3), 689–694. Google Scholar
  17. Levin, A., Lischinski, D., & Weiss, Y. (2006). A closed form solution to natural image matting. In Proceedings of IEEE CVPR 2006 (pp. 61–68). Google Scholar
  18. Levin, A., Rav-Acha, A., & Lischinski, D. (2007). Spectral matting. In Proceedings of IEEE CVPR 2007, June 2007. Google Scholar
  19. Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2, 91–110. CrossRefGoogle Scholar
  20. Protiere, A., & Sapiro, G. (2007). Interactive image segmentation via adaptive weighted distances. IEEE Transactions on Image Processing, 16, 1046–1057. CrossRefMathSciNetGoogle Scholar
  21. Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. In SIGGRAPH’04. Google Scholar
  22. Sapiro, G. (2001). Geometric partial differential equations and image processing. Cambridge: Cambridge University Press. Google Scholar
  23. Sinop, A. K., & Grady, L. (2007). A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In Proc. international conference computer vision, Rio de Janeiro, Brazil, 16–19 October 2007. Google Scholar
  24. Sun, J., Jia, J., Tang, C.-K., & Shum, H.-Y. (2004). Poisson matting. In SIGGRAPH’04 (pp. 315–321). Google Scholar
  25. Sun, J., Kang, S. B., Xu, Z., Tang, X., & Shum, H.-Y. (2007). Flash cut: Foreground extraction with flash and no-flash image pairs. In Proceedings of IEEE CVPR 2007, June 2007. Google Scholar
  26. Thorup, M. (1997). Undirected single source shortest paths in linear time. In Proceedings IEEE symposium on foundations of computer science (pp. 12–21). Google Scholar
  27. Wang, J., & Cohen, M. F. (2005). An iterative optimization approach for unified image segmentation and matting. In Proceedings of IEEE ICCV 2005 (pp. 936–943). Google Scholar
  28. Wang, J., & Cohen, M. (2007a). Image and video matting: A survey. Foundations and Trends in Computer Graphics and Vision, 3(2), 1–78. zbMATHGoogle Scholar
  29. Wang, J., & Cohen, M. (2007b). Optimized color sampling for robust matting. In Proceedings of IEEE CVPR 2007, June 2007. Google Scholar
  30. Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., & Cohen, M. F. (2005). Interactive video cutout. SIGGRAPH’05, 24(3), 585–594. Google Scholar
  31. Wang, J., Agrawala, M., & Cohen, M. F. (2007). Soft scissors: An interactive tool for realtime high quality matting. In SIGGRAPH’07. Google Scholar
  32. Yang, C., Duraiswami, R., Gumerov, N., & Davis, L. (2003). Improved fast Gauss transform and efficient kernel density estimation. In Proceedings IEEE ICCV 2003 (pp. 464–471). Nice, France. Google Scholar
  33. Yatziv, L., & Sapiro, G. (2005). Image and video data blending using intrinsic distances. Patent pending. Google Scholar
  34. Yatziv, L., & Sapiro, G. (2006). Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing, 15(5), 1120–1129. CrossRefGoogle Scholar
  35. Yatziv, L., Bartesaghi, A., & Sapiro, G. (2006). O(n) implementation of the fast marching algorithm. Journal of Computational Physics, 212, 393–399. zbMATHCrossRefGoogle Scholar
  36. Virtual colonoscopy screening resource center. (2005) http://www.vcscreen.com.

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Electrical and Computer EngineeringUniversity of MinnesotaMinneapolisUSA

Personalised recommendations