Advertisement

Alpha-Flow for Video Matting

  • Mikhail Sindeev
  • Anton Konushin
  • Carsten Rother
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)

Abstract

This work addresses the problem of video matting, that is extracting the opacity-layer of a foreground object from a video sequence. We introduce the notion of alpha-flow which corresponds to the flow in the opacity layer. The idea is derived from the process of rotoscoping, where a user-supplied object mask is smoothly interpolated between keyframes while preserving its correspondence with the underlying image. Our key contribution is an algorithm which infers both the opacity masks and the alpha-flow in an efficient and unified manner. We embed our algorithm in an interactive video matting system where the first and last frame of a sequence are given as keyframes, and additional user strokes may be provided in intermediate frames. We show high quality results on various challenging sequences, and give a detailed comparison to competing techniques.

Keywords

Tracking Error Motion Vector Video Object Video Segmentation Temporal Connection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rav-Acha, A., Kohli, P., Rother, C., Fitzgibbon, A.: Unwrap mosaics: A new representation for video editing. In: SIGGRAPH, pp. 17:1–17:11 (2008)Google Scholar
  2. 2.
    Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: robust video object cutout using localized classifiers. In: SIGGRAPH, pp. 1–11 (2009)Google Scholar
  3. 3.
    Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. Computer Vision and Pattern Recognition, 1–8 (2007)Google Scholar
  4. 4.
    Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004)CrossRefGoogle Scholar
  5. 5.
    He, K., Sun, J., Tang, X.: Guided Image Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., Cohen, M.F.: Interactive video cutout. In: SIGGRAPH, pp. 585–594 (2005)Google Scholar
  7. 7.
    Li, Y., Sun, J., Shum, H.Y.: Video object cut and paste. In: SIGGRAPH, pp. 595–600 (2005)Google Scholar
  8. 8.
    Bai, X., Sapiro, G.: Geodesic matting: A framework for fast interactive image and video segmentation and matting. IJVC 82, 113–132 (2009)Google Scholar
  9. 9.
    Ding, Z., Chen, H., Guan, Y., Chen, W., Peng, Q.: GPU accelerated interactive space-time video matting. In: Computer Graphics International (2010)Google Scholar
  10. 10.
    Tang, Z., Miao, Z., Wan, Y., Zhang, D.: Video matting via opacity propagation. The Visual Computer, 1–15 (2011)Google Scholar
  11. 11.
    Lee, S.Y., Yoon, J.C., Lee, I.K.: Temporally coherent video matting. Graphical Models 72, 25–33 (2010)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Chuang, Y.Y., Agarwala, A., Curless, B., Salesin, D.H., Szeliski, R.: Video matting of complex scenes. SIGGRAPH 21, 243–248 (2002)Google Scholar
  13. 13.
    Apostoloff, N.E., Fitzgibbon, A.W.: Automatic video segmentation using spatiotemporal T-junctions. In: BMVC (2006)Google Scholar
  14. 14.
    Lezama, J., Alahari, K., Sivic, J., Laptev, I.: Track to the future: Spatio-temporal video segmentation with long-range motion cues. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  15. 15.
    Sand, P., Teller, S.J.: Particle video: Long-range motion estimation using point trajectories. International Journal of Computer Vision 80, 72–91 (2008)CrossRefGoogle Scholar
  16. 16.
    Chen, J., Paris, S., Wang, J., Cohen, M., Cohen, M., Durand, F.: The video mesh: A data structure for image-based video editing. Artificial Intelligence (2009)Google Scholar
  17. 17.
    Agarwala, A., Hertzmann, A., Salesin, D.H., Seitz, S.M.: Keyframe-based tracking for rotoscoping and animation. In: SIGGRAPH, pp. 584–591 (2004)Google Scholar
  18. 18.
    Tsai, D., Flagg, M., Rehg, J.M.: Motion coherent tracking with multi-label MRF optimization. In: BMVC (2010)Google Scholar
  19. 19.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)Google Scholar
  20. 20.
    Levin, A., Lischinski, D., Weiss, Y.: A closed form solution to natural image matting. In: IEEE CVPR, vol. 1, pp. 61–68 (2006)Google Scholar
  21. 21.
    Rhemann, C., Rother, C., Kohli, P., Gelautz, M.: A Spatially Varying PSF-based Prior for Alpha Matting. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  22. 22.
    Liu, C.: Beyond pixels: Exploring new representations and applications for motion analysis. Doctoral Thesis. Massachusetts Institute of Technology (2009)Google Scholar
  23. 23.
    Steinbrucker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: ICCV, pp. 1609–1614 (2009)Google Scholar
  24. 24.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: A randomized correspondence algorithm for structural image editing. In: SIGGRAPH (2009)Google Scholar
  25. 25.
    Agrawal, A., Raskar, R., Chellappa, R.: What Is the Range of Surface Reconstructions from a Gradient Field? In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 578–591. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  26. 26.
    Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M.: Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 211–224. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  27. 27.
    Strecha, C., Fransens, R., Van Gool, L.: A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection. In: Comaniciu, D., Mester, R., Kanatani, K., Suter, D. (eds.) SMVP 2004. LNCS, vol. 3247, pp. 71–82. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  28. 28.
    Sindeev, M., Konushin, A., Rother, C.: Alpha-flow for video matting. Technical Report (2012)Google Scholar
  29. 29.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph based video segmentation. In: IEEE CVPR (2010)Google Scholar
  30. 30.
    Bai, X., Wang, J., Simons, D.: Towards Temporally-Coherent Video Matting. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2011. LNCS, vol. 6930, pp. 63–74. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mikhail Sindeev
    • 1
    • 2
  • Anton Konushin
    • 2
  • Carsten Rother
    • 3
  1. 1.Keldysh Institute of Applied MathematicsMoscowRussia
  2. 2.Lomonosov Moscow State UniversityMoscowRussia
  3. 3.Microsoft Research CambridgeUnited Kingdom

Personalised recommendations