Abstract
We present an algorithm for extracting high quality temporally coherent alpha mattes of objects from a video. Our approach extends the conventional image matting approach, i.e. closed-form matting, to video by using multi-frame nonlocal matting Laplacian. Our multi-frame nonlocal matting Laplacian is defined over a nonlocal neighborhood in spatial temporal domain, and it solves the alpha mattes of several video frames all together simultaneously. To speed up computation and to reduce memory requirement for solving the multi-frame nonlocal matting Laplacian, we use the approximate nearest neighbor(ANN) to find the nonlocal neighborhood and the k-d tree implementation to divide the nonlocal matting Laplacian into several smaller linear systems. Finally, we adopt the nonlocal mean regularization to enhance temporal coherence of the estimated alpha mattes and to correct alpha matte errors at low contrast regions. We demonstrate the effectiveness of our approach on various examples with qualitative comparisons to the results from previous matting algorithms.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Litwinowicz, P.: Processing images and video for an impressionist effect. In: ACM SIGGRAPH (1997)
Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A bayesian approach to digital matting. In: CVPR (2001)
Sun, J., Jia, J., Tang, C.K., Shum, H.Y.: Poisson matting. ACM Transactions on Graphics 23, 315–321 (2004)
Wang, J., Cohen, M.F.: An iterative optimization approach for unified image segmentation and matting. In: ICCV (2005)
Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. on PAMI 30 (2008) 0162–8828
He, K., Sun, J., Tang, X.: Fast matting using large kernel matting laplacian matrices. In: CVPR, pp. 2165–2172 (2010)
Wang, J., Cohen, M.F.: Optimized Color Sampling for Robust Matting. In: CVPR (2007)
Wang, J., Agrawala, M., Cohen, M.: Soft scissors: An interactive tool for realtime high quality matting. ACM Transactions on Graphics 26 (2006)
Wang, J., Cohen, M.F.: Image and video matting: a survey. Found. Trends. Comput. Graph. Vis. 3 (2007)
Zheng, Y., Kambhamettu, C.: Learning based digital matting. In: ICCV (2009)
Lee, P., Wu, Y.: Nonlocal matting. In: CVPR (2011)
He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: CVPR (2011)
Chen, Q., Li, D., Tang, C.K.: Knn matting. In: CVPR (2012)
Chuang, Y.Y., Agarwala, A., Curless, B., Salesin, D.H., Szeliski, R.: Video matting of complex scenes. ACM Transactions on Graphics 21 (2002)
Apostoloff, N., Fitzgibbon, A.: Bayesian video matting using learnt image priors. In: CVPR (2004)
Eisemann, M., Wolf, J., Magnor, M.: Spectral video matting. In: Proc. Vision, Modeling and Visualization, VMV (2009)
Lee, S.Y., Yoon, J.C., Lee, I.K.: Temporally coherent video matting. Graph. Models 72 (2010) 1524–0703
Buades, A., Coll, B.: A non-local algorithm for image denoising. In: CVPR (2005)
Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: CVPR (2009)
Rhemann, C., Rother, C., Gelautz, M.: Improving color modeling for alpha matting. In: BMVC (2009)
Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Eurographics (2009)
Joshi, N., Matusik, W., Avidan, S.: Natural video matting using camera arrays. ACM Transactions on Graphics 25 (2006)
McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. ACM Transactions on Graphics 24 (2005)
Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.: An optimal algorithm for approximate nearest neighbor searching. Journal of the ACM 45, 891–923 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Choi, I., Lee, M., Tai, YW. (2012). Video Matting Using Multi-frame Nonlocal Matting Laplacian. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33783-3_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-33783-3_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33782-6
Online ISBN: 978-3-642-33783-3
eBook Packages: Computer ScienceComputer Science (R0)