Skip to main content
SpringerLink
Log in

Video chroma keying via global sampling and trimap propagation

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Chroma keying is a widely used video editing technique, which finely separates the foreground objects from the background. Two major concerns are involved in chroma keying problems: alpha estimation and foreground color restoration. The alpha values reveal the opacity property of the foreground objects. The foreground color restoration removes the background color influence to the foreground appearance especially at transparent regions and objects’ boundaries. In this paper, the color range of the solid background is well analyzed to automatically separate foreground from background. Global sampling is utilized to robustly and reliably estimate the foreground color at boundaries and transparent regions. Furthermore, we propose to propagate the geometric shape of foreground boundaries between adjacent frames by using optical flow and thin plate splines interpolation. The trimap, which is an initial foreground/background/unknown segmentation of each frame can be automatically updated for each video frame by using our proposed propagation method. Compared to previous methods, our proposed matting method estimates high-quality alpha matte and reliable foreground color with least user interference.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Wang, J., Cohen, M.F.: Optimized color sampling for robust matting. In: Proceedings of IEEE CVPR, pp. 1–8 (2007)

  2. Rhemann, C., Rother, C., Gelautz, M.: Improving color modeling for alpha matting. In: Proceedings of the British machine vision conference (BMVC), pp. 115.1–115.10 (2008)

  3. Gastal, E.S.L., Oliveira, M.M.: Shared sampling for real-time alpha matting. In: Proceedings of eurographics, pp. 575–584 (2010)

  4. He, K., Rhemann, C., Rother, C., Tang, X., Sun, J.: A global sampling method for alpha matting. In: Proceedings of IEEE CVPR, pp. 2049–2056 (2011)

  5. Apostoloff, N., Fitzgibbon, A.: Bayesian video matting using learnt image priors. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 407–414 (2004)

  6. Choi, I., Lee, M., Tai, Y.-W.: Video matting using multi-frame nonlocal matting laplacian. In: Proceedings of European conference on computer vision (ECCV), pp. 540–553 (2012)

  7. Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D.H., Szeliski, R.: Video matting of complex scenes. ACM Trans. Gr. 21(3), 243–248 (2002)

    Article  Google Scholar 

  8. Hu, W.-C., Hsu, J.-F.: Automatic spectral video matting. ACM Pattern Recognit. 46(4), 1183–1194 (2013)

    Article  Google Scholar 

  9. Lee, S.-Y., Yoon, J.-C., Lee, I.-K.: Temporally coherent video matting. ACM Gr. Models 72(3), 25–33 (2010)

    Article  Google Scholar 

  10. Li, D., Chen, Q., Tang, C.-K.: Motion-aware KNN laplacian for video matting. In: Proceedings of IEEE international conference on computer vision (ICCV), pp. 540–553 (2012)

  11. Tang, Z., Miao, Z., Wan, Y., Zhang, D.: Video matting via opacity propagation. Vis. Comput. 28(1), 47–61 (2012)

    Article  Google Scholar 

  12. Cho, J.-H., Yamasaki, T., Aizawa, K., Lee, K.H.: Depth video camera based temporal alpha matting for natural 3D scene generation. In: 3DTV conference: the true vision—capture, transmission and display of 3D video, pp. 1–4 (2011)

  13. Porter, T., Duff, T.: Compositing digital images. In: Proceedings of ACM SIGGRAPH, pp. 253–259 (1984)

  14. Lloyd, S.: Least squares quantization in pcm. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 264–271 (2001)

  16. Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on artificial intelligence, pp. 674–679 (1982)

  17. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proceedings of European conference on computer vision (ECCV), pp. 25–36 (2004)

  18. Bai, X., Wang, J., Simons, D.: Towards temporally-coherent video matting. In: Proceedings of the 5th international conference on computer vision/computer graphics collaboration techniques, pp. 63–74 (2011)

  19. Duchon, J.: Splines minimizing rotation-invariant semi-norms in Sobolev spaces. Constr. Theory Funct. Several Var. 571, 85–100 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  20. Wahba, G.: Spline Models for Observational Data. Society for Industrial and Applied Mathematics, Philadelphia (1990)

    Book  MATH  Google Scholar 

  21. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: Alpha matting evaluation website. http://www.alphamatting.com (last visited 4/20/2014)

  22. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: Proceedings of IEEE CVPR, pp. 1826–1833 (2009)

  23. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, University of Ottawa, 800 King Edward Ave., Ottawa, ON, K1N 6N5, Canada

    Chengcheng Hao, Wenyi Wang & Jiying Zhao

Authors
  1. Chengcheng Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenyi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiying Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenyi Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, C., Wang, W. & Zhao, J. Video chroma keying via global sampling and trimap propagation. Multimedia Systems 22, 693–707 (2016). https://doi.org/10.1007/s00530-015-0493-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-015-0493-2

Keywords

Search

Navigation