The Visual Computer

, Volume 26, Issue 9, pp 1167–1176

Color invariant chroma keying and color spill neutralization for dynamic scenes and cameras

  • Anselm Grundhöfer
  • Daniel Kurz
  • Sebastian Thiele
  • Oliver Bimber
Original Article

Abstract

In this article we show how temporal backdrops that alternately change their color rapidly at recording rate can aid chroma keying by transforming color spill into a neutral background illumination. Since the chosen colors sum up to white, the chromatic (color) spill component is neutralized when integrating over both backdrop states. The ability to separate both states additionally allows to compute high-quality alpha mattes. Besides the neutralization of color spill, our method is invariant to foreground colors and supports applications with real-time demands. In this article, we explain different realizations of temporal backdrops and describe how keying and color spill neutralization are carried out, how artifacts resulting from rapid motion can be reduced, and how our approach can be implemented to be compatible with common real-time post-production pipelines.

Keywords

Chroma keying Color spill Image processing Digitization and image capture 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ben Ezra, M.: Segmentation with invisible keying signal. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 32–37. IEEE Computer Society, Los Alamitos (2000) Google Scholar
  2. 2.
    Chaplin, D.J.: Chroma keyer with fringe control offset. uS Patent number 5,313,304 (1994) Google Scholar
  3. 3.
    Choudhury, B., Singla, D., Chandran, S.: Fast color-space decomposition based environment matting. In: I3D ’08: Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games, p. 1. ACM, New York (2008) Google Scholar
  4. 4.
    Chuang, Y.Y., Zongker, DE, Hindorff, J., Curless, B., Salesin, D.H., Szeliski, R.: Environment matting extensions: towards higher accuracy and real-time capture. In: SIGGRAPH ’00: ACM SIGGRAPH 2000 Papers, pp. 121–130. ACM, New York (2000) Google Scholar
  5. 5.
    Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 2, pp. 264–271. IEEE Computer Society, Los Alamitos (2001) Google Scholar
  6. 6.
    Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., Hawkins, T.: A lighting reproduction approach to live-action compositing. ACM Trans. Graph. 21(3), 547–556 (2002) CrossRefGoogle Scholar
  7. 7.
    Dupont, J., Deschenes, F.: Toward a realistic interpretation of blue-spill for blue-screen matting. In: CRV ’06: Proceedings of the 3rd Canadian Conference on Computer and Robot Vision, p. 33. IEEE Computer Society, Washington (2006) CrossRefGoogle Scholar
  8. 8.
    Graham, T., Richard, R.: Flash-based keying. European Patent Application EP1499117 (2005) Google Scholar
  9. 9.
    Grau, O., Pullen, T., Thomas, G.: A combined studio production system for 3-d capturing of live action and immersive actor feedback. IEEE Trans. Circuits Syst. Video Technol. 14(3), 370–380 (2004) CrossRefGoogle Scholar
  10. 10.
    Grundhöfer, A., Bimber, O.: Virtualstudio2go: digital video composition for real environments. In: SIGGRAPH Asia ’08: ACM SIGGRAPH Asia 2008 Papers, pp. 1–8. ACM, New York (2008) Google Scholar
  11. 11.
    Grundhöfer, A., Seeger, M., Häntsch, F., Bimber, O.: Dynamic adaptation of projected imperceptible codes. In: ISMAR ’07: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–10. IEEE Computer Society, Washington (2007) CrossRefGoogle Scholar
  12. 12.
    Gvili, R., Kaplan, A., Ofek, E., Yahav, G.: Depth keying. In: Woods, A.J., Bolas, M.T., Merritt, J.O., Benton, S.A. (eds.) Stereoscopic Displays and Virtual Reality Systems X. SPIE, vol. 5006, pp. 564–574. SPIE, Bellingham (2003) Google Scholar
  13. 13.
    Kuechler, M., Kunz, A.: Imperceptible projection blanking for reliable segmentation within mixed reality applications. In: Kjems, E., Blach, R. (eds.) 9th International Workshop on Immersive Projection Technology, 11th Eurographics Workshop on Virtual Environments, pp. 23–30. Eurographics Association, Denmark (2005) Google Scholar
  14. 14.
    Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., McMillan, L.: Image-based 3d photography using opacity hulls. ACM Trans. Graph. 21(3), 427–437 (2002) CrossRefGoogle Scholar
  15. 15.
    McGuire, M., Matusik, W.: Real-time triangulation matting using passive polarization. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Sketches, p. 88. ACM, New York (2006) CrossRefGoogle Scholar
  16. 16.
    McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. In: SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers, pp. 567–576. ACM, New York (2005) CrossRefGoogle Scholar
  17. 17.
    McGuire, M., Matusik, W., Yerazunis, W.: Practical, Real-Time Studio Matting Using Dual Imagers, pp. 235–244 (2006) Google Scholar
  18. 18.
    Moon, J.H., Kim, D.O., Park, R.H.: Video matting based on background estimation. In: World Academy of Science, Engineering and Technology, vol. 2 (2005) Google Scholar
  19. 19.
    RA, Moses, Hart, W.M.: The temporal responsiveness of vision. Adler’s Physiology of the Eye (1987) Google Scholar
  20. 20.
    Peers, P., Dutré, P.: Wavelet environment matting. In: EGRW ’03: Proceedings of the 14th Eurographics Workshop on Rendering, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, pp. 157–166 (2003) Google Scholar
  21. 21.
    Porter, T., Duff, T.: Compositing digital images. In: SIGGRAPH ’84: ACM SIGGRAPH 1984 Papers, pp. 253–259. ACM, New York (1984) Google Scholar
  22. 22.
    Sharma, G.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002) Google Scholar
  23. 23.
    Smith, A.R., Blinn, J.F.: Blue screen matting. In: SIGGRAPH ’96: ACM SIGGRAPH 1996 Papers, pp. 259–268. ACM, New York (1996) Google Scholar
  24. 24.
    Sun, J., Jia, J., Keung Tang, C., Yeung Shum, H.: Poisson matting. In: SIGGRAPH ’04: ACM SIGGRAPH 2004 Papers, vol. 23, pp. 315–321. ACM, New York (2004) CrossRefGoogle Scholar
  25. 25.
    Sun, J., Sun, J., Kang, S.B., Xu, Z.B., Tang, X., Shum, H.Y.: Flash cut: foreground extraction with flash and no-flash image pairs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2007) Google Scholar
  26. 26.
    Vlahos, P.: Electronic composite photography with color control. uS Patent number 4,007,487 (1977) Google Scholar
  27. 27.
    Vlahos, P.: Backing color and luminance nonuniformity compensation for linear image compositing. uS Patent number 5,032,901 (1991) Google Scholar
  28. 28.
    Wang, O., Finger, J., Yang, Q., Davis, J., Yang, R.: Automatic natural video matting with depth. In: PG ’07: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 469–472. IEEE Computer Society, Washington (2007) Google Scholar
  29. 29.
    Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., Debevec, P.: Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Trans. Graph. 24(3), 756–764 (2005) CrossRefGoogle Scholar
  30. 30.
    Whitesides, T., Walls, M., Paolini, R., Sohn, S., Gates, H., McCreary, M., Jacobson, J.: 10.2: Towards video-rate microencapsulated dual-particle electrophoretic displays. SID Symp. Dig. Tech. Pap. 35(1), 133–135 (2004) CrossRefGoogle Scholar
  31. 31.
    Zhu, J., Yang, Y.H.: Frequency-based environment matting. In: PG ’04: Proceedings of the Computer Graphics and Applications, 12th Pacific Conference, pp. 402–410. IEEE Computer Society, Washington (2004) Google Scholar
  32. 32.
    Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.H.: Environment matting and compositing. In: SIGGRAPH ’99: ACM SIGGRAPH 1999 Papers, pp. 205–214. ACM, New York (1999) Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Anselm Grundhöfer
    • 1
  • Daniel Kurz
    • 1
  • Sebastian Thiele
    • 1
  • Oliver Bimber
    • 2
  1. 1.Bauhaus University WeimarWeimarGermany
  2. 2.JKU Institute of Computer GraphicsLinzAustria

Personalised recommendations