Advertisement

Separating Transparent Layers through Layer Information Exchange

  • Bernard Sarel
  • Michal Irani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3024)

Abstract

In this paper we present an approach for separating two transparent layers in images and video sequences. Given two initial unknown physical mixtures, I 1 and I 2, of real scene layers, L 1 and L 2, we seek a layer separation which minimizes the structural correlations across the two layers, at every image point. Such a separation is achieved by transferring local grayscale structure from one image to the other wherever it is highly correlated with the underlying local grayscale structure in the other image, and vice versa. This bi-directional transfer operation, which we call the “layer information exchange”, is performed on diminishing window sizes, from global image windows (i.e., the entire image), down to local image windows, thus detecting similar grayscale structures at varying scales across pixels. We show the applicability of this approach to various real-world scenarios, including image and video transparency separation. In particular, we show that this approach can be used for separating transparent layers in images obtained under different polarizations, as well as for separating complex non-rigid transparent motions in video sequences. These can be done without prior knowledge of the layer mixing model (simple additive, alpha-mated composition with an unknown alpha-map, or other), and under unknown complex temporal changes (e.g., unknown varying lighting conditions).

Keywords

Mutual Information Video Sequence Transfer Factor Entire Image Pixel Position 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical Model-Based Motion Estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992)Google Scholar
  2. 2.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)Google Scholar
  3. 3.
    Bronstein, A., Bronstein, M., Zibulevsky, M., Zeevi, Y.Y.: Separation of semireflective layers using Sparse ICA. In: Proc. ICASSP 2003, vol. 3, pp. 733–736 (2003)Google Scholar
  4. 4.
    Cover, T., Thomas, J.: Elements of Information Theory. Wiley and Sons, Chichester (1991)zbMATHCrossRefGoogle Scholar
  5. 5.
    Farid, H., Adelson, E.H.: Separating Reflections from Images by Use of Independent Components Analysis. JOSA 16(9), 2136–2145 (1999)CrossRefGoogle Scholar
  6. 6.
    Irani, M., Anandan, P.: Robust Multi-Sensor Image Alignment. In: ICCV 1998, pp. 959–966 (1998)Google Scholar
  7. 7.
    Levin, A.: Zomet, and Y. Weiss: Learning to perceive transparency from the statistics of natural scenes. In: NIPS 2002, pp.1247–1254 (2002)Google Scholar
  8. 8.
    Schechner, Y.Y., Shamir, J., Kiryati, N.: Polarization and statistical analysis of scenes containing a semi-reflector. JOSA A 17, 276–284 (2000)CrossRefGoogle Scholar
  9. 9.
    Szeliski, R., Avidan, S., Anandan, P.: Layer Extraction from Multiple Images containing Reflections and Transparency. In: CVPR 2000, pp. 246–253 (2000)Google Scholar
  10. 10.
    Tsin, Y., Kang, S.B., Szeliski, R.: Stereo Matching with Reflections and Translucency. In: CVPR 2003, 702–709 (2003)Google Scholar
  11. 11.
    Weiss, Y.: Deriving Intrinsic Images from Image Sequences. In: ICCV 2001, pp.68–75 (2001)Google Scholar
  12. 12.
    Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Bayesian Estimation of Layers from Multiple Images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 487–501. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    FastICA package, downloaded from: http://www.cis.hut.fi/projects/ica/fastica/

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Bernard Sarel
    • 1
  • Michal Irani
    • 1
  1. 1.Dept. of Computer Science and Applied MathematicsWeizmann Institute of ScienceRehovotISRAEL

Personalised recommendations