User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior

  • Anat Levin
  • Yair Weiss
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3021)


When we take a picture through transparent glass the image we obtain is often a linear superposition of two images: the image of the scene beyond the glass plus the image of the scene reflected by the glass. Decomposing the single input image into two images is a massively ill-posed problem: in the absence of additional knowledge about the scene being viewed there are an infinite number of valid decompositions. In this paper we focus on an easier problem: user assisted separation in which the user interactively labels a small number of gradients as belonging to one of the layers.

Even given labels on part of the gradients, the problem is still ill-posed and additional prior knowledge is needed. Following recent results on the statistics of natural images we use a sparsity prior over derivative filters. We first approximate this sparse prior with a Laplacian prior and obtain a simple, convex optimization problem. We then use the solution with the Laplacian prior as an initialization for a simple, iterative optimization for the sparsity prior. Our results show that using a prior derived from the statistics of natural images gives a far superior performance compared to a Gaussian prior and it enables good separations from a small number of labeled gradients.


Input Image Gaussian Mixture Model Single Image Natural Image Natural Scene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. B 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Farid, H., Adelson, E.H.: Separating reflections from images by use of independent components analysis. Journal of the optical society of america 16(9), 2136–2145 (1999)CrossRefGoogle Scholar
  3. 3.
    Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 823–836. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Irani, M., Peleg, S.: Image sequence enhancement using multiple motions analysis. In: Conf. on Computer Vision and Pattern Recognition, Champaign, Illinois, pp. 216–221 (June 1992)Google Scholar
  5. 5.
    Levin, A., Zomet, A., Weiss, Y.: Learning to perceive transparency from the statistics of natural scenes. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15 (2002)Google Scholar
  6. 6.
    Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. PAMI 11, 674–693 (1989)zbMATHGoogle Scholar
  7. 7.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–608 (1996)CrossRefGoogle Scholar
  8. 8.
    Shechner, Y., Shamir, J., Kiryati, N.: Polarization-based decorrelation of transparent layers: The inclination angle of an invisible surface. In: Int. Conf. on Computer Vision, pp. 814–819 (1999)Google Scholar
  9. 9.
    Simoncelli, E.P.: Statistical models for images:compression restoration and synthesis. In: Proc Asilomar Conference on Signals, Systems and Computers, pp. 673–678 (1997)Google Scholar
  10. 10.
    Simoncelli, E.P.: Bayesian denoising of visual images in the wavelet domain. In: Müller, P., Vidakovic, B. (eds.) Wavelet based models (1999)Google Scholar
  11. 11.
    Szeliksi, R., Avidan, S., Anandan, P.: Layer extraction from multiple images containing reflections and transparency. In: Conf. on Computer Vision and Pattern Recognition (2000)Google Scholar
  12. 12.
    Tappen, M., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15 (2002)Google Scholar
  13. 13.
    Tsin, Y., Kang, S.B., Szeliski, R.: Stereo matching with reflections and translucency. In: Conf. on Computer Vision and Pattern Recognition, pp. 702–709 (2003)Google Scholar
  14. 14.
    Vanderbei, R.: Loqo (2000),
  15. 15.
    Wainwright, M.J., Simoncelli, E.P., Willsky, A.S.: Random cascades of gaussian scale mixtures for natural images. In: Int. Conf. on Image Processing, pp. I:260– 263 (2000)Google Scholar
  16. 16.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: Proc. Intl. Conf. Computer Vision, pp. 68–75 (2001)Google Scholar
  17. 17.
    Zibulevsky, M., Kisilev, P., Zeevi, Y., Pearlmutter, B.: Blind source separation via multinode sparse representation. In: Dietterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Anat Levin
    • 1
  • Yair Weiss
    • 1
  1. 1.School of Computer Science and EngineeringThe Hebrew University of JerusalemJerusalemIsrael

Personalised recommendations