Estimating Intrinsic Images from Image Sequences with Biased Illumination

  • Yasuyuki Matsushita
  • Stephen Lin
  • Sing Bing Kang
  • Heung-Yeung Shum
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3022)


We present a method for estimating intrinsic images from a fixed-viewpoint image sequence captured under changing illumination directions. Previous work on this problem reduces the influence of shadows on reflectance images, but does not address shading effects which can significantly degrade reflectance image estimation under the typically biased sampling of illumination directions. In this paper, we describe how biased illumination sampling leads to biased estimates of reflectance image derivatives. To avoid the effects of illumination bias, we propose a solution that explicitly models spatial and temporal constraints over the image sequence. With this constraint network, our technique minimizes a regularization function that takes advantage of the biased image derivatives to yield reflectance images less influenced by shading.


Adjacent Pixel Cast Shadow Illumination Direction Smoothness Constraint Photometric Stereo 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. In: Hanson, A., Riseman, E. (eds.) Computer Vision Systems, pp. 3–26. Academic Press, New York (1978)Google Scholar
  2. 2.
    Adelson, E.H., Pentland, A.P.: The Perception of Shading and Reflectance. In: Knill, D., Richards, W. (eds.) Perception as Bayesian Inference, pp. 409–423 (1996)Google Scholar
  3. 3.
    Blake, A.: Boundary Conditions of lightness computation in mondrian world. Computer Vision, Graphics and Image Processing 32, 314–327 (1985)CrossRefGoogle Scholar
  4. 4.
    Wolff, L.B., Angelopoulou, E.: 3-d stereo using photometric ratios. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, pp. 247–258. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  5. 5.
    Kimmel, R., Elad, M., Shaked, D., Keshet, R., Sobel, I.: A Variational Framework for Retinex. International Journal of Computer Vision 52(1), 7–23 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Land, E.H.: An alternative technique for the computation of the designor in the Retinex theory of color vision. Proc. Nat. Acad. Sci. 83, 3078–3080 (1986)CrossRefGoogle Scholar
  7. 7.
    Land, E.H.: The Retinex theory of color vision. Scientific American 237(G), 108–128 (1977)Google Scholar
  8. 8.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. Journal of the Optical Society of America 61(1), 1–11 (1971)CrossRefGoogle Scholar
  9. 9.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: Proc. of 9th IEEE Int’l Conf. on Computer Vision, July 2001, pp. 68–75 (2001)Google Scholar
  10. 10.
    Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering Intrinsic Images from a Single Image. In: Advances in Neural Information Processing Systems 15 (NIPS), MIT Press, Cambridge (2002)Google Scholar
  11. 11.
    Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing Shadows from Images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 823–836. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  12. 12.
    Matsushita, Y., Nishino, K., Ikeuchi, K., Sakauchi, M.: Illumination Normalization with Time-dependent Intrinsic Images for Video Surveillance. In: Conf. on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 3–10 (2003)Google Scholar
  13. 13.
    Blake, A.: Comparison of the efficiency of deterministic and stochastic algorithms for visual reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(1), 2–12 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Horn, B.K.P., Schunk, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  15. 15.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741 (1984)zbMATHCrossRefGoogle Scholar
  16. 16.
    Olshausen, B.A., Field, D.J.: Emergence of simplecell receptive field properties by learning a sparse code for natural images. Nature 381, 607–608 (1996)CrossRefGoogle Scholar
  17. 17.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Generative models for recognition under variable pose and illumination. In: IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 277–284 (2000)Google Scholar
  18. 18.
    Shewchuck, J.R.: An introduction to the conjugate gradient method without agonizing pain. Tech. Rep. CMU-CS-94-125, Carnegie Mellon University (1994)Google Scholar
  19. 19.
    Hayakawa, H.: Photometric stereo under a light-source with arbitrary motion. Journal of Optical Society of America A 11(11), 3079–3089 (1994)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Basri, R., Jacobs, D.: Photometric stereo with general, unknown lighting. In: Proc. of Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 374–381 (2001)Google Scholar
  21. 21.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: Illumination-Based Image Synthesis: Creating Novel Images of Human Faces Under Differing Pose and Lighting. In: Proc. Workshop on Multi-View Modeling and Analysis of Visual Scenes, pp. 47–54 (1999)Google Scholar
  22. 22.
    Yuille, A.L., Snow, D., Epstein, R., Belhumeur, P.: Determining Generative Models for Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability. International Journal on Computer Vision 35(3), 203–222 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Yasuyuki Matsushita
    • 1
  • Stephen Lin
    • 1
  • Sing Bing Kang
    • 2
  • Heung-Yeung Shum
    • 1
  1. 1.Microsoft Research Asia, 3F, Beijing Sigma CenterBeijingChina
  2. 2.Microsoft ResearchRedmondU.S.A.

Personalised recommendations