Skip to main content

Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability

International Journal of Computer Vision volume 35, pages 203–222 (1999)Cite this article

Abstract

We describe a method of learning generative models of objects from a set of images of the object under different, and unknown, illumination. Such a model allows us to approximate the objects' appearance under a range of lighting conditions. This work is closely related to photometric stereo with unknown light sources and, in particular, to the use of Singular Value Decomposition (SVD) to estimate shape and albedo from multiple images up to a linear transformation (Hayakawa, 1994). Firstly we analyze and extend the SVD approach to this problem. We demonstrate that it applies to objects for which the dominant imaging effects are Lambertian reflectance with a distant light source and a background ambient term. To determine that this is a reasonable approximation we calculate the eigenvectors of the SVD on a set of real objects, under varying lighting conditions, and demonstrate that the first few eigenvectors account for most of the data in agreement with our predictions. We then analyze the linear ambiguities in the SVD approach and demonstrate that previous methods proposed to resolve them (Hayakawa, 1994) are only valid under certain conditions. We discuss alternative possibilities and, in particular, demonstrate that knowledge of the object class is sufficient to resolve this problem. Secondly, we describe the use of surface consistency for putting constraints on the possible solutions. We prove that this constraint reduces the ambiguities to a subspace called the generalized bas relief ambiguity (GBR) which is inherent in the Lambertian reflectance function (and which can be shown to exist even if attached and cast shadows are present (Belhumeur et al., 1997)). We demonstrate the use of surface consistency to solve for the shape and albedo up to a GBR and describe, and implement, a variety of additional assumptions to resolve the GBR. Thirdly, we demonstrate an iterative algorithm that can detect and remove some attached shadows from the objects thereby increasing the accuracy of the reconstructed shape and albedo.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Atick, J.J., Griffin, P.A., and Redlich, A.N. 1996. Statistical approach to shape from shading: Reconstruction of 3-dimensional face surfaces from single 2-dimensional images. Neural Computation, 8(6):1321–1340.

    Google Scholar 

  • Belhumeur, P.N. and Kriegman, D.J. 1996. What is the set of images of an object under all possible lighting conditions. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., pp. 270–277.

  • Belhumeur, P., Kriegman, D., and Yuille, A.L. 1997. The Bas-relief ambiguity. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition, CVPR97, Puerto Rico.

  • Biederman, I. 1987. Recognition-by-components: A theory of human image understanding. Psychological Review, 94:115–147.

    Google Scholar 

  • Epstein, R. 1996. Learning object representation from greyscale images. Ph.D. Thesis. Division of Applied Sciences. Harvard University.

  • Epstein, R., Hallinan, P.W., and Yuille, A.L. 1995. ± Eigenimages suffice: An empirical investigation of low-dimensional lighting models. In Proceedings of IEEEWorkshop on Physics-Based Modeling in Computer Vision, pp. 108–116.

  • Epstein, R., Yuille, A.L., and Belhumeur, P.N. 1996. Learning object representations from lighting variations. In Object Representation in Computer Vision II, J. Ponce, A. Zisserman, and M. Hebert (Eds.), Springer Lecture Notes in Computer Science 1144.

  • Fan, J. and Wolff, L.B. 1997. Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Computer Vision and Image Understanding, 65(2):347–359.

    Google Scholar 

  • Forsyth, D. and Zisserman, A. 1991. Reflections on Shading. IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7):671–679.

    Google Scholar 

  • Freeman, W. and Tennenbaum, 1997. Learning bilinear models for two-factor problems in vision. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition, CVPR97, Puerto Rico.

  • Georghiades, A., Kriegman, D., and Belhumeur, P. 1998. Illumination cones for recognition under variable lighting. In Proceedings Computer Vision and Pattern Recognition, CVPR'98.

  • Grenander, U., Chou, Y., and Keenan, K.M. 1991. HANDS: A Pattern Theoretic Study of Biological Shapes. Springer-Verlag: NewYork.

    Google Scholar 

  • Hallinan, P.W. 1994. A low-dimensional lighting representation of human faces for arbitrary lighting conditions. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., pp. 995–999.

  • Hallinan, P.W. 1995. A deformable model for face recognition under arbitrary lighting conditions. Ph.D. Thesis. Division of Applied Sciences, Harvard University.

  • Hayakawa, K. 1994. Photometric Stereo under a light source with arbitrary motion. Journal of the Optical Society of America A, 11(11).

  • Horn, B.K.P. and Brooks, M.J. (Eds.). 1989. Shape from Shading. MIT Press: Cambridge MA.

    Google Scholar 

  • Huber, P.J. 1980. Robust Statistics. JohnWiley and Sons: New York.

    Google Scholar 

  • Iwahori, Y., Woodham, R.J., and Bagheri, A. 1995. Principal components analysis and neural network implementation of photometric stereo. In Proceedings of the IEEE Workshop on Physics-Based Modeling in Computer Vision, pp. 117–125.

  • Kriegman, D. and Belhumeur, P.N. 1998. What shadows reveal about object structure. In Proceedings of the European Conference on Computer Vision, ECCV'98, pp. 399–414.

  • Marr, D. 1982. Vision. Freeman: New York.

    Google Scholar 

  • Moses, Y., Adini, Y., and Ullman, S. 1994. Face recognition: The problem of compensating for changes in illumination direction. In European Conf. on Computer Vision, pp. 286–296.

  • Murase, H. and Nayar, S. 1995. Visual learning and recognition of 3-D objects from appearance. Int. Journal of Computer Vision, 14:5–24.

    Google Scholar 

  • Nayar, S.K., Ikeuchi, K., and Kanade, T. 1991. Surface reflection: Physical and geometrical perspectives. ” IEEE Trans. Pattern Analysis and Machine Intelligence, 13:611–634.

    Google Scholar 

  • Pentland, A., Moghaddam, B., and Starner. 1994. View-based and modular eigenspaces for face recognition. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 84–91.

  • Poggio, T. and Edelman, S. 1990. A network that learns to recognize three-dimensional objects. Nature, 343:263–266.

    Google Scholar 

  • Poggio, T. and Sung, K. 1994. Example-based learning for viewbased human face detection. In Proc. Image UnderstandingWorkshop, pp. II:843–850.

    Google Scholar 

  • Rosch, E., Mervis, C.B., Gray, W.D., Johnson, D.M., and Boyes-Braem, P. 1976. Basic objects in natural categories.Cogn. Psychol., 8:382–439.

    Google Scholar 

  • Shashua, A. 1992. Geometry and photometry in 3D visual recognition. Ph.D. Thesis. MIT.

  • Shum, H-Y., Hebert, M.H. Ikeuchi, K., and Reddy, R. 1997. An integral approach to free-form object modeling. IEEE Transactions of Pattern Analysis and Machine Intelligence, 19(12).

  • Silver, W. 1980. Determining shape and reflectance using multiple images. Ph.D. Thesis. MIT, Cambridge, MA.

    Google Scholar 

  • Teo, P.C., Simoncelli, E.P., and Heeger, D.J. 1997. Efficient linear re-rendering for interactive lighting design. Report No. STAN-CSTN-97-60. Dept. Computer Science, Stanford University

  • Sirovitch, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. J. Optical Soc. of America A, 2:519–524.

    Google Scholar 

  • Solomon, F. and Ikeuchi, K. 1995. An illumination planner for convex and concave objects. In Proceedings of the IEEE Workshop on Physics-Based Modeling in Computer Vision, June 18, 19, pp. 100–107.

  • Tarr, M.J. 1995. Rotating objects to recognize them: A case study of the role of viewpoint dependency in the recognition of threedimensional objects. Psychonomic Bulletin and Review, 2(1):55–82.

    Google Scholar 

  • Tarr, M.J. and Bülthoff, H.H. 1995. Is human object recognition better described by geonstructural-descriptions or by multipleviews?” Journal of Experimental Psychology: Human Perception and Performance, 21:6.

    Google Scholar 

  • Tarr, M.J., Bülthoff, H.H., Zabinski, M., and Blanz, V. 1997. To what extent do unique parts influence recognition across changes in viewpoint? Psychological Science, 8(4):282–289.

    Google Scholar 

  • Tarr, M.J., Kersten, D., and Bülthoff, H.H. 1997. Why the visual system might encode the effects of illumination. Submitted to Vision Research.

  • Tarr, M.J. and Pinker, S. 1989. Mental rotation and orientationdependence in shape recognition. Cognitive Psychology, 21(28):233–282.

    Google Scholar 

  • Taubin, G., Cukierman, F., Sullivan, S., Ponce, J., and Kriegman, D. 1994. Parameterized families of polynomials for bounded algebraic curve and surface fitting. IEEE Trans. Pattern Anal. Mach. Intelligence, 16(3):287–303.

    Google Scholar 

  • Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. J. of Cognitive Neuroscience, 3(1).

  • Woodham, R. 1981. Analyzing images of curved surfaces. Artificial Intelligence, 17:117–140.

    Google Scholar 

  • Yuille, A.L. and Snow, D. 1997. Shape and albedo from multiple images using integrability. In Proceedings of Computer Vision and Pattern Recognition (CVPR'97), Puerto-Rico.

Download references

Author information

Authors and Affiliations

  1. Smith-Kettlewell Eye Research Institute, 2318 Fillmore St., c, San Francisco, CA, 94115

    A.L. Yuille, D. Snow & R. Epstein

  2. Department of Electrical Engineering, Yale University, USA

    P.N. Belhumeur

Authors
  1. A.L. Yuille
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Snow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Epstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. P.N. Belhumeur
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yuille, A., Snow, D., Epstein, R. et al. Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability. International Journal of Computer Vision 35, 203–222 (1999). https://doi.org/10.1023/A:1008180726317

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008180726317

  • Singular Value Decomposition
  • photometric stereo
  • illumination models