Skip to main content
SpringerLink
Log in

Photometric Stereo with General, Unknown Lighting

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Work on photometric stereo has shown how to recover the shape and reflectance properties of an object using multiple images taken with a fixed viewpoint and variable lighting conditions. This work has primarily relied on known lighting conditions or the presence of a single point source of light in each image. In this paper we show how to perform photometric stereo assuming that all lights in a scene are distant from the object but otherwise unconstrained. Lighting in each image may be an unknown and may include arbitrary combination of diffuse, point and extended sources. Our work is based on recent results showing that for Lambertian objects, general lighting conditions can be represented using low order spherical harmonics. Using this representation we can recover shape by performing a simple optimization in a low-dimensional space. We also analyze the shape ambiguities that arise in such a representation. We demonstrate our method by reconstructing the shape of objects from images obtained under a variety of lightings. We further compare the reconstructed shapes against shapes obtained with a laser scanner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Basri, R. and Jacobs, D. 2001. Photometric stereo with general, unknown lighting. IEEE International Conference on Computer Vision, Kauai, II:374–381.

  • Basri, R. and Jacobs, D. 2003. Lambertian reflectances and linear subspaces. IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(2):218–233.

    Article  Google Scholar 

  • Belhumeur, P.N. and Kriegman, D.J. 1998. What is the set of images of an object under all possible lighting conditions? International Journal of Computer Vision, 28(3):245–260.

    Article  Google Scholar 

  • Belhumeur, P.N., Kriegman, D.J., and Yuille, A.L. 1999. The Bas-relief ambiguity. International Journal of Computer Vision, 35(1):33–44.

    Article  Google Scholar 

  • Barsky, S. and Petrou, M. 2003. The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1239–1252.

    Article  Google Scholar 

  • Clark, J.J. 1992. Active photometric stereo. IEEE International Conference on Computer Vision, 29–34.

  • Coleman, Jr., E.N. and Jain, R. 1982. Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry, Computer Graphics and Image Processing, 18(4):309–328.

    Google Scholar 

  • Fan, J. and Wolff, L.B. 1997. Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Computer Vision and Image Understanding, 65(2):347–359.

    Article  Google Scholar 

  • Frolova, D., Simakov, D., and Basri, R. 2004. Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting. European Conf. on Computer Vision, Prague, I:574–587.

  • Georghiades, A. 2003. Recovering 3-D shape and reflectance from a small number of photographs. In Christensen, P. and Cohen-Or D. (Eds.), Eurographics Symposium on Rendering, pp. 230–240.

  • Georghiades, A. 2003. Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. IEEE International Conference on Computer Vision. 816–823.

  • Hartley, R.I. 1997. In Defense of the eight-point algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(6):580–593.

    Article  Google Scholar 

  • Hayakawa, H. 1994. Photometric stereo under a light source with arbitrary motion. Journal of the Optical Society in America, 11(11):3079–3089.

    Article  MathSciNet  Google Scholar 

  • Hertzmann, A. and Seitz, S.M. 2005. Example-based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(8):1254–1264.

    Article  Google Scholar 

  • Horn, B.K.P. 1986. Robot Vision. MIT Press.

  • Horovitz, I. and Kiryati, N. 2004. Depth from gradient fields and control points: Bias correction in photometric stereo. Image and Vision Computing, 22(9):681–694.

    Article  Google Scholar 

  • Ikeuchi, K. 1981. Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. on Pattern Analysis and Machine Intelligence, 3(6):661–669.

    Article  Google Scholar 

  • Jacobs, D. 2001. Linear fitting with missing data for structure-from-motion. Computer Vision and Image Understanding, 82(1):57–81.

    Article  MATH  Google Scholar 

  • Jacobs, D., Belhumeur, P., and Basri, R. 1998. Comparing images under variable illumination. IEEE International Conference on Computer Vision Santa Barbara, 610–617.

  • Kanatani, K. 1993. Geometric Computation for Machine Vision. Oxford University Press.

  • Kim, B. and Burger, P. 1991. Depth and shape form shading using the photometric stereo method. Computer Vision, Graphics and Image Processing, 54(3):416–427.

    Google Scholar 

  • Koenderink, J. and van Doorn, A. 1997. The generic bilinear calibration-estimation problem. International Journal of Computer Vision, 23(3):217–234.

    Article  Google Scholar 

  • Luong, Q.T., Fua, P., and Leclerc, Y.G. 2002. The radiometry of multiple images. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(1):19–33.

    Article  Google Scholar 

  • Moses, Y. 1993. Face Recognition: Generalization to Novel Images. Ph.D. Thesis, Weizmann Inst. of Science.

  • Nayar, S.K., Ikeuchi, K., and Kanade, T. 1990. Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Transactions on Robotics and Automation, 6(4):418–431.

    Article  Google Scholar 

  • Okatani, T. and Deguchi, K. 2001. On uniqueness of solutions of the three-light-source photometric stereo: Conditions on illumination configuration and surface reflectance. Computer Vision and Image Understanding, 81(2):211-226.

    Article  MATH  Google Scholar 

  • Onn, R. and Bruckstein, A.M. 1990. Integrability disambiguates surface recovery in two-image photometric stereo. International Journal of Computer Vision, 5(1):105–113.

    Article  Google Scholar 

  • Ramamoorthi, R. 2002. Analytic PCA construction for theoretical analysis of lighting variability in images of a lambertian object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10):1322–1333.

    Article  Google Scholar 

  • Ramamoorthi, R. and Hanrahan, P. 2001. On the relationship between radiance and irradiance: Determining the illumination from images of convex Lambertian object. Journal of the Optical Society in America A, 2448–2459.

  • Sakarya, U. and Erkmen, I. 2003. An improved method of photometric stereo using local shape from shading. Image and Vision Computing, 21(11):941–954.

    Article  Google Scholar 

  • Shashua, A. 1997. On photometric issues in 3D visual recognition from a single 2D image. International Journal of Computer Vision, 21(1–2):99–122.

    Article  Google Scholar 

  • Shum, H.Y., Ikeuchi, K., and Reddy, R. 1995. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9):854–867.

    Article  Google Scholar 

  • Simakov, D., Frolova, D., Basri, R. 2003. Dense shape reconstruction of a moving object under arbitrary, unknown lighting. IEEE Int. Conf. on Computer Vision, 1202–1209.

  • Smith, M.L. 1999. The analysis of surface texture using photometric stereo acquisition and gradient space domain mapping. Image and Vision Computing, 17(14):1009–1019.

    Article  Google Scholar 

  • Tagare, H.D. and de Figueiredo, R.J.P. 1991. A theory of photometric stereo for a class of diffuse Non-Lambertian surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(2):133–152.

    Article  Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9 (2):137–154.

    Article  Google Scholar 

  • Woodham, R.J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1):139–144.

    Google Scholar 

  • Woodham, R.J., Iwahori, Y., and Barman, R.A. 1991. Photometric stereo: Lambertian reflectance and light sources with unknown direction and strength. University of British Columbia, TR-91-18.

  • Wiberg, T. 1976. Computation of principal components when data are missing. In Proc. Second Symposium of Computational Statistics, pp. 229–236.

  • Yuille, A., Snow, D., Epstein, R., and Belhumeur, P. 1999. Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability. International Journal of Computer Vision, 35(3):203– 222.

    Article  Google Scholar 

  • Zhang, L., Curless, B., Hertzmann, A., and Seitz, S.M. 2003. Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. IEEE International Conference on Computer Vision pp. 618–625.

  • Zhang, R., Tsai, P.S., and Shah, M. 1996. Photomotion. Computer Vision and Image Understanding, 63(2):221–231.

    Article  Google Scholar 

  • Zhou, S.K., Chellappa, R., and Jacobs, D.W. 2004. Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints. European Conference on Computer Vision, Prague, I: 588–601.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Applied Math, The Weizmann Institute of Science, Rehovot, 76100, Israel

    Ronen Basri & Ira Kemelmacher

  2. Department of Computer Science, University of Maryland, College Park, MD, 20742

    David Jacobs

Authors
  1. Ronen Basri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Jacobs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ira Kemelmacher
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Basri, R., Jacobs, D. & Kemelmacher, I. Photometric Stereo with General, Unknown Lighting. Int J Comput Vision 72, 239–257 (2007). https://doi.org/10.1007/s11263-006-8815-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-8815-7

Keywords

Search

Navigation