International Journal of Computer Vision

, Volume 72, Issue 3, pp 239–257 | Cite as

Photometric Stereo with General, Unknown Lighting

  • Ronen Basri
  • David Jacobs
  • Ira Kemelmacher
Article

Abstract

Work on photometric stereo has shown how to recover the shape and reflectance properties of an object using multiple images taken with a fixed viewpoint and variable lighting conditions. This work has primarily relied on known lighting conditions or the presence of a single point source of light in each image. In this paper we show how to perform photometric stereo assuming that all lights in a scene are distant from the object but otherwise unconstrained. Lighting in each image may be an unknown and may include arbitrary combination of diffuse, point and extended sources. Our work is based on recent results showing that for Lambertian objects, general lighting conditions can be represented using low order spherical harmonics. Using this representation we can recover shape by performing a simple optimization in a low-dimensional space. We also analyze the shape ambiguities that arise in such a representation. We demonstrate our method by reconstructing the shape of objects from images obtained under a variety of lightings. We further compare the reconstructed shapes against shapes obtained with a laser scanner.

Keywords

photometric stereo Lambertian reflectance Lorentz transformation diffuse lighting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Basri, R. and Jacobs, D. 2001. Photometric stereo with general, unknown lighting. IEEE International Conference on Computer Vision, Kauai, II:374–381.Google Scholar
  2. Basri, R. and Jacobs, D. 2003. Lambertian reflectances and linear subspaces. IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(2):218–233.CrossRefGoogle Scholar
  3. Belhumeur, P.N. and Kriegman, D.J. 1998. What is the set of images of an object under all possible lighting conditions? International Journal of Computer Vision, 28(3):245–260.CrossRefGoogle Scholar
  4. Belhumeur, P.N., Kriegman, D.J., and Yuille, A.L. 1999. The Bas-relief ambiguity. International Journal of Computer Vision, 35(1):33–44.CrossRefGoogle Scholar
  5. Barsky, S. and Petrou, M. 2003. The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1239–1252.CrossRefGoogle Scholar
  6. Clark, J.J. 1992. Active photometric stereo. IEEE International Conference on Computer Vision, 29–34.Google Scholar
  7. Coleman, Jr., E.N. and Jain, R. 1982. Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry, Computer Graphics and Image Processing, 18(4):309–328.Google Scholar
  8. Fan, J. and Wolff, L.B. 1997. Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Computer Vision and Image Understanding, 65(2):347–359.CrossRefGoogle Scholar
  9. Frolova, D., Simakov, D., and Basri, R. 2004. Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting. European Conf. on Computer Vision, Prague, I:574–587.Google Scholar
  10. Georghiades, A. 2003. Recovering 3-D shape and reflectance from a small number of photographs. In Christensen, P. and Cohen-Or D. (Eds.), Eurographics Symposium on Rendering, pp. 230–240.Google Scholar
  11. Georghiades, A. 2003. Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. IEEE International Conference on Computer Vision. 816–823.Google Scholar
  12. Hartley, R.I. 1997. In Defense of the eight-point algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(6):580–593.CrossRefGoogle Scholar
  13. Hayakawa, H. 1994. Photometric stereo under a light source with arbitrary motion. Journal of the Optical Society in America, 11(11):3079–3089.MathSciNetCrossRefGoogle Scholar
  14. Hertzmann, A. and Seitz, S.M. 2005. Example-based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(8):1254–1264.CrossRefGoogle Scholar
  15. Horn, B.K.P. 1986. Robot Vision. MIT Press.Google Scholar
  16. Horovitz, I. and Kiryati, N. 2004. Depth from gradient fields and control points: Bias correction in photometric stereo. Image and Vision Computing, 22(9):681–694.CrossRefGoogle Scholar
  17. Ikeuchi, K. 1981. Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. on Pattern Analysis and Machine Intelligence, 3(6):661–669.CrossRefGoogle Scholar
  18. Jacobs, D. 2001. Linear fitting with missing data for structure-from-motion. Computer Vision and Image Understanding, 82(1):57–81.CrossRefMATHGoogle Scholar
  19. Jacobs, D., Belhumeur, P., and Basri, R. 1998. Comparing images under variable illumination. IEEE International Conference on Computer Vision Santa Barbara, 610–617.Google Scholar
  20. Kanatani, K. 1993. Geometric Computation for Machine Vision. Oxford University Press.Google Scholar
  21. Kim, B. and Burger, P. 1991. Depth and shape form shading using the photometric stereo method. Computer Vision, Graphics and Image Processing, 54(3):416–427.Google Scholar
  22. Koenderink, J. and van Doorn, A. 1997. The generic bilinear calibration-estimation problem. International Journal of Computer Vision, 23(3):217–234.CrossRefGoogle Scholar
  23. Luong, Q.T., Fua, P., and Leclerc, Y.G. 2002. The radiometry of multiple images. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(1):19–33.CrossRefGoogle Scholar
  24. Moses, Y. 1993. Face Recognition: Generalization to Novel Images. Ph.D. Thesis, Weizmann Inst. of Science.Google Scholar
  25. Nayar, S.K., Ikeuchi, K., and Kanade, T. 1990. Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Transactions on Robotics and Automation, 6(4):418–431.CrossRefGoogle Scholar
  26. Okatani, T. and Deguchi, K. 2001. On uniqueness of solutions of the three-light-source photometric stereo: Conditions on illumination configuration and surface reflectance. Computer Vision and Image Understanding, 81(2):211-226.CrossRefMATHGoogle Scholar
  27. Onn, R. and Bruckstein, A.M. 1990. Integrability disambiguates surface recovery in two-image photometric stereo. International Journal of Computer Vision, 5(1):105–113.CrossRefGoogle Scholar
  28. Ramamoorthi, R. 2002. Analytic PCA construction for theoretical analysis of lighting variability in images of a lambertian object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10):1322–1333.CrossRefGoogle Scholar
  29. Ramamoorthi, R. and Hanrahan, P. 2001. On the relationship between radiance and irradiance: Determining the illumination from images of convex Lambertian object. Journal of the Optical Society in America A, 2448–2459.Google Scholar
  30. Sakarya, U. and Erkmen, I. 2003. An improved method of photometric stereo using local shape from shading. Image and Vision Computing, 21(11):941–954.CrossRefGoogle Scholar
  31. Shashua, A. 1997. On photometric issues in 3D visual recognition from a single 2D image. International Journal of Computer Vision, 21(1–2):99–122.CrossRefGoogle Scholar
  32. Shum, H.Y., Ikeuchi, K., and Reddy, R. 1995. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(9):854–867.CrossRefGoogle Scholar
  33. Simakov, D., Frolova, D., Basri, R. 2003. Dense shape reconstruction of a moving object under arbitrary, unknown lighting. IEEE Int. Conf. on Computer Vision, 1202–1209.Google Scholar
  34. Smith, M.L. 1999. The analysis of surface texture using photometric stereo acquisition and gradient space domain mapping. Image and Vision Computing, 17(14):1009–1019.CrossRefGoogle Scholar
  35. Tagare, H.D. and de Figueiredo, R.J.P. 1991. A theory of photometric stereo for a class of diffuse Non-Lambertian surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(2):133–152.CrossRefGoogle Scholar
  36. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9 (2):137–154.CrossRefGoogle Scholar
  37. Woodham, R.J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1):139–144.Google Scholar
  38. Woodham, R.J., Iwahori, Y., and Barman, R.A. 1991. Photometric stereo: Lambertian reflectance and light sources with unknown direction and strength. University of British Columbia, TR-91-18.Google Scholar
  39. Wiberg, T. 1976. Computation of principal components when data are missing. In Proc. Second Symposium of Computational Statistics, pp. 229–236.Google Scholar
  40. Yuille, A., Snow, D., Epstein, R., and Belhumeur, P. 1999. Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability. International Journal of Computer Vision, 35(3):203– 222.CrossRefGoogle Scholar
  41. Zhang, L., Curless, B., Hertzmann, A., and Seitz, S.M. 2003. Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. IEEE International Conference on Computer Vision pp. 618–625.Google Scholar
  42. Zhang, R., Tsai, P.S., and Shah, M. 1996. Photomotion. Computer Vision and Image Understanding, 63(2):221–231.CrossRefGoogle Scholar
  43. Zhou, S.K., Chellappa, R., and Jacobs, D.W. 2004. Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints. European Conference on Computer Vision, Prague, I: 588–601.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Ronen Basri
    • 1
  • David Jacobs
    • 2
  • Ira Kemelmacher
    • 1
  1. 1.Department of Computer Science and Applied MathThe Weizmann Institute of ScienceRehovotIsrael
  2. 2.Department of Computer ScienceUniversity of MarylandCollege Park

Personalised recommendations