Illumination Modeling for Face Recognition

  • Ronen BasriEmail author
  • David Jacobs


In this chapter, we show that effective systems can account for the effects of lighting using fewer than 10 degrees of freedom. This can have considerable impact on the speed and accuracy of recognition systems. We will describe theoretical results that, with some simplifying assumptions, prove the validity of low-dimensional, linear approximations to the set of images produced by a face.


Face Recognition Linear Subspace Query Image Harmonic Image Cast Shadow 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Major portions of this research were conducted while Ronen Basri and David Jacobs were at the NEC Research Institute, Princeton, NJ. At the Weizmann Institute Ronen Basri is supported in part by European Community grants IST-2000-26001 VIBES and IST-2002-506766 Aim Shape and by the Israel Science Foundation grant 266/02. The vision group at the Weizmann Institute is supported in part by the Moross Foundation. David Jacobs was funded by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), through the Army Research Laboratory (ARL). All statements of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representing the official views or policies of IARPA, the ODNI, or the U.S. Government.


  1. 1.
    Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 721–732 (1997) CrossRefGoogle Scholar
  2. 2.
    Angelopoulou, E.: Understanding the color of human skin. In: SPIE Conf. on Human Vision and Electronic Imaging VI, vol. 4299, pp. 243–251. SPIE, Bellingham (2001) Google Scholar
  3. 3.
    Angelopoulou, E., Molana, R., Daniilidis, K.: Multispectral skin color modeling. In: IEEE Conf. on Computer Vision and Patt. Recognition, pp. 635–642 (2001) Google Scholar
  4. 4.
    Basri, R., Jacobs, D.: Lambertian reflectances and linear subspaces. In: IEEE Int. Conf. on Computer Vision, vol. II, pp. 383–390 (2001) Google Scholar
  5. 5.
    Basri, R., Jacobs, D.: Photometric stereo with general, unknown lighting. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. II, pp. 374–381 (2001) Google Scholar
  6. 6.
    Basri, R., Jacobs, D.: Lambertian reflectances and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003) CrossRefGoogle Scholar
  7. 7.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007) CrossRefGoogle Scholar
  8. 8.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997) CrossRefGoogle Scholar
  9. 9.
    Belhumeur, P., Kriegman, D.: What is the set of images of an object under all possible lighting conditions? Int. J. Comput. Vis. 28(3), 245–260 (1998) CrossRefGoogle Scholar
  10. 10.
    Blicher, A., Roy, S.: Fast lighting/rendering solution for matching a 2d image to a database of 3d models: ‘lightsphere’. IEICE Trans. Inf. Syst. E84-D(12), 1722–1727 (2001) Google Scholar
  11. 11.
    Borshukov, G., Lewis, J.: Realistic human face rendering for ‘the matrix reloaded’. In: SIGGRAPH-2003 Sketches and Applications Program (2003) Google Scholar
  12. 12.
    Brunelli, R., Poggio, T.: Face recognition: Features versus templates. IEEE Trans. Pattern Anal. Mach. Intell. 15(10), 1042–1062 (1993) CrossRefGoogle Scholar
  13. 13.
    Chen, H., Belhumeur, P., Jacobs, D.: In search of illumination invariants. In: IEEE Proc. Computer Vision and Pattern Recognition, vol. I, pp. 254–261 (2000) CrossRefGoogle Scholar
  14. 14.
    Epstein, R., Hallinan, P., Yuille, A.: pm2 eigenimages suffice: an empirical investigation of low-dimensional lighting models. In: IEEE Workshop on Physics-Based Vision, pp. 108–116 (1995) CrossRefGoogle Scholar
  15. 15.
    Frolova, D., Simakov, D., Basri, R.: Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting. In: ECCV, pp. 574–587 (2004) Google Scholar
  16. 16.
    Georghiades, A.: Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. In: International Conference on Computer Vision, vol. II, pp. 816–823 (2003) CrossRefGoogle Scholar
  17. 17.
    Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: generative models for recognition under variable pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001) CrossRefGoogle Scholar
  18. 18.
    Georghiades, A., Kriegman, D., Belhumeur, P.: Illumination cones for recognition under variable lighting: faces. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 52–59 (1998) Google Scholar
  19. 19.
    Hallinan, P.: A low-dimensional representation of human faces for arbitrary lighting conditions. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 995–999 (1994) Google Scholar
  20. 20.
    Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. J. Opt. Soc. Am. 11(11), 3079–3089 (1994) MathSciNetCrossRefGoogle Scholar
  21. 21.
    Ishiyama, R., Sakamoto, S.: Geodesic illumination basis: compensating for illumination variations in any pose for face recognition. In: IEEE Int. Conf. on Pattern Recognition, vol. 4, pp. 297–301 (2002) Google Scholar
  22. 22.
    Ho, J., Yang, M., Lim, J., Lee, K., Kriegman, D.: Clustering appearances of objects under varying illumination conditions. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 11–18 (2003) Google Scholar
  23. 23.
    Jacobs, D.: Linear fitting with missing data for structure-from-motion. Comput. Vis. Image Underst. 82(1), 57–81 (2001) zbMATHCrossRefGoogle Scholar
  24. 24.
    Jacobs, D., Belhumeur, P., Basri, R.: Comparing images under variable illumination. In: IEEE Proc. Computer Vision and Pattern Recognition, pp. 610–617 (1998) Google Scholar
  25. 25.
    Jensen, H., Marschner, S., Levoy, M., Hanrahan, P.: A practical model for subsurface light transport. In: Proc. SIGGRAPH, pp. 511–518 (2001) Google Scholar
  26. 26.
    Kemelmacher, I., Basri, R.: Molding face shapes by example. In: European Conf. on Computer Vision. LNCS, vol. 3951, pp. 277–288 (2006) Google Scholar
  27. 27.
    Kemelmacher-Shlizerman, I., Basri, R.: 3d face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. (forthcoming) Google Scholar
  28. 28.
    Koenderink, J., Doorn, A.V.: The generic bilinear calibration-estimation problem. Int. J. Comput. Vis. 23(3), 217–234 (1997) CrossRefGoogle Scholar
  29. 29.
    Lades, M., Vorbruggen, J., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R., Konen, W.: Distortion invariant object recognition in the dynamic link architecture. IEEE Trans. Comput. 42(3), 300–311 (1993) CrossRefGoogle Scholar
  30. 30.
    Lambert, J.: Photometria sive de mensura et gradibus luminus, colorum et umbrae. Eberhard Klett (1760) Google Scholar
  31. 31.
    Lee, K., Ho, J., Kriegman, D.: Nine points of light: acquiring subspaces for face recognition under variable lighting. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 519–526 (2001) Google Scholar
  32. 32.
    Marschner, S., Westin, S., Lafortune, E., Torrance, K., Greenberg, D.: Image-based brdf measurement including human skin. In: 10th Eurographics Workshop on Rendering, pp. 131–144 (1999) Google Scholar
  33. 33.
    Meglinski, I., Matcher, S.: Quantitative assessment of skin layers absorption and skin reflectance spectra simulation in the visible and near-infrared spectral regions. Physiol. Meas. 23, 741–753 (2002) CrossRefGoogle Scholar
  34. 34.
    Moses, Y.: Face recognition: generalization to novel images. Ph.D. thesis, Weizmann Institute of Science (1993) Google Scholar
  35. 35.
    Moses, Y., Ullman, S.: Limitations of non model-based recognition schemes. In: Second European Conference on Computer Vision, pp. 820–828 (1992) Google Scholar
  36. 36.
    Osadchy, M., Jacobs, D., Ramamoorthi, R.: Using specularities for recognition. In: International Conference on Computer Vision, vol. II, pp. 1512–1519 (2003) CrossRefGoogle Scholar
  37. 37.
    Ramamoorthi, R.: Analytic pca construction for theoretical analysis of lighting variability in a single image of a Lambertian object. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(10) (2002) Google Scholar
  38. 38.
    Ramamoorthi, R., Hanrahan, P.: On the relationship between radiance and irradiance: determining the illumination from images of convex Lambertian object. J. Opt. Soc. Am. 18(10), 2448–2459 (2001) MathSciNetCrossRefGoogle Scholar
  39. 39.
    Sato, I., Okabe, T., Sato, Y., Ikeuchi, K.: Appearance sampling for obtaining a set of basis images for variable illumination. In: IEEE Int. Conf. on Computer Vision, vol. II, pp. 800–807 (2003) CrossRefGoogle Scholar
  40. 40.
    Shashua, A.: On photometric issues in 3d visual recognition from a single 2d image. Int. J. Comput. Vis. 21(1–2), 99–122 (1997) CrossRefGoogle Scholar
  41. 41.
    Shirdhonkar, S., Jacobs, D.: Non-negative lighting and specular object recognition. In: IEEE International Conference on Computer Vision, vol. II, pp. 1323–1330 (2005) Google Scholar
  42. 42.
    Shum, H., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Trans. Pattern Anal. Mach. Intell. 17(9), 854–867 (1995) CrossRefGoogle Scholar
  43. 43.
    Simakov, D., Frolova, D., Basri, R.: Dense shape reconstruction of a moving object under arbitrary, unknown lighting. In: IEEE Int. Conf. on Computer Vision, pp. 1202–1209 (2003) CrossRefGoogle Scholar
  44. 44.
    Sirovitch, L., Kirby, M.: Low-dimensional procedure for the characterization of human faces. J. Opt. Soc. Am. 2, 586–591 (1987) Google Scholar
  45. 45.
    Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–96 (1991) CrossRefGoogle Scholar
  46. 46.
    Wiberg, T.: Computation of principal components when data are missing. In: Proc. Second Symp. Computational Statistics, pp. 229–236 (1976) Google Scholar
  47. 47.
    Yuille, A., Snow, D., Epstein, R., Belhumeur, P.: Determining generative models of objects under varying illumination: shape and albedo from multiple images using svd and integrability. Int. J. Comput. Vis. 35(3), 203–222 (1999) CrossRefGoogle Scholar
  48. 48.
    Zhang, L., Curless, B., Hertzmann, A., Seitz, S.: Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multi-view stereo. In: IEEE Int. Conf. on Computer Vision, pp. 618–625 (2003) CrossRefGoogle Scholar
  49. 49.
    Zhang, L., Samaras, D.: Face recognition under variable lighting using harmonic image exemplars. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. I, pp. 19–25 (2003) Google Scholar
  50. 50.
    Zou, X., Kittler, J., Messer, K.: Illumination invariant face recognition: A survey. In: Biometrics: Theory, Applications, and Systems, pp. 1–8 (2007) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.The Weizmann Institute of ScienceRehovotIsrael
  2. 2.University of MarylandCollege ParkUSA

Personalised recommendations