Face Recognition Across Pose and Illumination

  • Ralph Gross
  • Simon Baker
  • Iain Matthews
  • Takeo Kanade

Abstract

The last decade has seen automatic face recognition evolve from small-scale research systems to a wide range of commercial products. Driven by the FERET face database and evaluation protocol, the currently best commercial systems achieve verification accuracies comparable to those of fingerprint recognizers. In these experiments, only frontal face images taken under controlled lighting conditions were used. As the use of face recognition systems expands toward less restricted environments, the development of algorithms for view and illumination invariant face recognition becomes important. However, the performance of current algorithms degrades significantly when tested across pose and illumination, as documented in a number of evaluations. In this chapter, we review previously proposed algorithms for pose and illumination invariant face recognition. We then describe in detail two successful appearance-based algorithms for face recognition across pose, eigen light-fields, and Bayesian face subregions. We furthermore show how both of these algorithms can be extended toward face recognition across pose and illumination.

References

  1. 1.
    Adelson, E., Bergen, J.: The plenoptic function and elements of early vision. In: Landy, M., Movshon, J.A. (eds.) Computational Models of Visual Processing. MIT Press, Cambridge (1991) Google Scholar
  2. 2.
    Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 721–732 (1997) CrossRefGoogle Scholar
  3. 3.
    Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003) CrossRefGoogle Scholar
  4. 4.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997) CrossRefGoogle Scholar
  5. 5.
    Belhumeur, P., Kriegman, D.: What is the set of images of an object under all possible lighting conditions. Int. J. Comput. Vis. 28(3), 245–260 (1998) CrossRefGoogle Scholar
  6. 6.
    Beymer, D.: Face recognition under varying pose. Technical Report 1461, MIT AI Laboratory, Cambridge, MA (1993) Google Scholar
  7. 7.
    Beymer, D., Poggio, T.: Face recognition from one example view. A.I. Memo No. 1536, MIT AI Laboratory, Cambridge, MA (1995) Google Scholar
  8. 8.
    Black, M., Jepson, A.: Eigen-tracking: robust matching and tracking of articulated objects using a view-based representation. Int. J. Comput. Vis. 36(2), 101–130 (1998) Google Scholar
  9. 9.
    Blackburn, D., Bone, M., Phillips, P.: Facial recognition vendor test 2000: evaluation report (2000) Google Scholar
  10. 10.
    Blanz, V., Romdhani, S., Vetter, T.: Face identification across different poses and illumination with a 3D morphable model. In: Proceedings of the Fifth International Conference on Face and Gesture Recognition, pp. 202–207 (2002) CrossRefGoogle Scholar
  11. 11.
    Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003) CrossRefGoogle Scholar
  12. 12.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001) CrossRefGoogle Scholar
  13. 13.
    Cootes, T., Wheeler, G., Walker, K., Taylor, C.: View-based active appearance models. Image Vis. Comput. 20, 657–664 (2002) CrossRefGoogle Scholar
  14. 14.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, San Diego (1990) MATHGoogle Scholar
  15. 15.
    Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Generative models for recognition under variable pose and illumination. In: Proceedings of the Fourth International Conference on Face and Gesture Recognition, pp. 277–284 (2000) CrossRefGoogle Scholar
  16. 16.
    Georghiades, A., Kriegman, D., Belhumeur, P.: Illumination cones for recognition under variable lighting: faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1998) Google Scholar
  17. 17.
    Georghiades, A., Kriegman, D., Belhumeur, P.: From few to many: generative models for recognition under variable pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001) CrossRefGoogle Scholar
  18. 18.
    Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH), pp. 43–54 (1996) Google Scholar
  19. 19.
    Graham, D., Allison, N.: Face recognition from unfamiliar views: subspace methods and pose dependency. In: 3rd International Conference on Automatic Face and Gesture Recognition, pp. 348–353 (1998) CrossRefGoogle Scholar
  20. 20.
    Gross, R., Brajovic, V.: An image pre-processing algorithm for illumination invariant face recognition. In: 4th International Conference on Audio- and Video Based Biometric Person Authentication (AVBPA), pp. 10–18, June 2003 CrossRefGoogle Scholar
  21. 21.
    Gross, R., Matthews, I., Baker, S.: Eigen light-fields and face recognition across pose. In: Proceedings of the Fifth International Conference on Face and Gesture Recognition, pp. 1–7 (2002) Google Scholar
  22. 22.
    Gross, R., Matthews, I., Baker, S.: Fisher light-fields for face recognition across pose and illumination. In: Proceedings of the German Symposium on Pattern Recognition (DAGM), pp. 481–489 (2002) Google Scholar
  23. 23.
    Gross, R., Matthews, I., Baker, S.: Appearance-based face recognition and light-fields. IEEE Trans. Pattern Anal. Mach. Intell. 26(4), 449–465 (2004) CrossRefGoogle Scholar
  24. 24.
    Gross, R., Shi, J., Cohn, J.: Quo vadis face recognition. In: Third Workshop on Empirical Evaluation Methods in Computer Vision (2001) Google Scholar
  25. 25.
    Horn, B.: Determining lightness from an image. Comput. Graph. Image Process. 3(1), 277–299 (1974) CrossRefGoogle Scholar
  26. 26.
    Horn, B.: Robot Vision. MIT Press, Cambridge (1986) Google Scholar
  27. 27.
    Jacobs, D., Belhumeur, P., Basri, R.: Comparing images under variable illumination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 610–617 (1998) Google Scholar
  28. 28.
    Kanade, T., Yamada, A.: Multi-subregion based probabilistic approach toward pose-invariant face recognition. In: IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA2003), pp. 954–959 (2003) CrossRefGoogle Scholar
  29. 29.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998) CrossRefGoogle Scholar
  30. 30.
    Land, E., McCann, J.: Lightness and retinex theory. J. Opt. Soc. Am. 61(1), 1–11 (1971) CrossRefGoogle Scholar
  31. 31.
    Lando, M., Edelman, S.: Generalization from a single view in face recognition. In: International Workshop on Automatic Face-and Gesture-Recognition (1995) Google Scholar
  32. 32.
    Lanitis, A., Taylor, C., Cootes, T.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 743–756 (1997) CrossRefGoogle Scholar
  33. 33.
    Leonardis, A., Bischof, H.: Robust recognition using eigenimages. Comput. Vis. Image Underst. 78(1), 99–118 (2000) CrossRefGoogle Scholar
  34. 34.
    Levoy, M., Hanrahan, M.: Light field rendering. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH), pp. 31–41 (1996) Google Scholar
  35. 35.
    Maurer, T., von der Malsburg, C.: Single-view based recognition of faces rotated in depth. In: International Workshop on Automatic Face and Gesture Recogition, pp. 248–253 (1995) Google Scholar
  36. 36.
    Murase, H., Nayar, S.: Visual learning and recognition of 3-D objects from appearance. Int. J. Comput. Vis. 14, 5–24 (1995) CrossRefGoogle Scholar
  37. 37.
    Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 84–91 (1994) CrossRefGoogle Scholar
  38. 38.
    Phillips, P.J., Grother, P., Ross, J., Blackburn, D., Tabassi, E., Bone, M.: Face recognition vendor test 2002: evaluation report, March 2003 Google Scholar
  39. 39.
    Phillips, P.J., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000) CrossRefGoogle Scholar
  40. 40.
    Romdhani, S., Blanz, V., Vetter, T.: Face identification by matching a 3D morphable model using linear shape and texture error functions. In: Proceedings of the European Conference on Computer Vision, pp. 3–19 (2002) Google Scholar
  41. 41.
    Romdhani, S., Gong, S., Psarrou, A.: Multi-view nonlinear active shape model using kernel PCA. In: 10th British Machine Vision Conference, vol. 2, pp. 483–492 (1999) Google Scholar
  42. 42.
    Romdhani, S., Psarrou, A., Gong, S.: On utilising template and feature-based correspondence in multi-view appearance models. In: 6th European Conference on Computer Vision, vol. 1, pp. 799–813 (2000) Google Scholar
  43. 43.
    Shashua, A.: Geometry and photometry in 3D visual recognition. PhD thesis, MIT (1994) Google Scholar
  44. 44.
    Shashua, A., Riklin-Raviv, T.: The Quotient image: class-based re-rendering and recognition with varying illumination conditions. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 129–139 (2001) CrossRefGoogle Scholar
  45. 45.
    Sim, T., Kanade, T.: Combining models and exemplars for face recognition: an illuminating example. In: Workshop on Models Versus Exemplars in Computer Vision (2001) Google Scholar
  46. 46.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003) CrossRefGoogle Scholar
  47. 47.
    Turk, M., Pentland, A.: Face recognition using eigenfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1991) Google Scholar
  48. 48.
    Vetter, T., Poggio, T.: Linear object classes and image synthesis from a single example image. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 733–741 (1997) CrossRefGoogle Scholar
  49. 49.
    Wiskott, L., Fellous, J., Kruger, N., von der Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 775–779 (1997) CrossRefGoogle Scholar
  50. 50.
    Zhao, W., Chellappa, R.: Robust face recognition using symmetric shape-from-shading. Technical report, Center for Automation Research, University of Maryland (1999) Google Scholar
  51. 51.
    Zhao, W., Krishnaswamy, A., Chellappa, R., Swets, D., Weng, J.: Discriminant analysis of principal components for face recognition. In: Wechsler, H., Phillips, P.J., Bruce, V., Huang, T. (eds.) Face Recognition: From Theory to Applications. Springer, New York (1998) Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Ralph Gross
    • 1
  • Simon Baker
    • 1
  • Iain Matthews
    • 1
  • Takeo Kanade
    • 1
  1. 1.Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations