Pose Invariant Face Recognition Under Arbitrary Illumination Based on 3D Face Reconstruction

  • Xiujuan Chai
  • Laiyun Qing
  • Shiguang Shan
  • Xilin Chen
  • Wen Gao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3546)


Pose and illumination changes from picture to picture are two main barriers toward full automatic face recognition. In this paper, a novel method to handle both pose and lighting condition simultaneously is proposed, which calibrates the pose and lighting condition to a pre-set reference condition through an illumination invariant 3D face reconstruction. First, some located facial landmarks and a priori statistical deformable 3D model are used to recover an elaborate 3D shape. Based on the recovered 3D shape, the “texture image” calibrated to a standard illumination is generated by spherical harmonics ratio image and finally the illumination independent 3D face is reconstructed completely. The proposed method combines the strength of statistical deformable model to describe the shape information and the compact representations of the illumination in spherical frequency space, and handle both the pose and illumination variation simultaneously. This algorithm can be used to synthesize virtual views of a given face image and enhance the performance of face recognition. The experimental results on CMU PIE database show that this method can significantly improve the accuracy of the existed face recognition method when pose and illumination are inconsistent between gallery and probe sets.


Face Recognition Face Image Texture Image Face Recognition System Gallery Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basria, R., Jacobs, D.: Lambertian Reflectance and Linear Subspaces. In: Proc. ICCV 2001, pp. 383–390 (2001)Google Scholar
  2. 2.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs Fisherfaces: recognition using class specific linear projection. IEEE Trans. on PAMI 20(7) (1997)Google Scholar
  3. 3.
    Blanz, V., Vetter, T.: Face Recognition based on Fitting a 3D Morphable Model. IEEE Transactions on PAMI 25, 1063–1074 (2003)Google Scholar
  4. 4.
    Cao, B., Shan, S., Gao, W.: Localizing the Iris Center by Region Growing Search. In: Proceeding of the ICME 2002, vol. 2, pp. 129–132 (2002)Google Scholar
  5. 5.
    Gao, W., Cao, B., Shan, S.G., Zhou, D.L., Zhang, X.H., Zhao, D.B.: The CAS-PEAL Large-Scale Chinese Face DataBase and Evaluation Protocols., Technique Report No. JDL-TR_04_FR_001, Joint Research & Development Laboratory, CAS (2004)Google Scholar
  6. 6.
    Georghiades, A.S., Belhumeur, P.N., Keiegman, D.J.: From Few to Many: Illuminition Cone Models for Face Recognition under Variable Lighting and Poses. IEEE Transactions on PAMI 23, 643–660 (2001)Google Scholar
  7. 7.
    Gross, R., Matthews, I., Baker, S.: Eigen Light-Fields and Face Recognition Across Pose. In: Proc. FG 2002 (2002)Google Scholar
  8. 8.
    Huang, T.S., Lee, C.H.: Motion and Structure from Orthorgraphic Projections. IEEE Transactions on PAMI 2(5), 536–540 (1989)Google Scholar
  9. 9.
    Liu, C., Wechsler, H.: Gabor Feature Based Classification Using the Enhanced Fisher Linear Discriminant Model for Face Recognition. IEEE Trans. Image Processing 11(4), 467–476 (2002)CrossRefGoogle Scholar
  10. 10.
    Sim, T., Baker, S., Bsat, M.: The CMU Pose, Illumination, and Expression (PIE) Database. In: Proc. the 5th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2002), Washington, DC (May 2002)Google Scholar
  11. 11.
    Turk, M., Pentland, A.: Eigenfaces for Recognition. Journal of cognitive neuroscience 3(1), 71–86 (1991)CrossRefGoogle Scholar
  12. 12.
    Phillips, P., Grother, P., Micheals, R., Blackburn, D., Tabassi, E., Bone, M.: Face Recognition Vendor Test 2002: Evaluation Report. In: FRVT (2002)Google Scholar
  13. 13.
    Zhao, W., Chellappa, R.: SFS Based View Synthesis for Robust Face Recognition. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, Grenoble, pp. 285–292 (2000)Google Scholar
  14. 14.
    Zhou, K., Chellappa, R.: Illuminating Light Field: Image-based Face Recognition across Illuminations and Poses. In: Proc. FG 2004 (2004)Google Scholar
  15. 15.
    Zhang, L., Samaras, D.: Face Recognition Under Variable Lighting using Harmonic Image Exemplars. In: Proc. CVPR 2003, pp. I: 19–25 (2003)Google Scholar
  16. 16.
    Qing, L.Y., Shan, S.G., Gao, W.: Face Recognition under Generic Illumination based on Harmonic Relighting., International Journal of Pattern Recognition and Artificial Intelligence (2005) (to appear)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Xiujuan Chai
    • 1
  • Laiyun Qing
    • 2
  • Shiguang Shan
    • 2
  • Xilin Chen
    • 1
    • 2
  • Wen Gao
    • 1
    • 2
  1. 1.School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  2. 2.ICT-ISVISION Joint R&D Lab for Face RecognitionICT, CASBeijingChina

Personalised recommendations