Face Matching Between Near Infrared and Visible Light Images

  • Dong Yi
  • Rong Liu
  • RuFeng Chu
  • Zhen Lei
  • Stan Z. Li
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4642)


In many applications, such as E-Passport and driver’s license, the enrollment of face templates is done using visible light (VIS) face images. Such images are normally acquired in controlled environment where the lighting is approximately frontal. However, Authentication is done in variable lighting conditions. Matching of faces in VIS images taken in different lighting conditions is still a big challenge. A recent development in near infrared (NIR) image based face recognition [1] has well overcome the difficulty arising from lighting changes. However, it requires that enrollment face images be acquired using NIR as well.

In this paper, we present a new problem, that of matching a face in an NIR image against one in a VIS images, and propose a solution to it. The work is aimed to develop a new solution for meeting the accuracy requirement of face-based biometric recognition, by taking advantages of the recent NIR face technology while allowing the use of existing VIS face photos as gallery templates. Face recognition is done by matching an NIR probe face against a VIS gallery face. Based on an analysis of properties of NIR and VIS face images, we propose a learning-based approach for the different modality matching. A mechanism of correlation between NIR and VIS faces is learned from NIR→VIS face pairs, and the learned correlation is used to evaluate similarity between an NIR face and a VIS face. We provide preliminary results of NIR→VIS face matching for recognition under different illumination conditions. The results demonstrate advantages of NIR→VIS matching over VIS→VIS matching.


Face Recognition Near Infrared (NIR) Visible Light (VIS) Images Dimension Reduction Canonical Correlation Analysis (CCA) 


  1. 1.
    Li, S.Z., Chu, R., Liao, S., Zhang, L.: Illumination invariant face recognition using near-infrared images. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (Special issue on Biometrics: Progress and Directions) (2007)Google Scholar
  2. 2.
    Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6), 643–660 (2001)CrossRefGoogle Scholar
  3. 3.
    Gross, R., Brajovic, V.: An image preprocessing algorithm for illumination invariant face recognition. In: Kittler, J., Nixon, M.S. (eds.) AVBPA 2003. LNCS, vol. 2688, pp. 10–18. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Nayar, S.K., Bolle, R.M.: Reflectance based object recognition. International Journal of Computer Vision 17(3), 219–240 (1996)CrossRefGoogle Scholar
  5. 5.
    Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 721–732 (1997)CrossRefGoogle Scholar
  6. 6.
    Li, S.Z.: and His Face Team: AuthenMetric F1: A Highly Accurate and Fast Face Recognition System. In: ICCV 2005 - Demos (2005)Google Scholar
  7. 7.
    ISO/IEC JTC 1/SC 37: Proposed Draft Amendment to ISO/IEC 19794-5 Face Image Data on Conditions for Taking Pictures. ISO/IEC 19794-5:2005/PDAM 1 (2006)Google Scholar
  8. 8.
    Hotelling, H.: Relations between two sets of variates. Biometrika 28, 321–377 (1936)zbMATHGoogle Scholar
  9. 9.
    Fukunaga, K.: Introduction to statistical pattern recognition, 2nd edn. Academic Press, Boston (1990)zbMATHGoogle Scholar
  10. 10.
    Sun, Q., Heng, P., Zhong, J., Xia, D., Huang, D., Zhang, X., Huang, G.: Face recognition based on generalized canonical correlation analysis. In: Proceedings of International Conference on Intelligent Computing, Hefei, China, Springer, Heidelberg (2005)Google Scholar
  11. 11.
    He, Y., Zhao, L., Zou, C., Lipo, W., Ke, C., Soon, O.Y.: Face recognition based on pca/kpca plus cca. In: Proceedings of International Conference on Advances in Natural Computation, Changsha, China, Springer, Heidelberg (2005)Google Scholar
  12. 12.
    Reiter, M., Donner, R., Georg, L., Horst, B.: 3D and infrared face reconstruction from RGB data using canonical correlation analysis. In: Proceedings of International Conference on Pattern Recognition (2006)Google Scholar
  13. 13.
    Reiter, M., Donner, R., Georg, L., Horst, B.: Predicting near infrared face texture from color face images using canonical correlation analysis. In: Proceedings of the Workshop of the Austrian Association for Pattern Recognition (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Dong Yi
    • 1
  • Rong Liu
    • 1
  • RuFeng Chu
    • 1
  • Zhen Lei
    • 1
  • Stan Z. Li
    • 1
  1. 1.Center for Biometrics Security Research & National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100080China

Personalised recommendations