Fusion of Infrared and Visible Images for Face Recognition

  • Aglika Gyaourova
  • George Bebis
  • Ioannis Pavlidis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3024)


A number of studies have demonstrated that infrared (IR) imagery offers a promising alternative to visible imagery due to it’s insensitive to variations in face appearance caused by illumination changes. IR, however, has other limitations including that it is opaque to glass. The emphasis in this study is on examining the sensitivity of IR imagery to facial occlusion caused by eyeglasses. Our experiments indicate that IR-based recognition performance degrades seriously when eyeglasses are present in the probe image but not in the gallery image and vice versa. To address this serious limitation of IR, we propose fusing the two modalities, exploiting the fact that visible-based recognition is less sensitive to the presence or absence of eyeglasses. Our fusion scheme is pixel-based, operates in the wavelet domain, and employs genetic algorithms (GAs) to decide how to combine IR with visible information. Although our fusion approach was not able to fully discount illumination effects present in the visible images, our experimental results show substantial improvements recognition performance overall, and it deserves further consideration.


Facial Expression Face Recognition Visible Image Face Image Recognition Performance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zhao, W., C.R.P.P., Rosenfeld, A.: Face recognition: A literature survey. ACM Computing Surveys 35, 399–458 (2003)CrossRefGoogle Scholar
  2. 2.
    Turk, M., Pentland, A.: Eigenfaces for recognition. CogNeuro 3, 71–96 (1991)Google Scholar
  3. 3.
    Wolff, L., Quantitative, S.D., C.E.: measurement of illumination invariance for face recognition using thermal infrared imagery. In: IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methodsand Applications, Hawaii (2001)Google Scholar
  4. 4.
    Jain, A., Bolle, R., Pankanti, S.: Biometrics: Personal Identification in Networked Society. Kluwer Academic Publishers, Dordrecht (1999)Google Scholar
  5. 5.
    Pavlidis, I., Symosek, P.: The imaging issue in an automatic face/disguise detection system. In: IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, pp. 15–24 (2000)Google Scholar
  6. 6.
    Goldberg, D.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  7. 7.
    Prokoski, F.: History, current status, and future of infrared identification. In: IEEE Workshop on Computer Vision Beyond the Visible Spectrum, Hilton Head (2000)Google Scholar
  8. 8.
    Wilder, J., Phillips, J., Jiang, C., Wiener, S.: Comparison of visible and infra-red imagery for face recognition. In: 2nd International Conference on Automatic Face and Gesture Recognition, Killington, pp. 182–187 (1996)Google Scholar
  9. 9.
    Selinger, A., Socolinsky, D.: Appearance-based facial recognition using visible and thermal imagery: A comparative study. Technical report, Equinox Corporation n.02–01 (2002)Google Scholar
  10. 10.
    Socolinsky, D.A., Selinger, A.: Comparative study of face recognition performance with visible and thermal infrared imagery. In: International Conference on Pattern Recognition, pp. 217–222 (2002)Google Scholar
  11. 11.
    Chen, X., Flynn, P., Bowyer, K.: Pca-based face recognition in infrared imagery: Baseline and comparative studies. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Nice, France (2003)Google Scholar
  12. 12.
    Dowdall, J., Pavlidis, I., Bebis, G.: Face detection in the near-ir spectrum. Image and Vision Computing 21, 565–578 (2001)CrossRefGoogle Scholar
  13. 13.
    Scheunders, P.: Local mapping for multispectral image visualization. Image and Vision Computing 19, 971–978 (2001)CrossRefGoogle Scholar
  14. 14.
    Manjunath, B., Chellappa, R., von der Malsburg, C.: A feature based approach to face recognition. In: Computer Vision and Pattern Recognition, pp. 373–378 (1992)Google Scholar
  15. 15.
    Chui, C.: An introduction to wavelets. Academic Press, London (1992)zbMATHGoogle Scholar
  16. 16.
    Katz, A., Thrift, P.: Generating image filters for target recognition by genetic learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 16 (1994)Google Scholar
  17. 17.
    Bebis, G., Louis, S., Varol, Y., Yfantis, A.: Genetic object recognition using combinations of views. IEEE Transactions on Evolutionary Computing 6, 132–146 (2002)CrossRefGoogle Scholar
  18. 18.
    Swets, D., Punch, B.: Genetic algorithms for object localization in a complex scene. In: IEEE International Conference on Image Processing, pp. 595–598 (1995)Google Scholar
  19. 19.
    Bebis, G., Uthiram, S., Georgiopoulos, M.: Face detection and verification using genetic search. International Journal of Artificial Intelligence Tools 9, 225–246 (2000)CrossRefGoogle Scholar
  20. 20.
    Sun, Z., Yuan, X., Bebis, G., Louis, S.: Genetic feature subset selection for gender classification: A comparison study. In: IEEE Workshop on Applications of Computer Vision (2002)Google Scholar
  21. 21.
    Sun, Z., Bebis, G., Miller, R.: Boosting object detection using feature selection. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (2003)Google Scholar
  22. 22.
    Chipman, L., Orr, T.: Wavelets and image fusion. In: IEEE International Conference on Image Processing, vol. 3, pp. 248–251 (1995)Google Scholar
  23. 23.
    Li, H., Manjunath, B., Mitra, S.: Multisensor image fusion using the wavelet transform. In: IEEE International Conference on Image Processing, Austin, Texas, vol. 1, pp. 51–55 (1994)Google Scholar
  24. 24.
    Fogel, D.: Evolutionary Computation, Toward a New Philosophy of Machine Intelligence. IEEE Press, Los Alamitos (1992)Google Scholar
  25. 25.
    Koza, J.: Genetic Programming. MIT Press, Cambridge (1993)Google Scholar
  26. 26.
    Cantu-Paz, E.: Efficient and Accurate Parallel Genetic Algorithms and Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2000)Google Scholar
  27. 27.
  28. 28.
    Jin, Y., Sendhoff, B.: Fitness approximation in evolutionary computation – a survey. In: Genetic and Evolutionary Computation Conference (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Aglika Gyaourova
    • 1
  • George Bebis
    • 1
  • Ioannis Pavlidis
    • 2
  1. 1.Computer Vision LaboratoryUniversity of NevadaReno
  2. 2.Visual Computing LaboratoryUniversity of Houston 

Personalised recommendations