Thermal Face Recognition in Unconstrained Environments Using Histograms of LBP Features

  • Javier Ruiz-del-Solar
  • Rodrigo Verschae
  • Gabriel Hermosilla
  • Mauricio Correa
Part of the Studies in Computational Intelligence book series (SCI, volume 506)


Several studies have shown that the use of thermal images can solve limitations of visible spectrum based face recognition methods operating in unconstrained environments. The recognition of faces in the thermal domain can be tackled using the histograms of Local Binary Pattern (LBP) features method. The aim of this work is to analyze the advantages and limitations of this method by means of a comparative study against other methods. The analyzed methods were selected by considering their performance in former comparative studies, in addition to being real-time—10 fps or more—to require just one image per person, and to being fully online (no requirements of offline enrollment). Thus, in the analysis the following local-matching based methods are considered: Gabor Jet Descriptors (GJD), Weber Linear Discriminant (WLD) and Local Binary Pattern (LBP). The methods are compared using the UCHThermalFace database. The use of this database allows evaluating the methods in real-world conditions that include natural variations in illumination, indoor/outdoor setup, facial expression, pose, accessories, occlusions, and background. In addition, the fusion of some variants of the methods was evaluated. The main conclusions of the comparative study are: (i) All analyzed methods perform very well under the conditions in which they were evaluated, except for the case of GJD that has low performance in outdoor setups; (ii) the best tradeoff between high recognition rate and fast processing speed is obtained by LBP-based methods; and (iii) fusing some methods or their variants improve the results up to 5 %.


Face recognition  Thermal face recognition Unconstrained environments Local binary pattern 



This research was partially funded by the FONDECYT-Chile grant 1090250, by the FONDECYT-Chile grant 3120218, and by the Advanced Mining Technology Center.


  1. 1.
    Abidi, B., Huq, S., Abidi, M.: Fusion of visual, thermal, and range as a solution to illumination and pose restrictions in face recognition. In: Proceedings of IEEE Carnahan Conference on Security Technology, pp. 325–330, Albuquerque, NM (2004)Google Scholar
  2. 2.
    Ahmad, J., Ali, U., Qureshi, R.J.: Fusion of thermal and visual images for efficient face recognition using gabor filter. In: The 4th ACS/IEEE International Conference on Computer Systems and Applications, pp. 135–139, March 8–11. Dubai/Sharjah, UAE (2006)Google Scholar
  3. 3.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face Description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)Google Scholar
  4. 4.
    Akhloufi, M., Bendada, A.: Thermal faceprint: a new thermal face signature extraction for infrared face recognition, In: CRV 2008: Fifth Canadian Conference on Computer and Robot Vision (Windsor, Ontario), pp. 269–272, 28–30 May 2008Google Scholar
  5. 5.
    Bebis, G., Gyaourova, A., Singh, S., Pavlidis, I.: Face recognition by fusing thermal infrared and visible imagery. Image Vis. Comput. 24(7), 727–742 (2006)Google Scholar
  6. 6.
    Bhowmik, M.K., Bhattacharjee, D., Nasipuri, M., Basu, D.K., Kundu, M.: Optimum fusion of visual and thermal face images for recognition. In: Sixth International Conference on Information Assurance and Security (IAS 2010), pp. 311–316. IEEE Intelligent Transportation Systems Society, Atlanta, USA, 23–25 August 2010Google Scholar
  7. 7.
    Buddharaju, P., Pavlidis, I.: Multi-spectral face recognition—fusion of visual imagery with physiological information. In: Hammoud, R.I., Abidi, B.R., Abidi, M.A. (eds.) Face Biometrics for Personal Identification: Multi-Sensory Multi-Modal Systems, pp. 91–108. Springer, Berlin (2007)Google Scholar
  8. 8.
    Buddharaju, P., Pavlidis, I., Manohar, C.: Face Recognition Beyond the Visible Spectrum, Advances in Biometrics: Sensors, Algorithms and Systems, pp. 157–180. Springer, Berlin (2007)Google Scholar
  9. 9.
    Buddharaju, P., Pavlidis, I., Kakadiaris, I.: Physiology-based face recognition. In: Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 354–359. Lake Como, Italy (2005)Google Scholar
  10. 10.
    Buddharaju, P., Pavlidis, I., Kakadiaris, I.: Pose-invariant physiological face recognition in the thermal infrared spectrum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 53–60. New York, USA (2006)Google Scholar
  11. 11.
    Chen, J., Shan, S., He, Ch., Zhao, G., Pietikäinen, M., Chen, ChX, Gao, W.: WLD: a robust local image descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1705–1720 (2010)Google Scholar
  12. 12.
    Chen, X., Flynn, P., Bowyer, K.W.: PCA-based face recognition in infrared imagery: baseline and comparative studies. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 127–134. Nice, France (2003)Google Scholar
  13. 13.
    Cho, S.Y., Wang, L., Ong, W.L.: Thermal imprint feature analysis for face recognition. In: IEEE International Symposium on Industrial Electronics 2009, ISIE 2009, pp. 1875–1880 (2009)Google Scholar
  14. 14.
    Desa, S., Hati, S.: IR and visible face recognition using fusion of kernel based features. In: The 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4. Tampa, Florida, USA, 8–11 December 2008Google Scholar
  15. 15.
    Equinox, 2011. Equinox Database. Accessed July 2011
  16. 16.
    He, M., Horng, S.-J., Fan, P., Run, R.-S., Chen, R.-J., Lai, J.-L., Khan, M., Sentosa, K.: Performance evaluation of score level fusion in multimodal biometric systems. Pattern Recognit. 43(5), 1789–1800 (2010)Google Scholar
  17. 17.
    Hermosilla, G.,Loncomilla, P., Ruiz-del-Solar, J.: Thermal Face Recognition using Local Interest Points and Descriptors for HRI Applications. Lecture Notes in Computer Science, vol. 6556 (RoboCup, Symposium 2010), pp. 25–35Google Scholar
  18. 18.
    Hermosilla, G., Ruiz-del-Solar, J., Verschae, R., Correa, M.: Face recognition using thermal infrared images for human-robot interaction applications: a comparative study. In: The 6th IEEE Latin American Robotics Symposium—LARS 2009, Valparaíso, Chile (CD Proceedings), 29–30 OctoberGoogle Scholar
  19. 19.
    Hermosilla, G., Ruiz-del-Solar, J., Verschae, R., Correa, M.: A comparative study of thermal face recognition methods in unconstrained environments. Pattern Recognit. 45(7), 2445–2459. ISSN 0031–3203 (July 2012). doi: 10.1016/j.patcog.2012.01.001
  20. 20.
    Kwon, O.K., Kong, S.G.: Multiscale fusion of visual and thermal images for robust face recognition. In: Proceedings of IEEE International Conference on Computer Intelligence for Homeland Security and Personal Safety, vol. IV, pp. 112–116, FL, March 2005Google Scholar
  21. 21.
    Lan, W.: Face recognition system based on spatial constellation model and support vector machine. Master’s Thesis, Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology (2010)Google Scholar
  22. 22.
    Lategahn, H., Gross, S., Stehle, T., Aach, T.: Texture classification by modeling joint distributions of local patterns with Gaussian mixtures. IEEE Trans. Image Processing 19(6), 1548–1557 (2010)Google Scholar
  23. 23.
    Lei, Z., Li, S.Z.: Fast multi-scale local phase quantization histogram for face recognition. Pattern Recognit. Lett. 33(13), 1761–1767. ISSN 0167–8655 (2012). doi: 10.1016/j.patrec.2012.06.005
  24. 24.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)Google Scholar
  25. 25.
    Mendez, H., San Martín, C., Kittler, J., Plasencia, Y., García, E.: Face recognition with LWIR imagery using local binary patterns. LNCS, vol. 5558, pp. 327–336 (2009)Google Scholar
  26. 26.
    Nanni, L., Lumini, A., Brahnam, S.: High performance set of features for biometric data. Int. J. Autom. Ident. Technol 2(1), 1–7 (2010)Google Scholar
  27. 27.
    Narendra, P.: Reference-free nonuniformity compensation for IR imaging arrays. Proc. SPIE 252, 10–17 (1980)Google Scholar
  28. 28.
    Narendra, P., Foss, N.: Shutterless fixed pattern noise correction for infrared imaging arrays. Proc. SPIE 282, 44–51 (1981)Google Scholar
  29. 29.
    Pop, F.M., Gordan, M., Florea, C., Vlaicu, A.: Fusion based approach for thermal and visible face recognition under pose and expressivity variation. In: Roedunet Int. Conf. (RoEduNet), pp. 61–66 (2010)Google Scholar
  30. 30.
    Ruiz-del-Solar, J., Navarrete, P.: Eigenspace-based face recognition: a comparative study of different approaches. IEEE Trans. Syst. Man Cybernet.Part C 35(3), 315–325 (2005)Google Scholar
  31. 31.
    Ruiz-del-Solar, J., Quinteros, J.: Illumination compensation and normalization in eigenspace-based face recognition: a comparative study of different pre-processing approaches. Pattern Recognit. Lett. 29(14), 1966–1979 (2008)Google Scholar
  32. 32.
    Ruiz-del-Solar, J., Verschae, R., Correa, M.: Recognition of faces in unconstrained environments: a comparative study. EURASIP Journal on Advances in Signal Processing, special issue, Recent Advances in Biometric Systems: A Signal Processing Perspective, vol. 2009, Article ID 184617, 19 pages (2009). doi: 10.1155/2009/184617
  33. 33.
    Selinger, A., Socolinsky, D.: Appearance-based facial recognition using visible and thermal imagery: a comparative study. Tech. Rep., Equinox Corporation (2001)Google Scholar
  34. 34.
    Socolinsky, D., Selinger, A.: A comparative analysis of face recognition performance with visible and thermal infrared imagery. In: Proceedings of the International Conference on Pattern Recognition (ICPR), Quebec, Canada (2002)Google Scholar
  35. 35.
    Socolinsky, D., Selinger, A.: Thermal face recognition over time. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04), vol. 4, pp. 187–190 (2004)Google Scholar
  36. 36.
    Socolinsky, D., Wolff, L., Neuheisel, J., Eveland, C.: Illumination invariant face recognition using thermal infrared imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2001)Google Scholar
  37. 37.
    Tan, X., Chen, S., Zhou, Z.H., Zhang, F.: Face recognition from a single image per person: a survey. Pattern Recognit. 39, 1725–1745 (2006)Google Scholar
  38. 38.
    Yoshitomi, Y., Sung-Ill K., Kawano, T., Kilazoe, T.: Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proceedings of 9th IEEE International Workshop on Robot and Human Interactive Communication, pp. 178–183 (2000)Google Scholar
  39. 39.
    Zou, J., Ji, Q., Nagy, G.: A comparative study of local matching approach for face recognition. IEEE Trans. Image Process. 16(10), 2617–2628 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Javier Ruiz-del-Solar
    • 1
    • 2
  • Rodrigo Verschae
    • 2
  • Gabriel Hermosilla
    • 3
  • Mauricio Correa
    • 2
  1. 1.Department of Electrical EngineeringUniversidad de ChileSantiagoChile
  2. 2.Advanced Mining Technology CenterUniversidad de ChileSantiagoChile
  3. 3.Escuela de Ingeniería EléctricaPontificia Universidad Católica de ValparaísoValparaisoChile

Personalised recommendations