Using Score Fusion for Improving the Performance of Multispectral Face Recognition

  • Yufeng Zheng
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 292)


Score fusion combines several scores from multiple modalities and/or multiple matchers, which can increase the accuracy of face recognition meanwhile decrease false accept rate (FAR). Specifically, the face scores are generated from two-spectral bands (visible and thermal) and from three matchers (circular Gaussian filter, face pattern byte, elastic bunch graphic matching). In this chapter, we first review the three face recognition algorithms (matchers), then present and compare the fusion performance of seven fusion methods: linear discriminant analysis (LDA), k-nearest neighbor (KNN), artificial neural network (ANN), support vector machine (SVM), binomial logistic regression (BLR), Gaussian mixture model (GMM), and hidden Markov model (HMM). Our experiments are conducted with the Alcon State University Multispectral face dataset that currently consists of two spectral images from 105 subjects. The experimental results show that all score fusions can improve the accuracy meanwhile reduce the FAR, and the KNN score fusion gives the best performance.



The current research is funded by the Department of Defense Research and Education. The thermal face recognition research was previously supported by the Department of Homeland Security.


  1. 1.
    Chang H, Koschan A, Abidi M, Kong SG, Won C-H (2008) Multispectral visible and infrared imaging for face recognition. In: IEEE computer society conference on computer vision and pattern recognition workshopsGoogle Scholar
  2. 2.
    Bendada A, Akhloufi MA (2010) Multispectral face recognition in texture space. In: Canadian conference on computer and robot vision, pp 101–106Google Scholar
  3. 3.
    Bebis G, Gyaourova A, Singh S, Pavlidis I (2006) Face recognition by fusing thermal infrared and visible imagery. Image Vis Comput 24:727–742CrossRefGoogle Scholar
  4. 4.
    Arandjelovic O, Hammoud RI, Cipolla R (2006) On person authentication by fusing visual and thermal face biometrics. In: Proceedings of the IEEE international conference on video and signal based surveillanceGoogle Scholar
  5. 5.
    Nandakumar K, Chen Y, Dass SC, Jain AK (2008) Likelihood ratio-based biometric score fusion. IEEE Trans Pattern Anal Mach Intell 30(2):342–347CrossRefGoogle Scholar
  6. 6.
    Poh N, Bourlai T et al (2009) Benchmarking quality-dependent and cost-sensitive multimodal biometric fusion algorithms. IEEE Trans Inf Forensics Secur 4(4), 849–866Google Scholar
  7. 7.
    Zheng Y, Elmaghraby A (2011) A brief survey on multispectral face recognition and multimodal score fusion. In: IEEE international symposium on signal processing and information and technology (ISSPIT), pp 543–550Google Scholar
  8. 8.
    Zheng Y (2011) Orientation-based face recognition using multispectral imagery and score fusion. Opt Eng 50:117202CrossRefGoogle Scholar
  9. 9.
    Hill DLG, Batchelor P (2001) Registration methodology: concepts and algorithms. In: Hajnal JV, Hill DLG, Hawkes DJ (eds) Medical image registration. CRC, Boca RatonGoogle Scholar
  10. 10.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc CVPR 1:511–518Google Scholar
  11. 11.
    Viola P, Jones M (2001) Robust real-time object detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  12. 12.
    Lienhart R, Maydt J (2002) An extended set of Haar-like features for rapid object detection. Proc Int Conf Image Process 1:900–903CrossRefGoogle Scholar
  13. 13.
    Freund Y, Schapire R (1995) A decision theoretic generalization of on-line learning and an application to boosting. Computational Learning Theory: Eurocolt ’95, pp 23–37Google Scholar
  14. 14.
    Zhao W, Chellappa R, Krishnaswamy A (1998) Discriminant analysis of principal components for face recognition. In: Proceedings of IEEE international conference on face and gesture recognition, FG’98, 14–16, pp 336–341Google Scholar
  15. 15.
    Zheng Y (2012) Face detection and eyeglasses detection for thermal face recognition. In: Proceedings of SPIE 8300Google Scholar
  16. 16.
    Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 14(1):195–200CrossRefGoogle Scholar
  17. 17.
    Wiskott L, Fellous JM, Krüger N, von der Malsburg C (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19(7):775–779CrossRefGoogle Scholar
  18. 18.
    Wiskott L, Fellous J-M, Krüger N, von der Malsburg C (1999) Face recognition by elastic bunch graph matching. In: Jain LC et al (eds) Intelligent biometric techniques in fingerprint and face recognition. CRC Press, Boca Raton, pp 355–396Google Scholar
  19. 19.
    Zhao W, Chellappa R, Rosenfeld A, Phillips PJ (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458CrossRefGoogle Scholar
  20. 20.
    Kuncheva LI (2002) A theoretical study on six classifier fusion strategies. IEEE Trans Pattern Anal Mach Intell 24(2):281–286CrossRefGoogle Scholar
  21. 21.
    Brunelli R, Falavigna D (1995) Person identification using multiple cues. IEEE Trans Pattern Anal Mach Intell 17(10):955–966CrossRefGoogle Scholar
  22. 22.
    Fierrez-Aguilar J, Ortega-Garcia J, Gonzalez-Rodriguez J, Bigun J (2005) Discriminative multimodal biometric authentication based on quality measures. Pattern Recogn 38(5):777–779CrossRefGoogle Scholar
  23. 23.
    Duda RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New YorkzbMATHGoogle Scholar
  24. 24.
    McCullagh P, Nelder JA (1990) Generalized linear models. Chapman & Hall, New YorkGoogle Scholar
  25. 25.
    Kil RM, Koo I (2001) Optimization of a network with Gaussian kernel functions based on the estimation of error confidence intervals. Proc IJCNN 3:1762–1766Google Scholar
  26. 26.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Mining Knowl Discov 2:121–167CrossRefGoogle Scholar
  27. 27.
    Prabhakar S, Jain AK (2002) Decision-level fusion in fingerprint verification. Pattern Recogn 35(4):861–874CrossRefzbMATHGoogle Scholar
  28. 28.
    Ulery B, Hicklin AR, Watson C, Fellner W, Hallinan P (2006) Studies of biometric fusion. NIST Interagency ReportGoogle Scholar
  29. 29.
    McLachla G, Peel D (2000) Finite mixture models. Wiley, HobokenCrossRefGoogle Scholar
  30. 30.
    Figueiredo M, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396CrossRefGoogle Scholar
  31. 31.
    Baum LE, Petrie T (1966) Statistical inference for probability functions of finite state markov chains. Ann Math Stat 37:1554–1563CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Kahler B, Blasch E (2011) Decision-level fusion performance improvement from enhanced HRR radar clutter suppression. J Adv Inf Fusion 6 (2):101–118Google Scholar
  33. 33.
    Reese K, Zheng Y, Elmaghraby A (2012) A comparison of face detection algorithms in visible and thermal spectrums. In: International conference on advances in computer science and applicationGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Alcorn State UniversityLormanUSA

Personalised recommendations