An Open Source Framework for Standardized Comparisons of Face Recognition Algorithms

  • Manuel Günther
  • Roy Wallace
  • Sébastien Marcel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7585)


In this paper we introduce the facereclib, the first software library that allows to compare a variety of face recognition algorithms on most of the known facial image databases and that permits rapid prototyping of novel ideas and testing of meta-parameters of face recognition algorithms. The facereclib is built on the open source signal processing and machine learning library Bob. It uses well-specified face recognition protocols to ensure that results are comparable and reproducible. We show that the face recognition algorithms implemented in Bob as well as third party face recognition libraries can be used to run face recognition experiments within the framework of the facereclib. As a proof of concept, we execute four different state-of-the-art face recognition algorithms: local Gabor binary pattern histogram sequences (LGBPHS), Gabor graph comparisons with a Gabor phase based similarity measure, inter-session variability modeling (ISV) of DCT block features, and the linear discriminant analysis on two different color channels (LDA-IR) on two different databases: The Good, The Bad, and The Ugly, and the BANCA database, in all cases using their fixed protocols. The results show that there is not one face recognition algorithm that outperforms all others, but rather that the results are strongly dependent on the employed database.


Face Recognition Discrete Cosine Transform Gaussian Mixture Model Local Binary Pattern Equal Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Phillips, P., Rauss, P., Der, S.: FERET (face recognition technology) recognition algorithm development and test results. Technical report, Army Research Lab (1996)Google Scholar
  2. 2.
    Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., Zhao, D.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems, Man, and Cybernetics 38, 149–161 (2008)CrossRefGoogle Scholar
  3. 3.
    AT&T Laboratories Cambridge: AT&T database of faces (2004),
  4. 4.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, Amherst (2007)Google Scholar
  5. 5.
    Phillips, P.J., Beveridge, J.R., Draper, B.A., Givens, G.H., O’Toole, A.J., Bolme, D.S., Dunlop, J.P., Lui, Y.M., Sahibzada, H., Weimer, S.: An introduction to the good, the bad, & the ugly face recognition challenge problem. In: Ninth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 346–353 (2011)Google Scholar
  6. 6.
    Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Worek, W.: Preliminary face recognition grand challenge results. In: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 15–24 (2006)Google Scholar
  7. 7.
    Messer, K., Matas, J., Kittler, J., Luettin, J., Maître, G.: XM2VTSDB: the extended M2VTS database. In: Proceedings of the Second International Conference on Audio- and Video-Based Biometric Person Authentication (1999)Google Scholar
  8. 8.
    McCool, C., Marcel, S., Hadid, A., Pietikainen, M., Matejka, P., Cernocky, J., Poh, N., Kittler, J., Larcher, A., Levy, C., Matrouf, D., Bonastre, J.F., Tresadern, P., Cootes, T.: Bi-modal person recognition on a mobile phone: using mobile phone data. In: IEEE ICME Workshop on Hot Topics in Mobile Multimedia (2012)Google Scholar
  9. 9.
    Tan, X., Chen, S., Zhang, Z.: Face recognition from a single image per person: A survey. Pattern Recognition 39, 1725–1745 (2006)zbMATHCrossRefGoogle Scholar
  10. 10.
    Serrano, Á., de Diego, I.M., Conde, C., Cabello, E.: Recent advances in face biometrics with Gabor wavelets: A review. Pattern Recognition Letters 31, 372–381 (2010)CrossRefGoogle Scholar
  11. 11.
    Huang, D., Member, S., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local binary patterns and its application to facial image analysis: A survey. IEEE Transactions on Systems Man and Cybernetics Part C Applications and Reviews 41, 765–781 (2011)CrossRefGoogle Scholar
  12. 12.
    Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. Journal of Information Processing Systems 5, 41–68 (2009)CrossRefGoogle Scholar
  13. 13.
    Shen, L., Bai, L.: A review on Gabor wavelets for face recognition. Pattern Analysis and Applications 9, 273–292 (2006)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Beveridge, R., Bolme, D., Teixeira, M., Draper, B.: The CSU face identification evaluation system user’s guide version 5.0. Technical report, Colorado State University (2003)Google Scholar
  15. 15.
    Blackburn, D., Bone, M., Phillips, P.: Face recognition vendor test 2000: evaluation report. Technical report, National Institute of Standards and Technology (2001)Google Scholar
  16. 16.
    Phillips, P., Grother, P., Micheals, R., Blackburn, D., Tabassi, E., Bone, M.: Face recognition vendor test 2002: evaluation report. Technical report, National Institute of Standards and Technology (2003)Google Scholar
  17. 17.
    Phillips, P., Scruggs, T., O’Toole, A., Flynn, P., Bowyer, K., Schott, C., Sharpe, M.: FRVT 2006 and ICE 2006 large-scale results. Technical report, National Institute of Standards and Technology (2007)Google Scholar
  18. 18.
    Anjos, A., Shafey, L.E., Wallace, R., Günther, M., McCool, C., Marcel, S.: Bob: a free signal processing and machine learning toolbox for researchers. In: 20th ACM Conference on Multimedia Systems. ACM Press (2012)Google Scholar
  19. 19.
    Beveridge, R., Bolme, D.S.: CSU Face Recognition Resources (2011),
  20. 20.
    Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H.: Local Gabor binary pattern histogram sequence (LGBPHS): A novel non-statistical model for face representation and recognition. In: IEEE International Conference on Computer Vision, vol. 1, pp. 786–791 (2005)Google Scholar
  21. 21.
    Wiskott, L., Fellous, J.M., Krüger, N., Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 775–779 (1997)CrossRefGoogle Scholar
  22. 22.
    Günther, M., Haufe, D., Würtz, R.P.: Face Recognition with Disparity Corrected Gabor Phase Differences. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012, Part I. LNCS, vol. 7552, pp. 411–418. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  23. 23.
    Wallace, R., McLaren, M., McCool, C., Marcel, S.: Inter-session variability modelling and joint factor analysis for face authentication. In: International Joint Conference on Biometrics (2011)Google Scholar
  24. 24.
    Bailly-Baillière, E., Bengio, S., Bimbot, F., Hamouz, M., Kittler, J., Mariéthoz, J., Matas, J., Messer, K., Popovici, V., Porée, F., Ruiz, B., Thiran, J.P.: The BANCA Database and Evaluation Protocol. In: Kittler, J., Nixon, M.S. (eds.) AVBPA 2003. LNCS, vol. 2688, pp. 625–638. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  25. 25.
    Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing 19, 1635–1650 (2010)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face Recognition with Local Binary Patterns. In: Pajdla, T., Matas, J. (eds.) ECCV 2004, Part I. LNCS, vol. 3021, pp. 469–481. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  27. 27.
    Wallace, R., McLaren, M., McCool, C., Marcel, S.: Cross-pollination of normalisation techniques from speaker to face authentication using Gaussian mixture models. IEEE Transactions on Information Forensics and Security (2012)Google Scholar
  28. 28.
    Reynolds, D.A., Quatieri, T.F., Dunn, R.B.: Speaker verification using adapted Gaussian mixture models. Digital Signal Processing 10, 19–41 (2000)CrossRefGoogle Scholar
  29. 29.
    Grgic, M., Delac, K., Grgic, S.: SCface - surveillance cameras face database. Multimedia Tools and Applications 51, 863–879 (2011)CrossRefGoogle Scholar
  30. 30.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)CrossRefGoogle Scholar
  31. 31.
    Moghaddam, B., Wahid, W., Pentland, A.: Beyond eigenfaces: Probabilistic matching for face recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 30–35 (1998)Google Scholar
  32. 32.
    Günther, M., Würtz, R.P.: Face detection and recognition using maximum likelihood classifiers on Gabor graphs. International Journal of Pattern Recognition and Artificial Intelligence 23, 433–461 (2009)CrossRefGoogle Scholar
  33. 33.
    Prince, S.J.D.: Probabilistic linear discriminant analysis for inferences about identity. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Manuel Günther
    • 1
  • Roy Wallace
    • 1
  • Sébastien Marcel
    • 1
  1. 1.Idiap Research InstituteMartignySwitzerland

Personalised recommendations