Skip to main content
SpringerLink
Log in
Book cover

Face Recognition Across the Imaging Spectrum pp 247–280Cite as

Face Recognition in Challenging Environments: An Experimental and Reproducible Research Survey

  • Chapter
  • First Online:

Abstract

One important type of biometric authentication is face recognition , a research area of high popularity with a wide spectrum of approaches that have been proposed in the last few decades. The majority of existing approaches are conceived for or evaluated on constrained still images. However, more recently research interests have shifted toward unconstrained “in-the-wild ” still images and videos. To some extent, current state-of-the-art systems are able to cope with variability due to pose, illumination, expression, and size, which represent the challenges in unconstrained face recognition. To date, only few attempts have addressed the problem of face recognition in mobile environment , where high degradation is present during both data acquisition and transmission. This book chapter deals with face recognition in mobile and other challenging environments, where both still images and video sequences are examined. We provide an experimental study of one commercial off-the-shelf (COTS) and four recent open-source face recognition algorithms , including color-based linear discriminant analysis (LDA) , local Gabor binary pattern histogram sequences (LGBPHSs) , Gabor grid graphs , and intersession variability (ISV) modeling . Experiments are performed on several freely available challenging still image and video face databases, including one mobile database, always following the evaluation protocols that are attached to the databases. Finally, we supply an easily extensible open-source toolbox to rerun all the experiments, which includes the modeling techniques, the evaluation protocols, and the metrics used in the experiments and provides a detailed description on how to regenerate the results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://pypi.python.org/pypi/xfacereclib.book.FRaES2016.

  2. 2.

    http://www.idiap.ch/software/bob.

  3. 3.

    http://www.cs.colostate.edu/facerec/algorithms/baselines2011.php.

  4. 4.

    http://pypi.python.org/pypi/facereclib.

  5. 5.

    http://www.reproducibleresearch.net.

  6. 6.

    For example, the results on LFW [16] are published under: http://vis-www.cs.umass.edu/lfw/results.html.

  7. 7.

    One example for reproducible research based on the FaceRecLib can be found under: http://pypi.python.org/pypi/xfacereclib.paper.BeFIT2012.

  8. 8.

    http://www.beat-eu.org/platform.

  9. 9.

    To avoid misunderstandings, we do not use the name CohortLDA as in [22], but we stick to the old name of the algorithm (LDA-IR).

  10. 10.

    The COTS vendor requested to stay anonymous.

  11. 11.

    http://github.com/idiap/bob/wiki/Packages.

  12. 12.

    The website http://www2.ece.ohio-state.edu/˜aleix/ARdatabase.html re-ports more than 4000 images, but we could not reach the controller of the database to clarify the difference.

  13. 13.

    http://lear.inrialpes.fr/people/guillaumin/data.php.

  14. 14.

    To be comparable to the occlusion and both protocols, the same training set, i.e., including occluded faces, was also used in the illumination protocol.

  15. 15.

    We just use the pickle module of Python to store the LDA-IR data. Table 11.2(b) shows the resulting file size on disk.

  16. 16.

    http://www.idiap.ch/resource/biometric.

References

  1. Kanade, T.: Picture Processing System by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University, Japan (1973)

    Google Scholar 

  2. Sirovich, L., Kirby, M.: Low-dimensional procedure for the characterization of human faces. J. Opt. Soc. Am. A 4(3), 519–524 (1987)

    Article  Google Scholar 

  3. Rowley, H.A., Baluja, S., Kanade, T.: Rotation invariant neural network-based face detection. In: CVPR, pp. 38–44, Springer (1998)

    Google Scholar 

  4. Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vision 57(2), 137–154 (2002)

    Article  Google Scholar 

  5. Heusch, G., Rodriguez, Y., Marcel, S.: Local binary patterns as an image preprocessing for face authentication. In: FG, pp. 9–14. IEEE Computer Society (2006)

    Google Scholar 

  6. Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. Trans. Image Process. 19(6), 1635–1650 (2010)

    Article  MathSciNet  Google Scholar 

  7. Sanderson, C., Paliwal, K.K.: Fast features for face authentication under illumination direction changes. Pattern Recogn. Lett. 24(14), 2409–2419 (2003)

    Article  Google Scholar 

  8. Günther, M., Haufe, D., Würtz, R.P.: Face recognition with disparity corrected Gabor phase differences. In ICANN, volume 7552 of LNCS, pp. 411–418. Springer, Berlin (2012)

    Google Scholar 

  9. Zhang, B., Shan, S., Chen, X., Gao, W.: Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. Trans. Image Process. 16(1), 57–68 (2007)

    Article  MathSciNet  Google Scholar 

  10. Zhao, W., Krishnaswamy, A., Chellappa, R., Swets, D.L., Weng, J.: Discriminant analysis of principal components for face recognition. In: Face Recognition: From Theory to Applications, pp. 73–85. Springer, Berlin (1998)

    Google Scholar 

  11. Gao, W., Cao, B., Shan, S., Zhou, D., Zhang, X., Zhao, D.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. In: Technical report, Joint Research & Development Laboratory for Face Recognition, Chinese Academy of Sciences (2004)

    Google Scholar 

  12. Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H.: Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: ICCV, vol. 1, pp. 786–791. IEEE Computer Society (2005)

    Google Scholar 

  13. Wallace, R., McLaren, M., McCool, C., Marcel, S.: Cross-pollination of normalization techniques from speaker to face authentication using Gaussian mixture models. Trans. Inf. Forensics Secur. 7(2), 553–562 (2012)

    Article  Google Scholar 

  14. El Shafey, L., McCool, C., Wallace, R., Marcel, S.: A scalable formulation of probabilistic linear discriminant analysis: Applied to face recognition. Trans. Pattern Anal. Mach. Intell. 35(7), 1788–1794 (2013)

    Article  Google Scholar 

  15. McCool, C. et al.: Bi-modal person recognition on a mobile phone: using mobile phone data. In: ICME Workshop on Hot Topics in Mobile Multimedia, pp. 635–640. IEEE Computer Society (2012)

    Google Scholar 

  16. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.G.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Technical report, University of Massachusetts, Amherst (2007)

    Google Scholar 

  17. Wolf, L., Hassner, T., Maoz. I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2011

    Google Scholar 

  18. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  19. Martínez, A.M., Benavente, R.: The AR face database. In: Technical Report 24, Computer Vision Center (1998)

    Google Scholar 

  20. Anjos, A., El Shafey, L., Wallace, R., Günther, M., McCool, C., Marcel, S.: Bob: a free signal processing and machine learning toolbox for researchers. In: ACM-MM, pp. 1449–1452. ACM press (2012)

    Google Scholar 

  21. Phillips, P.J., Beveridge, J.R., Draper, B.A., Givens, G., O’Toole, A.J., Bolme, D.S., Dunlop, J., Lui, Y.M., Sahibzada, H., Weimer, S.: An introduction to the good, the bad, and the ugly face recognition challenge problem. In: FG, pp. 346–353. IEEE Computer Society (2011)

    Google Scholar 

  22. Lui, Y.M., Bolme, D.S., Phillips, P.J., Beveridge, J.R., Draper, B.A.: Preliminary studies on the good, the bad, and the ugly face recognition challenge problem. In: CVPR Workshops, pp. 9–16. IEEE Computer Society (2012)

    Google Scholar 

  23. Günther, M., Wallace, R., Marcel, S.: An open source framework for standardized comparisons of face recognition algorithms. In: ECCV. Workshops and Demonstrations, volume 7585 of LNCS, pp. 547–556. Springer, Berlin (2012)

    Google Scholar 

  24. Tan, X., Chen, S., Zhou, Z.-H., Zhang, F.: Face recognition from a single image per person: a survey. Pattern Recogn. 39, 1725–1745 (2006)

    Article  MATH  Google Scholar 

  25. Serrano, Á, Martín de Diego, I., Conde, C., Cabello, E.: Recent advances in face biometrics with Gabor wavelets: a review. Pattern Recogn. Lett. 31(5), 372–381 (2010)

    Google Scholar 

  26. Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local binary patterns and its application to facial image analysis: a survey. Syst. Man Cybern. Part C: Appl. Rev. 41(6), 765–781 (2011)

    Article  Google Scholar 

  27. Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. J. Inf. Process. Syst. 5(2), 41–68 (2009)

    Article  Google Scholar 

  28. Shen, L., Bai, L.: A review on Gabor wavelets for face recognition. Pattern Anal. Appl. 9(2), 273–292 (2006)

    Article  MathSciNet  Google Scholar 

  29. Vandewalle, P., Kovacevic, J., Vetterli, M.: Reproducible research in signal processing—what, why, and how. IEEE Signal Process. Mag. 26(3), 37–47 (2009)

    Google Scholar 

  30. Vandewalle, P.: Code sharing is associated with research impact in image processing. Comput. Sci. Eng. 14(4), 42–47 (2012)

    Article  Google Scholar 

  31. Ko, K.: User’s guide to NIST biometric image software (NBIS). In: Technical report, NIST Interagency/Internal Report (NISTIR)—7392 (2007)

    Google Scholar 

  32. Klontz, J.C., Klare, B.F., Klum, S., Jain, A.K., Burge, M.J.: Open source biometric recognition. In: IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8 (2013)

    Google Scholar 

  33. Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTSDB: the extended M2VTS database. In: AVBPA, pp. 72–77. LNCS (1999)

    Google Scholar 

  34. Martin, A., Przybocki, M., Campbell, J.P.: The NIST Speaker Recognition Evaluation Program, chapter 8. Springer, Berlin (2005)

    Google Scholar 

  35. Günther, M. et al.: The 2013 face recognition evaluation in mobile environment. In: The 6th IAPR International Conference on Biometrics (2013)

    Google Scholar 

  36. Khoury, E. et al.: The 2013 speaker recognition evaluation in mobile environment. In: The 6th IAPR International Conference on Biometrics (2013)

    Google Scholar 

  37. Bansé, A.D., Doddington, G.R., Garcia-Romero, D., Godfrey, J.J., Greenberg, C.S., McCree, A.F.M., Przybocki, M., Reynolds, D.A.: Summary and initial results of the 2013–2014 speaker recognition i-vector machine learning challenge. In: Fifteenth Annual Conference of the International Speech Communication Association (2014)

    Google Scholar 

  38. O’Toole, A.J., Phillips, P.J., Jiang, F., Ayyad, J., Penard, N., Abdi, H.: Face recognition algorithms surpass humans matching faces over changes in illumination. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1642–1646 (2007)

    Article  Google Scholar 

  39. Burton, A.M., Wilson, S., Cowan, M., Bruce, V.: Face recognition in poor-quality video: Evidence from security surveillance. Psychol. Sci. 10(3), 243248 (1999)

    Article  Google Scholar 

  40. Grgic, M., Delac, K., Grgic, S.: SCface–surveillance cameras face database. Multimedia Tools Appl. 51(3), 863–879 (2011)

    Article  Google Scholar 

  41. Beveridge, J.R., Phillips, P.J., Bolme, D.S., Draper, B.A., Givens, G.H., Lui, Y.M., Teli, M.N., Zhang, H., Scruggs, W.T., Bowyer, K.W., Flynn, P.J., Cheng, S.: The challenge of face recognition from digital point-and-shoot cameras. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8 (2013)

    Google Scholar 

  42. Taigman, Y., Yang, M., Ranzato, M.’A., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2014)

    Google Scholar 

  43. Y. Sun, X. Wang, and X. Tang. Deeply learned face representations are sparse, selective, and robust. CoRR (2014)

    Google Scholar 

  44. Lu, C., Tang, X.: Learning the face prior for Bayesian face recognition. In: Computer Vision ECCV 2014, volume 8692 of Lecture Notes in Computer Science. Springer International Publishing, Switzerland (2014)

    Google Scholar 

  45. Moghaddam, B., Wahid, W., Pentland, A.: Beyond eigenfaces: probabilistic matching for face recognition. In: FG, pp. 30–35. IEEE Computer Society (1998)

    Google Scholar 

  46. Beveridge, J.R., Zhang, H., Flynn, P.J., Lee, Y., Liong, V.E., Lu, J., de Assis Angeloni, M., de Freitas Pereira, T., Li, H., Hua G., Struc, V., Krizaj, J., Phillips, P.J.: The IJCB 2014 PaSC video face and person recognition competition. In: IEEE International Joint Conference on Biometrics IJCB, pp. 1–8 (2014)

    Google Scholar 

  47. Li, H., Hua, G., Shen, X., Lin, Z., Brandt, J.: Eigen-PEP for video face recognition. In: Asian Conference on Computer Vision (ACCV) (2014)

    Google Scholar 

  48. Cox, D., Pinto, N.: Beyond simple features: a large-scale feature search approach to unconstrained face recognition. In: Automatic Face Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pp. 8–15, Mar 2011

    Google Scholar 

  49. Ruiz-del Solar, J., Verschae, R., Correa, M.: Recognition of faces in unconstrained environments: A comparative study. EURASIP J. Adv. Signal Process. 2009(1), 2009

    Google Scholar 

  50. McCool, C., Wallace, R., McLaren, M., El Shafey, L., Marcel, S.: Session variability modeling for face authentication. IET Biometrics 2(3), 117–129 (2013)

    Article  Google Scholar 

  51. Khoury, E., Günther, M., El Shafey, L., Marcel, S.: On the improvements of uni-modal and bi-modal fusions of speaker and face recognition for mobile biometrics. In: Biometric Technologies in Forensic Science, Oct 2013

    Google Scholar 

  52. C. Atanasoaei. Multivariate Boosting with Look-up Tables for Face Processing. PhD thesis, EPFL, 2012

    Google Scholar 

  53. Uřičář, M., Franc, V., Hlaváč, V.: Detector of facial landmarks learned by the structured output SVM. In: Csurka, G., Braz, J. (eds.) VISAPP ’12: Proceedings of the 7th International Conference on Computer Vision Theory and Applications, vol. 1, pp. 547–556. SciTePress (2012)

    Google Scholar 

  54. K. Ram´ırez-Guti´errez, D. Cruz-P´erez, and H. P´erez-Meana. Face recognition and verification using histogram equalization. In ACS, WSEAS, 85–89 (2010)

    Google Scholar 

  55. H.Wang, S. Z. Li, and Y.Wang. Face recognition under varying lighting conditions using self quotient image. In FG. IEEE Computer Society, 819–824, (2004)

    Google Scholar 

  56. L. Wiskott, J.-M. Fellous, N. Kr¨uger, and C. van der Malsburg. Face recognition by elastic bunch graph matching. Transactions on Pattern Analysis and Machine Intelligence, 19, 775–779 (1997)

    Google Scholar 

  57. M. G¨unther. Statistical Gabor Graph Based Techniques for the Detection, Recognition, Classification, and Visualization of Human Faces. PhD thesis, Institut f¨ur Neuroinformatik, Technische Universit¨at Ilmenau, Germany (2011)

    Google Scholar 

  58. González Jiménez, D., Bicego, M., Tangelder, J.W.H., Schouten, B.A.M., Ambekar, O.O., Alba-Castro, J., Grosso, E., Tistarelli, M.: Distance measures for Gabor jets-based face authentication: a comparative evaluation. In: ICB, pp. 474–483. Springer (2007)

    Google Scholar 

  59. W. Zhang, S. Shan, L. Qing, X. Chen, and W. Gao. Are Gabor phases really useless for face recognition? Pattern Analysis & Applications, 12, 301–307 (2009)

    Google Scholar 

  60. T. Ojala, M. Pietik¨ainen, and D. Harwood. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51–59 (1996)

    Google Scholar 

  61. T. Ojala, M. Pietik¨ainen, and T. M¨aenp¨a¨a. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Transactions on Pattern Analysis and Machine Intelligence, 24(7):971–987 2002

    Google Scholar 

  62. T. Ahonen, A. Hadid, and M. Pietikainen. Face recognition with local binary patterns. In ECCV. Springer, 469–481 (2004)

    Google Scholar 

  63. F. Cardinaux, C. Sanderson, and S. Marcel. Comparison of MLP and GMM classifiers for face verification on XM2VTS. In AVBPA, volume 2688 of LNCS, 911–920. Springer, (2003)

    Google Scholar 

  64. D. A. Reynolds, T. F. Quatieri, and R. B. Dunn. Speaker verification using adapted Gaussian mixture models. Digital Signal Processing, 10(1-3):19–41 (2000)

    Google Scholar 

  65. J.-L. Gauvain and C.-H. Lee. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains. Transactions on Speech and Audio Processing, 2(2):291–298 (1994)

    Google Scholar 

  66. F. Cardinaux, C. Sanderson, and S. Bengio. User authentication via adapted statistical models of face images. Transactions on Signal Processing, 54(1):361–373 (2006)

    Google Scholar 

  67. R. J. Vogt and S. Sridharan. Explicit modelling of session variability for speaker verification. Computer Speech & Language, 22(1):17–38 (2008)

    Google Scholar 

  68. Wallace, R., McLaren, M., McCool, C., Marcel, S.: Inter-session variability modeling and joint factor analysis for face authentication. In: IJCB, pp. 1–8. IEEE (2011)

    Google Scholar 

  69. A. K. Jain, P. Flynn, and A. A. Ross. Handbook of Biometrics. Springer, 2008

    Google Scholar 

  70. E. Bailly-Bailli´ere et al. The BANCA database and evaluation protocol. In AVBPA, volume 2688 of LNCS, SPIE, 625–638 (2003)

    Google Scholar 

  71. G. B. Huang, V. Jain, and E. G. Learned-Miller. Unsupervised joint alignment of complex images. In ICCV, IEEE, 1–8 (2007)

    Google Scholar 

  72. M. Guillaumin, J. Verbeek, and C. Schmid. Is that you? Metric learning approaches for face identification. In ICCV, IEEE, 498–505 (2009)

    Google Scholar 

  73. Khoury, E., El Shafey, L., McCool, C., Günther, M., Marcel, S.: Bi-modal biometric authentication on mobile phones in challenging conditions. In: Image and Vision Computing (2013)

    Google Scholar 

  74. Ocegueda, O., Shah, S.K., Kakadiaris, I.A.: VWhich parts of the face give out your identity? In: CVPR, pp. 641–648. IEEE Computer Society (2011)

    Google Scholar 

  75. Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., Zhao, D.: The CAS-PEAL large- scale Chinese face database and baseline evaluations. Syst. Man Cybern. Part A Syst. Hum. 38, 149–161 (2008)

    Article  Google Scholar 

  76. Arashloo, S.R., Kittler, J.: Class-specific kernel fusion of multiple descriptors for face verification using multiscale binarised statistical image features. IEEE Trans. Inf. Forensics Secur. 9(12), 2100–2109 (2014)

    Article  Google Scholar 

  77. Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: IEEE International Conference on Computer Vision (ICCV), Oct 2009

    Google Scholar 

  78. Chen, D., Cao, X., Wang, L., Wen, F., Sun, J.: Bayesian face revisited: A joint formulation. In: Proceedings of the 12th European Conference on Computer Vision—Volume Part III, pp. 566–579 (2012)

    Google Scholar 

  79. Khoury, E., Senac, C., Joly, P.: Face-and-clothing based people clustering in video content. In: Proceedings of the International Conference on Multimedia Information Retrieval, MIR ’10, pp. 295–304. ACM, New York, NY, USA, (2010)

    Google Scholar 

  80. A. Dutta, M. Günther, L. El Shafey, S. Marcel, R. Veldhuis, and L. Spreeuwers. Impact of eye detection error on face recognition performance. IET Biometrics, 2014

    Google Scholar 

  81. Auckenthaler, R., Carey, M., Lloyd-Thomas, H.: Score normalization for text-independent speaker verification systems. Digit. Signal Proc. 10(1), 42–54 (2000)

    Article  Google Scholar 

  82. Barr, J.R., Bowyer, K.W., Flynn, P.J., Biswas, S.: Face recognition from video: a review. Int. J. Pattern Recogn. Artif. Intell. 26(5) (2012)

    Google Scholar 

  83. Müller, M.K., Tremer, M., Bodenstein, C., Würtz, R.P.: Learning invariant face recognition from examples. Neural Netw. 41:137–146 (2013)

    Google Scholar 

Download references

Acknowledgements

This evaluation has received funding from the European Community’s FP7 under grant agreements 238803 (BBfor2: bbfor2.net ) and 284989 (BEAT: beat-eu.org ). This work is based on open-source software provided by the Idiap Research Institute and the Colorado State University. The authors want to thank all contributors of the software for their great work.

Author information

Authors and Affiliations

  1. Idiap Research Institute, Martigny, Switzerland

    Manuel Günther, Laurent El Shafey & Sébastien Marcel

Authors
  1. Manuel Günther
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laurent El Shafey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sébastien Marcel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sébastien Marcel .

Editor information

Editors and Affiliations

  1. Multispectral Imagery Lab—MILab, LCSEE, West Virginia University, MORGANTOWN, West Virginia, USA

    Thirimachos Bourlai

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Günther, M., Shafey, L.E., Marcel, S. (2016). Face Recognition in Challenging Environments: An Experimental and Reproducible Research Survey. In: Bourlai, T. (eds) Face Recognition Across the Imaging Spectrum. Springer, Cham. https://doi.org/10.1007/978-3-319-28501-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28501-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28499-6

  • Online ISBN: 978-3-319-28501-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation