Multimedia Tools and Applications

, Volume 76, Issue 22, pp 23383–23411 | Cite as

Leveraging implicit demographic information for face recognition using a multi-expert system

  • Maria De MarsicoEmail author
  • Michele Nappi
  • Daniel Riccio
  • Harry Wechsler


This paper describes a novel biometric architecture to implement unsupervised face recognition across varying demographics. The present proposal deals with ethnicity, gender and age, but the same strategy can be crafted for any mix of soft/hard biometrics, sensors, and/or methods. Our aim is not to explicitly distinguish demographic features of a subject (e.g., male vs. female). We rather aim at implicitly exploiting such information to improve the accuracy of subject identification. The role demographics plays in authentication has been reported by many recent studies. Exploiting demographic information can entail two possible strategies. Both require pre-determination of relevant demographic classes, that drive the choice of the best suited recognizer in a set of ad-hoc trained ones. In the first strategy, a human operator visually classifies demographic features of the subject to recognize, and runs the appropriate “strong” recognizer. In the second one, the identification of the most appropriate “strong” recognizer follows the results obtained from a set of upstream classifiers for soft biometrics. Both solutions are poorly suited to most real world applications, e.g., video - surveillance. Our architecture mediates recognition across different demographics without any pre-determination of demographic features. We still have different “strong” classifiers, each trained on a demographic class. The probe is submitted to all of them at once. A supervisor module estimates reliability of the single responses, and the most reliable result is returned. In this approach, classifier reliability is not a static feature, but it is estimated for each probe. The proposed multiple-expert system provides similar performance to pre-determination of demographics. Experimental results show higher flexibility, efficacy and interoperability. We also focus on interoperability across face datasets by adopting EGA (Ethnicity, Gender and Age) database as a benchmark, which is obtained by combining images from several publicly available face datasets.


Biometric systems Demographics Face recognition Interoperability Soft biometrics A-posteriori demographics categorization 


  1. 1.
    Ahonen T, Hadid A, Pietikinen M (2004) Face recognition with local binary patterns. In: Proceedings of European Conference on Computer Vision - ECCV 2004, pp 469–481Google Scholar
  2. 2.
    Branson S, Van Horn G, Wah C, Perona P, Belongie S (2014) The ignorant led by the blind: a hybrid Human-Machine vision system for Fine-Grained categorization. Int J Comput Vis 108(1-2):3–29zbMATHMathSciNetGoogle Scholar
  3. 3.
    Bruyer R, Leclere S, Quinet P (2004) Ethnic categorisation of faces is not independent of face identity. Perception 33(2):169–180CrossRefGoogle Scholar
  4. 4.
    Burton AM, Jenkins R, Schweinberger SR (2011) Mental representations of familiar faces. Br J Psychol 102:943–958CrossRefGoogle Scholar
  5. 5.
    CASIA-FaceV5, Available at Accessed January 2015
  6. 6.
    Chellappa R, Wilson CL, Sirohey S (1995) Human and machine recognition of faces: a survey. Proc IEEE 83(5):705–740CrossRefGoogle Scholar
  7. 7.
    Chiroro P, Valentine T (1995) An investigation of the contact hypothesis of the own-race bias in face recognition. Q J Exp Psychol Hum Exp Psychol 48A:879–894CrossRefGoogle Scholar
  8. 8.
    Cole SA (2004) Fingerprint identification and the criminal justice system: historical lessons for the DNA debate, in David Lazer (Ed.), DNA And The Criminal Justice System - The Technology of Justice (pp. 63–89) MIT PressGoogle Scholar
  9. 9.
    De Marsico M, Nappi M, Riccio D, Tortora G (2011) NABS: Novel Approaches for biometric systems. IEEE Trans Syst Man Cybern Part C Appl Rev 41 (4):481–493CrossRefGoogle Scholar
  10. 10.
    El Khiyari H, De Marsico M, Abate AF, Wechsler H (2012) Biometric Interoperability Across Training, Enrollment, and Testing for Face Authentication. In: Proceedings of 2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (bioMS 2012), Salerno (Italy),September 14 2012, pp 1–8Google Scholar
  11. 11.
    Galton F (1892) Finger prints. MacMillan and Co., New YorkGoogle Scholar
  12. 12.
    Goldstein AG (1979) Race-related variation of facial features: Anthropometric Data I. Bull Psychonomic Soc 13:187–190CrossRefGoogle Scholar
  13. 13.
    Goldstein AG (1979) Facial feature variation: Anthropometric data II. Bull Psychonomic Soc 13:191–193CrossRefGoogle Scholar
  14. 14.
    Gutta S, Huang JRJ, Jonathon P, Wechsler H (2000) Mixture of experts for classification of gender, ethnic origin, and pose of human faces. IEEE Trans Neural Netw 11(4):948–960CrossRefGoogle Scholar
  15. 15.
    Han H, Otto C, Jain AK (2013) Age estimation from face images: Human vs. machine performance. In: Proceedings of 2013 International Conference on Biometrics (ICB), vol 8, p 1Google Scholar
  16. 16.
    Hancock PJB, Bruce V, Burton AM (2000) Recognition of unfamiliar faces. Trends Cognitive Sci 4:330–337CrossRefGoogle Scholar
  17. 17.
    Herlitz A, Lovén J (2013) Sex differences and the own-gender bias in face recognition A meta-analytic review. Vis Cogn 21(9-10):1306–1336CrossRefGoogle Scholar
  18. 18.
    Hosoi S, Takikawa E, Kawade M (2004) Ethnicity estimation with facial images. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition 2004, pp 195–200Google Scholar
  19. 19.
    Huang GB, Ramesh M, Berg T, Learned-Miller E Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, University of Massachusetts, Amherst, Technical Report 07-49, October (2007). Available online at Accessed January 2015
  20. 20.
    Jain AK, Klare B, Park U (2011) Face recognition: Some challenges in forensicsGoogle Scholar
  21. 21.
    Jain V, Mukherjee A The Indian Face Database (2002). Available online at Accessed January 2015
  22. 22.
    Kasiński A., Florek A, Schmidt A (2008) The PUT database. Image Processing & Communications 13(3-4):59–64Google Scholar
  23. 23.
    Klare BF, Burge MJ, Klontz JC, Vorder Bruegge RW, Jain AK (2012) Face recognition performance: Role of demographic information. IEEE Transactions on Information Forensics and Security 7(6):1789–1801CrossRefGoogle Scholar
  24. 24.
    Leng L, Zhang J, Khan MK, Chen X, Alghathbar K (2010) Dynamic weighted discrimination power analysis: a novel approach for face and palmprint recognition in DCT domain. International Journal of Physical Sciences 5(17):2543–2554Google Scholar
  25. 25.
    Leng L, Zhang J, Xu J, Khan MK, Alghathbar K (2010) Dynamic weighted discrimination power analysis in DCT domain for face and palmprint recognition. In: 2010 International Conference on Information and Communication Technology Convergence (ICTC), pp 467–471Google Scholar
  26. 26.
    Leng L, Zhang J, Chen G, Khan MK, Alghathbar K (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition. In: International Conference on Computational Science and Its Applications, LNCS, vol 6786, pp 458–470Google Scholar
  27. 27.
    Ling H, Soatto S, Ramanathan N, Jacobs DW (2007) A Study of Face Recognition as People Age. In: Proc of the IEEE 11Th International Conference on Computer Vision, ICCV 2007, pp 1–8Google Scholar
  28. 28.
    Lu X, Jain AK (2004) Ethnicity identification from face image. proc SPIE:114–123Google Scholar
  29. 29.
    Lyons MJ, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: Proceedings of the IEEE international conference on automatic face and gesture recognition, FG 1998, pp 200–205Google Scholar
  30. 30.
    McGarry DP, Arndt CM, McCabe SA, D’Amato DP (2004) Effects of compression and individual variability on face recognition performance. SPIE Defense & Security Symposium, Biometric Technology for Human Identification, 5404-34 (pp. 362-372) International Society for Optics and PhotonicsGoogle Scholar
  31. 31.
    Meyers E, Wolf L (2008) Using biologically inspired features for face processing. Int J Comput Vis 76(1):93–104CrossRefGoogle Scholar
  32. 32.
    Modi SK, Elliott SJ, Whetsone J, Hakil Kim (2007) Impact of age groups on fingerprint recognition performance. Proceedings of the 2007 IEEE Workshop on Automatic Identification Advanced Technologies:19–23Google Scholar
  33. 33.
    Ng W, Lindsay RC (1994) Cross-race facial recognition: Failure of the contact hypothesis. J Cross-Cultural Psychol 25:217–232CrossRefGoogle Scholar
  34. 34.
    Ng CB, Tay YH, Goi BM (2012) Vision-based Human Gender Recognition: A Survey. Available online at arXiv:1204.1611. Accessed January 2015
  35. 35.
    O’Toole AJ, Phillips PJ, Jiang F, Ayyad J, Pénard N., Abdi H (2007) Face recognition algorithms surpass humans matching faces over changes in illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(9):1642–1646CrossRefGoogle Scholar
  36. 36.
    O’Toole AJ, An X, Dunlop J, Natu V, Phillips PJ (2012) Comparing face recognition algorithms to humans on challenging tasks. ACM Transactions on Applied Perception 9(4):16Google Scholar
  37. 37.
    Phillips PJ, Wechsler H, Huang J, Rauss P (1998) The FERET database and evaluation procedure for Face-Recognition algorithms. Image Vis Comput J 16 (5):295–306CrossRefGoogle Scholar
  38. 38.
    Phillips PJ, Grother P, Micheals R, Blackburn DM, Tabassi E, Bone M (2003) Face recognition vendor test 2002. In: Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures,AMFG 2003Google Scholar
  39. 39.
    Phillips PJ, Flynn P, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of the face recognition grand challenge. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVFR 2005Google Scholar
  40. 40.
    Phillips PJ, Scruggs W, O’Toole A, Flynn P, Bowyer K, Schott C, Sharpe M, FRVT 2006 and ICE 2006 large-scale experimental results (2010). IEEE Trans Pattern Anal Mach Intell 32 (5):831–846CrossRefGoogle Scholar
  41. 41.
    Phillips PJ, Beveridge JR, Draper BA, Givens G, O’Toole AJ, Bolme DS, Dunlop J, Lui YM, Sahibzada H, Weimer S (2011) An introduction to the good, the bad, & the ugly face recognition challenge problem. In: Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, FG 2011, pp 346–353Google Scholar
  42. 42.
    Rhodes MG, Anastasi JS (2012) The own-age bias in face recognition: a meta-analytic and theoretical review. Psychol Bull 138(1):146CrossRefGoogle Scholar
  43. 43.
    Ricanek K, Tesafaye T (2006) MORPH: A longitudinal image database of normal adult age-progression. In: Proceedings of 7th International Conference on Automatic Face and Gesture Recognition, FGR 2006, pp 341–345Google Scholar
  44. 44.
    Riccio D, Tortora G, De Marsico M, Wechsler H (2012) EGA - Ethnicity,Gender and Age, a pre-annotated face database. In: Proceedings of 2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications, bioMS 2012, Salerno (Italy), September 14 2012, pp 38–45Google Scholar
  45. 45.
    Ross A, Jain AK (2004) Biometric sensor interoperability: A case study in fingerprints. Biometric Authentication. Lect Notes Comput Sci 3087:134–145CrossRefGoogle Scholar
  46. 46.
    Shakhnarovich G, Viola PA (2002) Moghaddam,b., A unified learning framework for real time face detection and classification. In: Proceedings 5th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2002, pp 14–21Google Scholar
  47. 47.
    Spacek L The Essex database, , University of Essex
  48. 48.
    Sun Z, Bebis G, Yuan X, Louis SJ (2002) Genetic feature subset selection for gender classification: a comparison study. In: Proceedings of the WACV 2002, pp 165–170Google Scholar
  49. 49.
    Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Advances in Neural Information Processing Systems, pp 1988–1996Google Scholar
  50. 50.
    Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1701–1708Google Scholar
  51. 51.
    The FEI face database, Available at, Accessed January 2015
  52. 52.
    Toderici G, O’Malley SM, Passalis G, Theoharis T, Kakadiaris IA (2010) Ethnicity- and gender-based subject retrieval using 3-D face-recognition techniques. Int J Comput Vis 89(2-3):382–391CrossRefGoogle Scholar
  53. 53.
    Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: Proceedings of the International Conference on Pattern Recognition, ICPR 1991, pp 586–591Google Scholar
  54. 54.
    Vasuhi S, Vaidehi V (2009) Identification of human faces using orthogonal locality preserving projections. In: Proceedings of the 2009 International Conference on Signal Processing Systems, pp 718– 722Google Scholar
  55. 55.
    Veropoulos K, Bebis G, Webster M (2005) Investigating the impact of face categorization on recognition performance. Proceedings of the ISVC, 2005:207–218Google Scholar
  56. 56.
    Zhao W, Chellappa R, Rosenfeld A, Phillips PJ (2003) Face recognition: a literature survey. ACM Comput Surv:399–458Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Maria De Marsico
    • 1
    Email author
  • Michele Nappi
    • 2
  • Daniel Riccio
    • 3
  • Harry Wechsler
    • 4
  1. 1.Sapienza University of RomeRomeItaly
  2. 2.University of SalernoSalernoItaly
  3. 3.University of Naples Federico IINapoliItaly
  4. 4.George Mason UniversityFairfaxUSA

Personalised recommendations