Multimedia Tools and Applications

, Volume 76, Issue 4, pp 4695–4711 | Cite as

Multi-scale score level fusion of local descriptors for gender classification in the wild

  • M. Castrillón-Santana
  • J. Lorenzo-Navarro
  • E. Ramón-Balmaseda


The 2015 FRVT gender classification (GC) report evidences the problems that current approaches tackle in situations with large variations in pose, illumination, background and facial expression. The report suggests that both commercial and research solutions are hardly able to reach an accuracy over 90 % for The Images of Groups dataset, a proven scenario exhibiting unrestricted or in the wild conditions. In this paper, we focus on this challenging dataset, stepping forward in GC performance by observing: 1) recent literature results combining multiple local descriptors, and 2) the psychophysics evidences of the greater importance of the ocular and mouth areas to solve this task. We therefore make use of holistic and inner facial patches to extract features, that are later combined via a score level fusion strategy. The achieved results support the main information provided by the ocular and the mouth areas. Indeed, the combination of multiscale extracted features increases the overall accuracy to over 94 %, reducing notoriously the classification error if compared with tuned holistic and deep learning approaches.


Soft biometrics Gender classification Local descriptors Score level fusion CNN 


  1. 1.
    Ahonen T, Hadid A, Pietikäinen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12):2037–204CrossRefzbMATHGoogle Scholar
  2. 2.
    Alexandre LA (2010) Gender recognition: a multiscale decision fusion approach. Pattern Recogn Lett 31(11):1422–1427CrossRefGoogle Scholar
  3. 3.
    Antipov G, Berrania SA, Dugelay JL (2016) Minimalistic CNN-based ensemble model for gender prediction from face images. Pattern Recogn Lett 70:59–65CrossRefGoogle Scholar
  4. 4.
    Baluja S, Rowley HA (2007) Boosting sex identification performance. Int J Comput Vis 71(1):111–119CrossRefGoogle Scholar
  5. 5.
    Bekios-Calfa J, Buenaposada JM, Baumela L (2011) Revisiting linear discriminant techniques in gender recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(4):858–864CrossRefGoogle Scholar
  6. 6.
    Bekios-Calfa J, Buenaposada JM, Baumela L (2014) Robust gender recognition by exploiting facial attributes dependencies. Pattern Recogn Lett 36:228–234CrossRefGoogle Scholar
  7. 7.
    Castrillón-Santana M, De Marsico M, Nappi M, Riccio D (2015) MEG: Multi-Expert Gender classification in a demographics-balanced dataset. In: 18Th international conference on image analysis and processing (ICIAP)Google Scholar
  8. 8.
    Castrillón-Santana M, Lorenzo-Navarro J, Ramón-Balmaseda E (2013) Improving gender classification accuracy in the wild. In: 18Th iberoamerican congress on pattern recognition (CIARP), pp 270–277Google Scholar
  9. 9.
    Castrillón-Santana M, Lorenzo-Navarro J, Ramón-Balmaseda E (2015) Descriptors and regions of interest fusion for gender classification in the wild ArXiv e-printsGoogle Scholar
  10. 10.
    Castrillón-Santana M, Lorenzo-Navarro J, Ramón-Balmaseda E (2015) Fusion of holistic and part based features for gender classification in the wild. In: New trends in image analysis and processing–ICIAP 2015 workshops. Springer International Publishing, pp 43–50Google Scholar
  11. 11.
    Castrillón-Santana M, Lorenzo-Navarro J, Ramón-Balmaseda E (2016) On using periocular biometric for gender classification in the wild. Pattern Recogn Lett (in press). doi: Google Scholar
  12. 12.
    Chai Z, Sun Z, Tan T, Mendez-Vazquez H (2013) Local salient patterns - a novel local descriptor for face recognition International conference on biometrics (ICB)Google Scholar
  13. 13.
    Chen H, Gallagher AC, Girod B (2014) The hidden sides of names - face modeling with first name attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(9):1860–1873CrossRefGoogle Scholar
  14. 14.
    Chen J, Shan S, He C, Zhao G, Pietikainen M, Chen X, Gao W (2010) Wld: A robust local image descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9):1705–1720. doi: 10.1109/TPAMI.2009.155 CrossRefGoogle Scholar
  15. 15.
    Dago-Casas P, González-Jiménez D, Long-Yu L, Alba-Castro JL (2011) Single- and cross- database benchmarks for gender classification under unconstrained settings Proceedings of the 1st IEEE international workshop on benchmarking facial image analysis technologies, pp 2152–2159Google Scholar
  16. 16.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Schmid C, Soatto S, Tomasi C (eds) International Conference on Computer Vision & Pattern Recognition (CVPR), vol 2, pp 886–893Google Scholar
  17. 17.
    Dantcheva A, Elia P, Ross A (2016) What else does your biometrics data reveal? a survey on soft biometrics. IEEE Transactions on Information Forensics And Security 11:441–467CrossRefGoogle Scholar
  18. 18.
    Erdogmus N, Vanoni M, Marcel S (2014) Within- and cross- database evaluations for face gender classification via befit protocols IEEE 16Th international workshop on multimedia signal processing (MMSP), pp 1–6Google Scholar
  19. 19.
    Fazl-Ersi E, Mousa-Pasandi ME, Laganiere R, Awad M (2014) Age and gender recognition using informative features of various types International conference on image processingGoogle Scholar
  20. 20.
    Gallagher A, Chen T (2009) Understanding images of groups of people IEEE Computer society conference on computer vision and pattern recognition (CVPR), pp 256–263Google Scholar
  21. 21.
    García-Olalla O, Alegre E, Fernández-Roble L, González-Castro V (2014) Local oriented statistics information booster (LOSIB) for texture classification International conference on pattern recognition (ICPR)Google Scholar
  22. 22.
    Gosselin F, Schyns PG (2001) Bubbles: a technique to reveal the use of information in recognition tasks. Vis Res 41(17):2261–2271CrossRefGoogle Scholar
  23. 23.
    Han H, Jain AK (2014) Age, gender and race estimation from unconstrained face images. Tech. Rep. MSU-CSE-14-5, Michigan State UniversityGoogle Scholar
  24. 24.
    Heisele B, Serre T, Poggio T (2007) A component-based framework for face detection and identification, vol 74Google Scholar
  25. 25.
    Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. Rep. 07-49. University of Massachusetts, AmherstGoogle Scholar
  26. 26.
    Jain AK, Dass SC, Nandakumar K (2004) Soft biometric traits for personal recognition systems International conference on biometric authentication, pp 731–738CrossRefGoogle Scholar
  27. 27.
    Jain AK, Kumar A (2012) Secong Generation Biometrics, chap. Biometrics of Next Generation: An Overview, pp 49–79, SpringerGoogle Scholar
  28. 28.
    Jia S, Cristianini N (2015) Learning to classify gender from four million images. Pattern Recogn Lett 58:35–41CrossRefGoogle Scholar
  29. 29.
    Jun B, Kim D (2012) Robust face detection using local gradient patterns and evidence accumulation. Pattern Recogn 45(9):3304–3316CrossRefGoogle Scholar
  30. 30.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges C, Bottou L, Weinberger K (eds) Advances in neural information processing systems 25. Curran Associates, Inc., pp 1097–1105Google Scholar
  31. 31.
    Kumar N, Berg AC, Belhumeur PN, Nayar SK (2011) Describable visual attributes for face verification and image search. IEEE Transactions on Pattern Analysis and Machine Intelligence:1962– 1977Google Scholar
  32. 32.
    Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition Proceedings of the IEEE, vol 86, pp 2278–2324Google Scholar
  33. 33.
    Levi G, Hassner T (2015) Age and gender classification using convolutional neural networks IEEE Workshop on analysis and modeling of faces and gestures (AMFG), at the IEEE conf. on computer vision and pattern recognition (CVPR), pp 34–42. BostonGoogle Scholar
  34. 34.
    Liu L, Fieguth P, Zhao L, Long Y, Kuang G (2012) Extended local binary patterns for texture classification. Image Vis Comput 30(2):86–99. doi: 10.1016/j.imavis.2012.01.001 CrossRefGoogle Scholar
  35. 35.
    Ludwig O, Delgado D, Goncalves V, Nunes U (2009) Trainable classifier-fusion schemes: an application to pedestrian detection 12Th international IEEE conference on intelligent transportation systems (ITSC), pp 1–6Google Scholar
  36. 36.
    Mansanet J, Albiol A, Paredes R (2016) Local deep neural networks for gender recognition. Pattern Recogn Lett 70:80–86CrossRefGoogle Scholar
  37. 37.
    Ngan M, Grother P (2015) Face recognition vendor test (FRVT) performance of automated gender classification algorithms. Tech. Rep. NIST IR 8052 National Institute of Standars and TechnologyGoogle Scholar
  38. 38.
    Nixon M, Correia P, Nasrollahi K, Moeslund T, Hadid A, Tistarelli M (2015) On soft biometrics. Pattern Recogn Lett 68, Part 2:218–230CrossRefGoogle Scholar
  39. 39.
    Ojansivu V, Heikkil J (2008) Blur insensitive texture classification using local phase quantization. In: Elmoataz A, Lezoray O, Nouboud F., Mammass D. (eds) Image and Signal Processing, LNCS 5099. Springer, pp 236–243Google Scholar
  40. 40.
    Ren H, Li ZN (2014) Gender recognition using complexity-aware local features. In: International conference on pattern recognition, pp 2389–2394Google Scholar
  41. 41.
    Shafey LE, Khoury E, Marcel S (2014) Audio-visual gender recognition in uncontrolled environment using variability modeling techniques International joint conference on biometrics, pp 1–8Google Scholar
  42. 42.
    Shan C (2012) Learning local binary patterns for gender classification on realworld face images. Pattern Recogn Lett 33:431–437CrossRefGoogle Scholar
  43. 43.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification IEEE Conference on computer vision and pattern recognition (CVPR), pp 1701–1708Google Scholar
  44. 44.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetCrossRefGoogle Scholar
  45. 45.
    Tapia JE, Pérez CA (2013) Gender classification based on fusion of different spatial scale features selected by mutual information from histogram of lbp, intensity and shape. IEEE Transactions on Information Forensics and Security 8(3):488–499CrossRefGoogle Scholar
  46. 46.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefzbMATHGoogle Scholar
  47. 47.
    van de Wolfshaar J, Karaaba MF, Wiering MA (2015) Deep convolutional neural networks and support vector machines for gender recognition IEEE Symposium series on computational intelligence: Symposium on computational intelligence in biometrics and identity managementGoogle Scholar
  48. 48.
    Zhang H, Beveridge JR, Draper BA, Phillips PJ (2015) On the effectiveness of soft biometrics for increasing face verification rates. Comput Vis Image Underst 137:50–62CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • M. Castrillón-Santana
    • 1
  • J. Lorenzo-Navarro
    • 1
  • E. Ramón-Balmaseda
    • 1
  1. 1.SIANIUniversidad de Las Palmas de Gran Canaria (ULPGC)Las PalmasSpain

Personalised recommendations