Advertisement

Real-time demographic profiling from face imagery with Fisher vectors

  • Lorenzo Seidenari
  • Alessandro Rozza
  • Alberto Del Bimbo
Original paper
  • 22 Downloads

Abstract

In the last decade, demographic profiling from facial imagery has grown in its importance in the computer vision field. For demographic profiling, we usually mean gender, ethnicity, and age identification from face images. In this paper, we propose an efficient and effective profiling framework and we assess the quality of the proposed approach comparing the results obtained by our system with those achieved by other recently published methods on large datasets of facial images with different age, gender, and ethnicity. These results show how a carefully engineered pipeline of efficient image analysis and pattern recognition techniques leads to state-of-the-art results at 20 FPS using a single thread on a 1.6 GHZ i5-2467M processor.

Keywords

Demographic face profiling Age estimation Gender classification Ethnicity classification 

Notes

Acknowledgements

Lorenzo Seidenari is partially supported by “THE SOCIAL MUSEUM AND SMART TOURISM”, MIUR project no. CTN01_00034_23154_SMST.

References

  1. 1.
    Acharya, K.A., Babu, R.V., Vadhiyar, S.S.: A real-time implementation of sift using gpu. J. Real Time Image Process. 14(2), 267–277 (2018)CrossRefGoogle Scholar
  2. 2.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 507–520 (2014)CrossRefGoogle Scholar
  3. 3.
    Alexandre, L.A.: Gender recognition: a multiscale decision fusion approach. Pattern Recognit. Lett. 31(11), 1422–1427 (2010)CrossRefGoogle Scholar
  4. 4.
    Antipov, G., Baccouche, M., Berrani, S.-A., Dugelay, J.-L.: Apparent age estimation from face images combining general and children-specialized deep learning models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 96–104 (2016)Google Scholar
  5. 5.
    Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)
  6. 6.
    Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: A ranking approach for human ages estimation based on face images. In: Proceedings of ICPR, Aug, pp. 3396–3399 (2010)Google Scholar
  7. 7.
    Chang, K.-Y., Chen, C.-S., Hung, Y.-P.: Ordinal hyperplanes ranker with cost sensitivities for age estimation. In: Proceedings of CVPR, June, pp. 585–592 (2011)Google Scholar
  8. 8.
    Cirne, M.V.M., Pedrini, H.: Gender recognition from face images using a geometric descriptor. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2006–2011. IEEE (2017)Google Scholar
  9. 9.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)CrossRefGoogle Scholar
  10. 10.
    Demirkus, M., Precup, D., Clark, J., Arbel, T.: Hierarchical spatio-temporal probabilistic graphical model with multiple feature fusion for estimating binary facial attribute classes in real-world face videos. IEEE Trans. Pattern Anal. Mach. Intell. 6(1185), 1203 (2015)Google Scholar
  11. 11.
    Du, S., Ward, R.: Wavelet-based illumination normalization for face recognition. In: Proceedings of ICPR (2005)Google Scholar
  12. 12.
    Duan, M., Li, K., Li, K.: An ensemble cnn2elm for age estimation. IEEE Trans. Inf. Forensics Secur. 13(3), 758–772 (2018)CrossRefGoogle Scholar
  13. 13.
    Duan, M., Li, K., Yang, C., Li, K.: A hybrid deep learning cnn-elm for age and gender classification. Neurocomputing 275, 448–461 (2018)CrossRefGoogle Scholar
  14. 14.
    Eidinger, E., Enbar, R., Hassner, T.: Age and gender estimation of unfiltered faces. IEEE Trans. Inf. Forensics Secur. 9(12), 2170–2179 (2014)CrossRefGoogle Scholar
  15. 15.
    Escalera, S., Torres Torres, M., Martinez, B., Baró, X., Jair Escalante, H., Guyon, I., Tzimiropoulos, G., Corneou, C., Oliu, M., Ali Bagheri, M., et al: Chalearn looking at people and faces of the world: face analysis workshop and challenge 2016. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2016)Google Scholar
  16. 16.
    Fourney, A., Laganiere, R.: Constructing face image logs that are both complete and concise. In: Proceedings of CRV, pp. 488–494 (2007)Google Scholar
  17. 17.
    Geng, X., Yin, C., Zhou, Z.-H.: Facial age estimation by learning from label distributions. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2401–2412 (2013)CrossRefGoogle Scholar
  18. 18.
    Guo, G., Mu, G., Fu, Y., Huang, T.S.: Human age estimation using bio-inspired features. In: Proceedings of CVPR, June, pp. 112–119 (2009)Google Scholar
  19. 19.
    Guo, G., Mu, G.: A study of large-scale ethnicity estimation with gender and age variations. In: Proceedings of CVPRW, pp. 79–86. IEEE (2010)Google Scholar
  20. 20.
    Guo, G., Mu, G.: Human age estimation: what is the influence across race and gender? In: Proceedings of CVPRW, June, pp. 71–78 (2010)Google Scholar
  21. 21.
    Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)Google Scholar
  22. 22.
    Guo, G., Mu, G.: Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression. In: Proceedings of CVPR, June, pp. 657–664 (2011)Google Scholar
  23. 23.
    Guo, G., Zhang, C.: A study on cross-population age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4257–4263 (2014)Google Scholar
  24. 24.
    Gurpinar, F., Kaya, H., Dibeklioglu, H., Salah, A.: Kernel elm and cnn based facial age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 80–86 (2016)Google Scholar
  25. 25.
    Han, H., Otto, C., Jain, A.K.: Age estimation from face images: human vs. machine performance. In: Proceedings of ICB (2013)Google Scholar
  26. 26.
    Hansen, P.C.: The truncated SVD as a method for regularization. BIT Numer. Math. 27(4), 534–553 (1987)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of CVPR, June 2015Google Scholar
  28. 28.
    Hou, L., Yu, C.-P., Samaras, D.: Squared earth movers distance loss for training deep neural networks on ordered-classes. In: Proceedings of NIPS (2017)Google Scholar
  29. 29.
    Huang, G.-B., Chen, L., Siew, C.K., et al.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)CrossRefGoogle Scholar
  30. 30.
    Huerta, I., Fernández, C., Prati, A.: Facial age estimation through the fusion of texture and local appearance descriptors. In: European Conference on Computer Vision, pp. 667–681. Springer (2014)Google Scholar
  31. 31.
    Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., Geng, X.: Deep age distribution learning for apparent age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 17–24 (2016)Google Scholar
  32. 32.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of CVPR (2014)Google Scholar
  33. 33.
    King, D.E.: Max-Margin Object Detection. ArXiv e-prints, January (2015)Google Scholar
  34. 34.
    Lapuschkin, S., Binder, A., Müller, K.-R., Samek, W.: Understanding and comparing deep neural networks for age and gender classification. arXiv preprint arXiv:1708.07689 (2017)
  35. 35.
    Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of CVPRW, June (2015)Google Scholar
  36. 36.
    Liu, X., Li, S., Kan, M., Zhang, J., Wu, S., Liu, W., Han, H., Shan, S., Chen, X.: Agenet: Deeply learned regressor and classifier for robust apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015Google Scholar
  37. 37.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Malli, R.C., Aygün, M., Ekenel, H.K.: Apparent age estimation using ensemble of deep learning models. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 714–721. IEEE (2016)Google Scholar
  39. 39.
    Meyers, E., Wolf, L.: Using biologically inspired features for face processing. Int. J. Comput. Vis. 76(1), 93–104 (2008)CrossRefGoogle Scholar
  40. 40.
    Nasrollahi, K., Moeslund, T.B: Face quality assessment system in video sequences. In: Biometrics and Identity Management, pp. 10–18. Springer (2008)Google Scholar
  41. 41.
    Ng, C.B., Tay, Y.H., Goi, B.-M.: Vision-based human gender recognition: a survey. CoRR, abs/1204.1611 (2012)Google Scholar
  42. 42.
    Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated age estimation algorithms. NIST Interagency Report, 7995 (2014)Google Scholar
  43. 43.
    Ngan, M., Grother, P.: Face recognition vendor test (frvt) performance of automated gender classification algorithms. In: Technical Report NIST IR 8052. National Institute of Standards and Technology (2015)Google Scholar
  44. 44.
    Ni, B., Song, Z., Yan, S.: Web image and video mining towards universal and robust age estimator. IEEE Trans. Multimed. 13(6), 1217–1229 (2011)CrossRefGoogle Scholar
  45. 45.
    Nvidia cublas library for fast gpu-accelerated implementation of the standard basic linear algebra subroutines (blas). https://developer.nvidia.com/cublas. Accessed 18 Sep 2018
  46. 46.
    Nvidia embedded gpu boards. https://developer.nvidia.com/embedded/develop/hardware. Accessed 18 Sep 2018
  47. 47.
    Platt, J.C., Cristianini, N., Shawe-taylor, J.: Large margin DAGs for multiclass classification. In: Proceedings of NIPS, pp. 547–553. MIT Press (2000)Google Scholar
  48. 48.
    Riesenhuber, Maximilian, Poggio, Tomaso: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2(11), 1019–1025 (1999)CrossRefGoogle Scholar
  49. 49.
    Rothe, R., Timofte, R., Van Gool, L.: Dex Deep expectation of apparent age from a single image. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015Google Scholar
  50. 50.
    Rozza, A., Lombardi, G., Casiraghi, E.: Novel ipca-based classifiers and their application to spam filtering. In: Proceedings of ISDA, pp. 797–802 (2009)Google Scholar
  51. 51.
    Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E: O-ipcac and its application to eeg classification. In: Proceedings of WAPA, pp. 4–11 (2010)Google Scholar
  52. 52.
    Rozza, Alessandro, Lombardi, Gabriele, Casiraghi, Elena, Campadelli, Paola: Novel fisher discriminant classifiers. Pattern Recognit. 45(10), 3725–3737 (2012)CrossRefGoogle Scholar
  53. 53.
    Seidenari, L., Rozza, A., Del Bimbo, A.: Real-time age estimation from face imagery using fisher vectors. In: Proceedings of ICIAP (2015)Google Scholar
  54. 54.
    Seidenari, L., Serra, G., Badanov, A.D., Bimbo, A.: Local pyramidal descriptors for image recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1033–1040 (2014)CrossRefGoogle Scholar
  55. 55.
    Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher Vector Faces in the Wild. In: Proceedings of British Machine Vision Conference (BMVC) (2013)Google Scholar
  56. 56.
    Snchez, Jorge, Perronnin, Florent, Mensink, Thomas, Verbeek, Jakob: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)MathSciNetCrossRefGoogle Scholar
  57. 57.
    Uricár, M., Timofte, R., Rothe, R., Matas, J., Van Gool, L.: Structured output svm prediction of apparent age, gender and smile from deep features. In: Proceedings CVPRW, pp. 25–33 (2016)Google Scholar
  58. 58.
    Wang, J.G., Li, J., Yau, W.Y., Sung, E.: Boosting dense SIFT descriptors and shape contexts of face images for gender recognition. In: IEEE Workshop on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 96–102 (2010)Google Scholar
  59. 59.
    Yang, X., Gao, B.-B., Xing, C., Huo, Z.W., Wei, X.S., Zhou, Y., Wu, J., Geng, X.: Deep label distribution learning for apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015Google Scholar
  60. 60.
    Yun, F., Huang, T.S.: Human age estimation with regression on discriminative aging manifold. IEEE Trans. Multimed. 10(4), 578–584 (2008)CrossRefGoogle Scholar
  61. 61.
    Zhang, K., Gao, C., Guo, L., Sun, M., Yuan, X., Han, T.X., Zhao, Z., Li, B.: Age group and gender estimation in the wild with deep ror architecture. IEEE Access 5, 22492–22503 (2017)CrossRefGoogle Scholar
  62. 62.
    Zhang, K., Sun, M., Han, X., Yuan, X., Guo, L., Liu, T.: Residual networks of residual networks: multilevel residual networks. IEEE Trans. Circuits Syst. Video Technol. 28, 1303–1314 (2017)CrossRefGoogle Scholar
  63. 63.
    Zhu, Y., Li, Y., Mu, G., Guo, G.: A study on apparent age estimation. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, December 2015Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Lastminute.com groupChiassoSwitzerland

Personalised recommendations