Skip to main content
SpringerLink
Log in

Gender and ethnicity classification of the 3D nose region based on scaling invariant harmonic wave kernel signature

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In 3D face analysis research, automated classification to recognize gender and ethnicity has received an increasing amount of attention in recent years. Feature extraction and feature calculation have a fundamental role in the process of classification construction. In particular, the challenge of 3D low-quality face data, including inconsistent mesh numbers or holes, makes it difficult to extract and calculate facial features. To overcome this challenge, we propose a shape descriptor Scaling invariant harmonic wave kernel signature (SIHWKS) that is robust to scaling, topology and sampling and can effectively describe the global and local properties of 3D shapes simultaneously by involving two energy parameters. We extract a local nose region in the center of the face using isogeodesic stripes replacing full facial information, which has a lower probability of occlusion and lower calculation complexity. Actually, the local nose region is constrained by the skull so that it has high distinction gender and ethnicity property and stability property that are robust to 3D facial expression for gender and ethnicity classification. Compared with gender and ethnicity classification based on 2D deep-learning methods influenced by texture information, the proposed method does not require complex processes for model training and only considers the geometric information of the 3D nose region. In addition, to estimate the effectiveness of our point descriptor SIHWKS for gender and ethnicity classification, we compare our SIHWKS with four existing descriptors – global point signature (GPS), heat kernel signature (HKS), wave kernel signature (WKS) and harmonic wave kernel signature (HWKS) – on four databases, namely, FRGC2.0, Bosphorus3D, Facewarehouse and Asian Mongolian craniofacial. Finally, we perform experiments comparing our method with other recent existing classification methods. The experimental results show that our proposed method can achieve a higher accuracy rate for gender and ethnicity classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aflalo Y, Kimmel R, Raviv D (2013) Scale invariant geometry for nonrigid shapes[J]. SIAM J Imaging Sci

  2. Altameemi A, Abbas HH (2019) Biological landmark Vs quasi-landmarks for 3D face recognition and gender classification[J]. Int J Electric Comput Eng 9 (5):1–9

    Google Scholar 

  3. Aubry M, Schlickewei U, Cremers D (2011) The wave kernel signature: a quantum mechanical approach to shape analysis. In: Computer vision workshop conf, pp 1626–1633

  4. Ballihi L, Amor B, Daoudi M et al (2012) Geometric based 3D facial gender classification. In: International symposium on communications control and signal processing, pp 1–5

  5. Ballihi L, Amor B et al (2013) Boosting 3-D-Geometric features for efficient face recognition and gender classification[J]. IEEE Trans Inf Foren Secur 7(6):1766–1779

    Article  Google Scholar 

  6. Bekios-Calfa J, Buenaposada JM, Baumela L (2011) Revisiting linear discriminant techniques in gender recognition. IEEE Trans Pattern Anal Mach Intell 33(4):858–864

    Article  Google Scholar 

  7. Berretti S, Bimbo D, Pala P (2010) 3D face recognition using isogeodesic stripes. IEEE Trans Pattern Anal Mach Intell 32(12):2162–2177

    Article  Google Scholar 

  8. Bronstein AM, Bronstein MM, Castellani U et al (2010) Shrec2010: robust large-scale shape retrieval benchmark

  9. Bronstein MM, Kokkinos I (2010) Scale-invariant heat kernel signatures for non-rigid shape recognition[C]. In: 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 1704–1711

  10. Chenlei L, Zhongke W, Xingce W, Mingquan Z (2019) Constructing 3D facial hierarchical structure based on surface measurements. Multimedia Tools Appl 78(11):14753–14776

    Article  Google Scholar 

  11. Drira H, Amor B, Srivastava A, Daoudi M (2009) A Riemannian analysis of 3D nose shapes for partial human biometrics. In: Computer vision conf, pp 459–451

  12. Drira H, Amor BB, Srivastava A et al (2009) A riemannian analysis of 3D nose shapes for partial human biometrics. In: Computer vision conf. pp 1–8

  13. Emambakhsh M, Evans A (2016) Nasal patches and curves for Expression-Robust 3D face recognition. IEEE Trans Pattern Anal Mach Intell 1:12–19

    Google Scholar 

  14. Gilani SZ, Shafait F, Mian A (2013) Biologically significant facial landmarks: how significant are they for gender classification. In: Digital image computing conf, pp 1–8

  15. Guo G, Mu G (2010) A study of large-scale ethnicity estimation with gender and age variations. In: IEEE Computer society conference on computer vision and pattern recognition workshops, pp 79–86

  16. Guo G, Mu G (2010) A study of large-scale ethnicity estimation with gender and age variations. In: IEEE Computer society conference on computer vision and pattern recognition workshops, pp 79–86

  17. Han H, Otto C, Liu X, Jain A (2015) Demographic estimation from face images: Human vs. machine performance. IEEE Trans Pattern Anal Mach Intell 37(6):1148–1161

    Article  Google Scholar 

  18. Han X, Ugail H, Palmer I (2009) Gender classification based on 3d face geometry features using SVM. In: Cyberworlds conf, pp 114–118

  19. Hawraa A, Yulia H et al (2018) A 3D morphometric perspective for facial gender analysis and classification using geodesic path curvature features. Comput Vis Media 4(1):1–16

    Google Scholar 

  20. Hosoi S, Takikawa E, Kawade M (2004) Ethnicity estimation with facial images. In: Automatic face and gesture recognition conf, pp 195–200

  21. Hu J, Hua J (2009) Salient spectral geometric features for shape matching and retrieval. Vis Comput 25(7):667–675

    Article  Google Scholar 

  22. Huang D, Ding H, Wang C et al (2014) Local circular patterns for multi-modal facial gender and ethnicity classification. Image Vis Comput 32(12):1181–1193

    Article  Google Scholar 

  23. Li H, Li S, Wu X et al (2018) Scale-Invariant Wave kernel signature for Non-Rigid 3D shape retrieval. In: Big data and smart computing conf

  24. Lian HC, Lu BL, Talikawa E, Hosoi S (2005) Gender recognition using a min-max modular support vector machine. In: Natural computation conf

  25. Lu X, Jain AK (2004) Ethnicity identification from face images. In: SPIE international symposium on defense and security: biometric technology for human identification, pp 114–123

  26. Lu H, Lin H (2007) Gender recognition using adaboosted feature. In: IEEE Natural computation conf

  27. Lv C, Wu Z, Zhang D et al (2018) 3D Nose shape net for human gender and ethnicity classification. Pattern Recognit Lett

  28. Moeini A, Moeini H (2015) Pose-invariant gender classification based on 3D face reconstruction and synthesis from single 2D image. Electron Lett 51 (10):760–762

    Article  Google Scholar 

  29. Mostafa E, Farag A, Shalaby A et al (2013) Long term facial parts tracking in thermal imaging for uncooperative emotion recognition. In: IEEE 6Th international conference on BTAS

  30. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans PAMI 24(7):971–987

    Article  MATH  Google Scholar 

  31. Ovsjanikov M, Sun J, Guibas L (2008) Global intrinsic symmetries of shapes. Comput Graphics Forum 27:1341–1348

    Article  Google Scholar 

  32. Ra PA, Shaogang GB, Michael W (1991) A Eigenfaces for recognition

  33. Ranjan R, Patel VM, Chellappa R (2019) Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell (PAMI) 41(1):121–135

    Article  Google Scholar 

  34. Ren J, Poulenard A, Wonka P et al (2018) Continuous and orientation-preserving correspondences via functional maps. In: SIGGRAPH Asia, pp 1–16

  35. Savelonas MA, Pratikakis I, Sfikas K (2015) An overview of partial 3D object retrieval methodologies[J]. Multimed Tools Appl 74(24):11783–11808

    Article  Google Scholar 

  36. Shakhnarovich G, Viola PA, Moghaddam B (2002) A unified learning framework for real time face detection and classification. In: Automatic face and gesture recognition conf

  37. Sol J, Dodson K, Scharcanski J (2017) Face recognition based on geodesic distance approximations between multivariate normal distributions. In: Imaging systems and techniques conf

  38. Sun J, Ovsjanikov M, Guibas L (2009) A concise and provably informative multi-scale signature based on heat diffusion. Comput Graphics Forum 28:1383–1392

    Article  Google Scholar 

  39. Ter Haar FB, Remco C, Veltkamp A (2009) 3D face matching framework for facial curves. Graph Model 71(2):77–91

    Article  Google Scholar 

  40. Tian Q, Chen S (2016) Joint gender classification and age estimation by nearly orthogonalizing their semantic spaces[J]. Image Vis Comput 69

  41. Tokola R, Mikkilineni A, Boehnen C. (2015) 3D face analysis for demographic biometrics. In: International conference on biometrics, pp 201–207

  42. Wagner A, Wright J, Ganesh A, Zhou Z, Ma Y (2009) Towards a practical face recognition system: Robust registration and illumination by sparse representation. In: IEEE conference on computer vision and pattern recognition, pp 597–604

  43. Wang C, Deng C, Wang S (2020) Imbalance-XGBoost: leveraging weighted and focal losses for binary label-imbalanced classification with XGBoost. Pattern Recognit Lett

  44. Xia B, Amor B, Daoudi M, Gender Joint (2017) Ethnicity and age estimation from 3D faces: An experimental illustration of their correlations. Image Vis Comput 64:90–102

    Article  Google Scholar 

  45. Xia B, Amor B, Drira H, Daoudi M, Ballihi L (2015) Combining face averageness and symmetry for 3D based gender classification. Pattern Recognit 48(3):746–758

    Article  Google Scholar 

  46. Xia B, Amor B, Drira H et al (2013) Gender and 3D facial symmetry: What’s the relationship. In: Workshops on automatic face and gesture recognition conf, vol 5, pp 1–6

  47. Zhang J, Cai H, Guo Y et al (2020) Landmark detection and 3D face reconstruction for caricature using a nonlinear parametric model. Comput Vis Pattern Recognit

  48. Zhang D et al (2020) 3D skull and face similarity measurements based on a harmonic wave kernel signature. Vis Comput 7:142–156

    Google Scholar 

  49. Zhao HY (2012) 3D facial gender classification based on multi-angle LBP feature. Acta Automatica Sinica 38(9):1544

    Article  Google Scholar 

Download references

Acknowledgements

We thank the National Natural Science Foundation of China (No. 61972041, No. 62072045), National Key Cooperation between the BRICS of China (No. 2017YFE0100500), National Key R&D Program of China (No. 2017YFB1002604 and No. 2017YFB1402105), National Natural Science Foundation of China (No. 62102213), and Young Middle-aged Scientific Research Foundation of Qinghai Normal University (No. KJQN2021004). We also thank the facial database providers (FRGCv2.0, Bosphorus3D and FaceWareHouse) and the method code provider in GitHub.

Funding

This work is partially supported by the National Natural Science Foundation of China (No. 61972041, No. 62072045), National Key Cooperation between the BRICS of China (No. 2017YFE0100500), National Key R&D Program of China (No. 2017YFB1002604 and No. 2017YFB1402105), National Natural Science Foundation of China (No. 62102213), and Young Middle-aged Scientific Research Foundation of Qinghai Normal University (No. KJQN2021004).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Beijing Normal University (BNU), 19 Xinjiekou Wai Street, Haidian District, Beijing, China

    Na Liu, Xingce Wang & Zhongke Wu

  2. Computer College, Qinghai Normal University, No. 38, Wusi West Road, Chengxi District, Xining, China

    Dan Zhang

Authors
  1. Na Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingce Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongke Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongke Wu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This study was funded by Beijing Normal University, Beijing, China, and the Key Laboratory of Digital Protection of Cultural Heritage and Virtual Reality, Beijing, China.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, N., Zhang, D., Wang, X. et al. Gender and ethnicity classification of the 3D nose region based on scaling invariant harmonic wave kernel signature. Multimed Tools Appl 82, 41791–41811 (2023). https://doi.org/10.1007/s11042-023-14750-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14750-1

Keywords

Search

Navigation