Skip to main content

3D-guided facial shape clustering and analysis

Multimedia Tools and Applications volume 81, pages 8785–8806 (2022)Cite this article

Abstract

Facial shape classification is of crucial importance in facial characteristics analysis and product recommendation. In this paper, we develop a 3D-guided facial shape clustering and analysis method to classify facial shapes without supervision, which is more reliable and accurate. This method consists of four steps: 3D face reconstruction, facial shape normalization, facial feature extraction and facial contour clustering. Firstly, we incorporate two 3D face reconstruction methods to reconstruct 3D face mesh without expression component from 1997 male and 2493 female facial images. Secondly, we normalize these 3D facial contours by translation and scaling. Thirdly, we propose two facial contour representations: geometric and anthropometric features. Fourthly, we use and compare three clustering methods to cluster these facial contours based on the extracted contour features by using Silhouette Coefficient and Calinski-Harabasz Index. The Circular Dendrogram of the hierarchical clustering result based on geometric features shows the optimal cluster number is 6 for 3D female and male faces and the analysis results demonstrate the K-means clustering on geometric features can achieve better performance. A further investigation between the beauty distribution and facial shape clusters reveals that the facial shapes with more pointed chin have higher beauty ratings, regardless of male or female. The facial shape analysis results can be applied in face-related product design, hairstyle recommendation and cartoon character creation. The code will be released to the public for research purpose: https://github.com/Easy-Shu/facial_shape_clustering

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Notes

  1. https://www.byrdie.com/is-your-face-round-square-long-heart-or-oval-shaped-345761

References

  1. Alzahrani T, Al-Nuaimy W, Al-Bander B (2019) Hybrid Feature Learning and Engineering Based Approach for Face Shape Classification. In: 2019 International Conference on Intelligent Systems and Advanced Computing Sciences (ISACS). IEEE, pp 1–4

    Google Scholar 

  2. Arthur D, Vassilvitskii S (2006) K-means++: the advantages of careful seeding, Stanford

  3. Ball GH, Hall DJ (1967) A clustering technique for summarizing multivariate data. Behav Sci 12(2):153–155

    Article  Google Scholar 

  4. Bansode N, Sinha P (2016) Face shape classification based on region similarity, correlation and fractal dimensions. International Journal of Computer Science Issues (IJCSI) 13(1):24

    Article  Google Scholar 

  5. Bezdek JC (2013) Pattern recognition with fuzzy objective function algorithms. Springer Science & Business Media

    MATH  Google Scholar 

  6. Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Siggraph, vol 1999, pp 187–194

    Google Scholar 

  7. Blanz V, Vetter T (2003) Face recognition based on fitting a 3d morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074

    Article  Google Scholar 

  8. Caliński T, Harabasz J (1974) A dendrite method for cluster analysis. Communications in Statistics-theory and Methods 3(1):1–27

    MathSciNet  Article  Google Scholar 

  9. Cao C, Weng Y, Zhou S, Tong Y, Zhou K (2013) Facewarehouse: a 3d facial expression database for visual computing. IEEE Trans Vis Comput Graph 20(3):413–425

    Google Scholar 

  10. Chu C-H, Huang S-H, Yang C-K, Tseng C-Y(2015) Design customization of respiratory mask based on 3D face anthropometric data. Int J Precis Eng Manuf 16(3):487–494

    Article  Google Scholar 

  11. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  12. Dornaika F, Elorza A, Wang K, Arganda-Carreras I (2020)Image-based face beauty analysis via graph-based semi-supervised learning. Multimed Tools Appl 79(3):3005–3030

    Article  Google Scholar 

  13. Dunn JC (1973) A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters

  14. Ellena T, Subic A, Mustafa H, Pang TY (2016) The helmet fit index–an intelligent tool for fit assessment and design customisation. Appl Ergon 55:194–207

    Article  Google Scholar 

  15. Ellena T, Subic A, Mustafa H, Yen Pang T (2018) A novel hierarchical clustering algorithm for the analysis of 3D anthropometric data of the human head. Computer-Aided Design and Applications 15(1):25–33

    Article  Google Scholar 

  16. Feng Y, Wu F, Shao X, Wang Y, Zhou X (2018) Joint 3d face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 534–551

    Google Scholar 

  17. Gu Z, Gu L, Eils R, Schlesner M, Brors B (2014) Circlize implements and enhances circular visualization in R. Bioinformatics 30(19):2811–2812

    Article  Google Scholar 

  18. Guo J, Zhu X, Yang Y, Yang F, Lei Z, Li SZ (2020) Towards fast, accurate and stable 3D dense face alignment. arXiv preprint arXiv:200909960

  19. Hu X, Ren W, LaMaster J, Cao X, Li X, Li Z, Menze B, Liu W (2020) Face super-resolution guided by 3d facial priors. In: European Conference on Computer Vision. Springer, pp 763–780

    Google Scholar 

  20. Jackson AS, Bulat A, Argyriou V (2017) Tzimiropoulos G large pose 3D face reconstruction from a single image via direct volumetric CNN regression. Proceedings of the IEEE International Conference on Computer Vision, In, pp 1031–1039

    Google Scholar 

  21. Joblove GH, Greenberg D (1978) Color spaces for computer graphics. In: Proceedings of the 5th annual conference on Computer graphics and interactive techniques, pp 20–25

    Google Scholar 

  22. Jolliffe I (2011) Principal component analysis. Springer

    MATH  Google Scholar 

  23. Liang L, Lin L, Jin L, Xie D, Li M SCUT-FBP5500: a diverse benchmark dataset for multi-paradigm facial beauty prediction. In: 2018 24th International Conference on Pattern Recognition (ICPR), 2018. IEEE, pp 1598–gercll

  24. Liu S, Fan Y-Y, Samal A, Guo Z (2016) Advances in computational facial attractiveness methods. Multimed Tools Appl 75(23):16633–16663

    Article  Google Scholar 

  25. Luximon Y, Ball RM, Chow EH (2016) A design and evaluation tool using 3D head templates. Computer-Aided Design and Applications 13(2):153–161

    Article  Google Scholar 

  26. Murtagh F, Legendre P (2011) Ward's hierarchical clustering method: clustering criterion and agglomerative algorithm. arXiv preprint arXiv:11116285

  27. Pasupa K, Sunhem W, Loo CK (2019) A hybrid approach to building face shape classifier for hairstyle recommender system. Expert Syst Appl 120:14–32

    Article  Google Scholar 

  28. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter TA (2009) 3D face model for pose and illumination invariant face recognition. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE, pp 296–301

    Chapter  Google Scholar 

  29. Rahmat RF, Syahputra MD, Andayani U, Lini TZ (2018)Proba-bilistic neural network and invariant moments for men face shape classification. In: IOP Conference Series: Materials Science and Engineering, vol 1, p 012095

    Google Scholar 

  30. Ren W, Yang J, Deng S, Wipf D, Cao X, Tong X (2019) Face video deblurring using 3d facial priors. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 9388–9397

    Google Scholar 

  31. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

    Article  Google Scholar 

  32. Skals S, Ellena T, Subic A, Mustafa H, Pang TY (2016) Improving fit of bicycle helmet liners using 3D anthropometric data. Int J Ind Ergon 55:86–95

    Article  Google Scholar 

  33. Smith AR (1978) Color gamut transform pairs. ACM Siggraph Computer Graphics 12(3):12–19

    Article  Google Scholar 

  34. Sunhem W, Pasupa K An approach to face shape classification for hairstyle recommendation. In: 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI), 2016. IEEE, pp 390–394

  35. Tio AE (2019) Face shape classification using inception v3. arXiv preprint arXiv:191107916

  36. Xiao Y, Yu J (2012) Partitive clustering (K-means family). Wiley interdisciplinary reviews-data mining and knowledge discovery 2 (3):209-225. https://doi.org/10.1002/Widm.1049

  37. Xie D, Liang L, Jin L, Xu J, Li M (2015) Scut-fbp: A benchmark dataset for facial beauty perception. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, pp 1821–1826

    Chapter  Google Scholar 

  38. Xu Y, Qiu J, Ma L (2010) Measurement and classification of shanghai female face shape based on 3D image feature. In: 2010 3rd International Congress on Image and Signal Processing. IEEE, pp 2588–2592

    Chapter  Google Scholar 

  39. Yang H, Zhu H, Wang Y, Huang M, Shen Q, Yang R, Cao X (2020) FaceScape: a large-scale high quality 3D face dataset and detailed Riggable 3D face prediction. arXiv preprint arXiv:200313989

  40. Zhang S-C, Fang B, Liang Y-Z, Wen J, Wu L (2011) A face clustering method based on facial shape information. In: 2011 International Conference on Wavelet Analysis and Pattern Recognition. IEEE, pp 44–49

    Chapter  Google Scholar 

  41. Zhang J, Zhou K, Luximon YA (2020) Brief Review of 3D Face Reconstruction Methods for Face-Related Product Design. In: Joint Conference of the Asian Council on Ergonomics and Design and the Southeast Asian Network of Ergonomics Societies. Springer, pp 357–366

    Google Scholar 

  42. Zhao J, Cao M, Xie X, Zhang M, Wang L (2019)Data-driven facial attractiveness of Chinese male with epoch characteristics. IEEE Access 7:10956–10966

    Article  Google Scholar 

  43. Zhao J, Zhang M, He C, Xie X, Li J (2020) A novel facial attractiveness evaluation system based on face shape, facial structure features and skin. Cogn Neurodyn 14(5):643–656. https://doi.org/10.1007/s11571-020-09591-9

    Article  Google Scholar 

  44. Zhu X, Liu X, Lei Z, Li SZ (2017) Face alignment in full pose range: a 3d total solution. IEEE Trans Pattern Anal Mach Intell 41(1):78–92

    Article  Google Scholar 

Download references

Acknowledgements

This work was financially supported by the Research Grants Council (RGC) of Hong Kong to conduct General Research Fund (GRF) (15603419).

Author information

Authors and Affiliations

  1. School of Design, The Hong Kong Polytechnic University, Hong Kong, Hong Kong SAR

    Jie Zhang, Yan Luximon & Hassan Iftikhar

  2. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China

    Kangneng Zhou

  3. Department of Computing, The Hong Kong Polytechnic University, Hong Kong, SAR, Hong Kong

    Ping Li

Authors
  1. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kangneng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Luximon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hassan Iftikhar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Luximon.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Zhou, K., Luximon, Y. et al. 3D-guided facial shape clustering and analysis. Multimed Tools Appl 81, 8785–8806 (2022). https://doi.org/10.1007/s11042-022-12190-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12190-x

Keywords

  • Facial shape clustering
  • Facial shape analysis
  • 3D face reconstruction
  • Facial beauty analysis