Skip to main content
SpringerLink
Log in

3D face recognition using image decomposition and POEM descriptor

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This article presents a novel algorithm for automatic 3D face recognition, which is robust to facial expression alterations, missing data, and outliers. This algorithm is divided into three main components: First, the 3D face scan is decomposed into structure–texture images. Second, feature vectors are extracted from each component. Third, a postprocessing is applied to deal with the outlier embedded in a feature. The proposed technique was tested on two public datasets (i.e., Gavab and Bosphorus). Experimental testing shows that our proposed methods can increase facial recognition performance, as compared to relevant state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets used during the current study are publicly available (https://www.face-rec.org/databases/).

References

  1. Li, M., Huang, B., Tian, G.: A comprehensive survey on 3d face recognition methods. Eng. Appl. Artif. Intell.Artif. Intell. 110, 104669 (2022)

    Article  Google Scholar 

  2. Jing, Y., Lu, X. and Gao, S.: 3d face recognition: A survey. arXiv preprint arXiv:2108.11082 (2021)

  3. Marvadi, D., Paunwala, C., Joshi, M. and Vora, A.: Comparative analysis of 3D face recognition using 2D-PCA and 2D-LDA approaches, in Engineering (NUiCONE), 2015 5th Nirma University International Conference on, pp. 1–5. (2015)

  4. Hesher, C., Srivastava, A. and Erlebacher, G.: A novel technique for face recognition using range imaging," in Signal processing and its applications, 2003. Proceedings. Seventh international symposium on, pp. 201–204. (2003)

  5. Tonchev, K., Manolova, A. and Paliy, I.: Comparative analysis of 3d face recognition algorithms using range image and curvature-based representations, in Intelligent Data Acquisition and Advanced Computing Systems (IDAACS), 2013 IEEE 7th International Conference on, pp. 394–398. (2013)

  6. Kamencay, P., Hudec, R., Benco, M., Zachariasova, M.: 2D-3D Face recognition method basedon a modified CCA-PCA algorithm. Int. J. Adv. Rob. Syst. 11, 36 (2014)

    Article  Google Scholar 

  7. Gervei, O., Ayatollahi, A., Gervei, N.: 3D face recognition using modified PCA methods. World Acad. Sci. Eng. Technol. 4(39), 264 (2010)

    Google Scholar 

  8. Agbolade, O., Nazri, A., Yaakob, R., Ghani, A.A., Cheah, Y.K.: 3-Dimensional facial expression recognition in human using multi-points warping. BMC Bioinformatics 20, 619 (2019)

    Article  Google Scholar 

  9. Turk, M.A. and Pentland, A.P.: Face recognition using eigenfaces, in Proceedings. 1991 IEEE computer society conference on computer vision and pattern recognition, pp. 586,587,588,589,590,591–586,587,588,589,590,591. (1991)

  10. Chang, K., Bowyer, K. and Flynn, P.: Face recognition using 2D and 3D facial data, in ACM Workshop on Multimodal User Authentication, pp. 25–32. (2003)

  11. Yuan, X., Lu, J. and Yahagi, T.: A method of 3d face recognition based on principal component analysis algorithm," in Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on, pp. 3211–3214. (2005)

  12. Kong, W., You, Z., Lv, X.: 3D face recognition algorithm based on deep Laplacian pyramid under the normalization of epidemic control. Comput. Commun.. Commun. 199, 30–41 (2023)

    Article  Google Scholar 

  13. Niu, W., Zhao, Y., Yu, Z., Liu, Y., Gong, Y.: Research on a face recognition algorithm based on 3D face data and 2D face image matching. J. Vis. Commun. Image Represent.Commun. Image Represent. 1(91), 103757 (2023)

    Article  Google Scholar 

  14. Lu, X. and Jain, A.K.: Integrating range and texture information for 3D face recognition, in Application of Computer Vision, 2005. WACV/MOTIONS'05 Volume 1. Seventh IEEE Workshops on, pp. 156-163 (2005)

  15. Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. Comput. Vis. Eccv 2004, 469–481 (2004)

    Google Scholar 

  16. Liu, C., Wechsler, H.: Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans. Image Process. 11, 467–476 (2002)

    Article  Google Scholar 

  17. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. VisionComput. Vision 60, 91–110 (2004)

    Article  Google Scholar 

  18. Li, S.Z., Zhao, C. Ao, M. and Lei, Z.: Learning to fuse 3D+ 2D based face recognition at both feature and decision levels, in AMFG, pp. 44–54 (2005)

  19. Wang, Y., Liu, J., Tang, X.: Robust 3D face recognition by local shape difference boosting. IEEE Trans. Pattern Anal. Mach. Intell.Intell. 32, 1858–1870 (2010)

    Article  Google Scholar 

  20. Xu, C., Li, S., Tan, T., Quan, L.: Automatic 3D face recognition from depth and intensity Gabor features. Pattern Recogn.Recogn. 42, 1895–1905 (2009)

    Article  Google Scholar 

  21. Huang, D., Ardabilian, M., Wang, Y., Chen, L.: 3-D face recognition using eLBP-based facial description and local feature hybrid matching. IEEE Trans. Inf. Forensics Secur.Secur. 7, 1551–1565 (2012)

    Article  Google Scholar 

  22. Huang, D., Ardabilian, M. Wang, Y. and Chen, L.: Automatic asymmetric 3D-2D face recognition, in Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 1225–1228 (2010)

  23. Tang, H., Yin, B., Sun, Y., Hu, Y.: 3D face recognition using local binary patterns. Signal Process. 93, 2190–2198 (2013)

    Article  Google Scholar 

  24. Wang, X., Ruan, Q., and Ming, Y.: 3D face recognition using corresponding point direction measure and depth local features, in Signal Processing (ICSP), 2010 IEEE 10th International Conference on, pp. 86–89 (2010)

  25. Li, X., Ruan, Q., Jin, Y., An, G., Zhao, R.: Fully automatic 3D facial expression recognition using polytypic multi-block local binary patterns. Signal Process. 108, 297–308 (2015)

    Article  Google Scholar 

  26. Xiong, P., Huang, L. and Liu, C.: Real-time 3D face recognition with the integration of depth and intensity images, Image Analysis and Recognition, pp. 222–232, (2011)

  27. Huang, D., Zhang, G., Ardabilian, M., Wang, Y., Chen, L.: 3D face recognition using distinctiveness enhanced facial representations and local feature hybrid matching, in Biometrics: Theory Applications and Systems (BTAS). Fourth IEEE Int. Conf. 2010, 1–7 (2010)

    Google Scholar 

  28. Shi, L., Wang, X., Shen, Y.: Research on 3D face recognition method based on LBP and SVM. Optik 220, 165157 (2020)

    Article  Google Scholar 

  29. Neto, J.B.C., Ferrari, C., Marana, A.N., Berretti, S., Del Bimbo, A.: Learning streamed attention network from descriptor images for cross-resolution 3D face recognition. ACM Trans. Multimed. Comput. Commun. Appl.Multimed. Comput. Commun. Appl. 19, 1–20 (2023)

    Article  Google Scholar 

  30. Wang, Y., Chua, C.-S., Ho, Y.-K.: Facial feature detection and face recognition from 2D and 3D images. Pattern Recogn. Lett. Recogn. Lett. 23, 1191–1202 (2002)

    Article  Google Scholar 

  31. Hiremath, P. and Manjunatha, H.: 3D face recognition based on depth and intensity Gabor features using symbolic PCA and AdaBoost, (2014)

  32. Torkhani, G., Ladgham, A., Sakly, A., Mansouri, M.N.: A 3D–2D face recognition method based on extended Gabor wavelet combining curvature and edge detection. SIViP 11, 969–976 (2017)

    Article  Google Scholar 

  33. Freund, Y. and Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting, in European conference on computational learning theory, pp. 23–37 (1995)

  34. A. Mian, M. Bennamoun, and R. Owens, An efficient multimodal 2D-3D hybrid approach to automatic face recognition, IEEE Transactions on pattern analysis and machine intelligence, vol. 29, (2007).

  35. Huang, Y. Wang, Y. and Tan, T.: Combining Statistics of Geometrical and Correlative Features for 3D Face Recognition, pp. 90.1–90.10, (2006)

  36. Szeptycki, P., Ardabilian, M. and Chen, L.: A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking, in 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–6 (2009)

  37. Aujol, J.-F., Chambolle, A.: Dual norms and image decomposition models. Int. J. Comput. Vis. Comput. Vis. 63, 85–104 (2005)

    Article  Google Scholar 

  38. Vese, L.A., Osher, S.J.: Modeling textures with total variation minimization and oscillating patterns in image processing. J. Sci. Comput. Comput. 19, 553–572 (2003)

    Article  MathSciNet  Google Scholar 

  39. Vese, L.A., Osher, S.J.: Image denoising and decomposition with total variation minimization and oscillatory functions. J. Math. Imag. Vis. 20, 7–18 (2004)

    Article  MathSciNet  Google Scholar 

  40. Vese, L.A., Osher, S.J.: Color texture modeling and color image decomposition in a variational-PDE approach in. Eighth Int. Symp. Symb. Numer. Algorithms Sci Comput. 2006, 103–110 (2006)

    Google Scholar 

  41. Meyer, Y.: Oscillating patterns in image processing and nonlinear evolution equations: the fifteenth Dean Jacqueline B. Lewis memorial lectures vol. 22: American Mathematical Soc., (2001).

  42. Vu, N.-S., Caplier, A.: Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans. Image Process. 21, 1352–1365 (2012)

    Article  MathSciNet  Google Scholar 

  43. Hyvarinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw.Netw. 10, 626–634 (1999)

    Article  Google Scholar 

  44. Abbad, A., Elharrouss, O., Abbad, K., Tairi, H.: Application of MEEMD in post-processing of dimensionality reduction methods for face recognition. IET Biometrics 8, 59–68 (2019)

    Article  Google Scholar 

  45. Abbad, A., Douini, Y., Abbad, K., Tairi, H.: Post-processing of dimensionality reduction methods for face recognition. Pattern Recognit. Image Anal. Recognit. Image Anal. 27, 266–275 (2017)

    Article  Google Scholar 

  46. Wu, Z., Huang, N.E., Chen, X.: The multi-dimensional ensemble empirical mode decomposition method. Adv. Adapt. Data Anal. 1, 339–372 (2009)

    Article  MathSciNet  Google Scholar 

  47. Moreno, A.B. and Sánchez, A.: GavabDB: a 3D face database, in Proc. 2nd COST275 Workshop on Biometrics on the Internet, Vigo (Spain), pp. 75–80 (2004)

  48. Alyuz, N., Gokberk, B. and Akarun, L.: A 3D face recognition system for expression and occlusion invariance, in Biometrics: Theory, Applications and Systems, 2008. BTAS 2008. 2nd IEEE International Conference on, pp. 1–7 (2008)

  49. Yang, M., Zhang, L., Yang, J., Zhang, D.: Regularized robust coding for face recognition. IEEE Trans. Image Process. 22, 1753–1766 (2013)

    Article  MathSciNet  Google Scholar 

  50. Yang, W., Wang, Z., Yin, J., Sun, C., Ricanek, K.: Image classification using kernel collaborative representation with regularized least square. Appl. Math. Comput. Comput. 222, 13–28 (2013)

    MathSciNet  Google Scholar 

  51. Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: which helps face recognition?, in. Int. Conf. Comput. Vis. 2011, 471–478 (2011)

    Google Scholar 

  52. Mousavi, M.H., Faez, K. and Asghari, A.: Three Dimensional Face Recognition Using SVM Classifier, pp. 208–213, (2008)

  53. Zhang, L., Ding, Z., Li, H., Shen, Y., Lu, J.: 3D face recognition based on multiple keypoint descriptors and sparse representation. PLoS ONE 9, e100120 (2014)

    Article  Google Scholar 

  54. Moreno, A.B., Sanchez, A., Velez, J. and Diaz, J.: Face recognition using 3D local geometrical features: PCA vs. SVM," in Image and Signal Processing and Analysis, 2005. ISPA 2005. Proceedings of the 4th International Symposium on, pp. 185–190 (2005)

  55. Li, X., Jia, T. and Zhang, H.: Expression-insensitive 3D face recognition using sparse representation, in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 2575-2582 (2009)

  56. Mahoor, M.H., Abdel-Mottaleb, M.: Face recognition based on 3D ridge images obtained from range data. Pattern Recogn. Recogn. 42, 445–451 (2009)

    Article  Google Scholar 

  57. Drira, H., Amor, B.B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell.Intell. 35, 2270–2283 (2013)

    Article  Google Scholar 

  58. Berretti, S., Werghi, N., del Bimbo, A., Pala, P.: Matching 3D face scans using interest points and local histogram descriptors. Comput. Graph. Graph. 37, 509–525 (2013)

    Article  Google Scholar 

  59. Lei, Y., Guo, Y., Hayat, M., Bennamoun, M., Zhou, X.: A two-phase weighted collaborative representation for 3D partial face recognition with single sample. Pattern Recogn. Recogn. 52, 218–237 (2016)

    Article  Google Scholar 

  60. Abbad, A., Abbad, K., Tairi, H.: 3D face recognition: Multi-scale strategy based on geometric and local descriptors. Comput. Electr. Eng. Electr. Eng. 70, 525–537 (2018)

    Article  Google Scholar 

  61. Abbad, A., Abbad, K. and Tairi, H.: 3D face recognition in the presence of facial expressions based on empirical mode decomposition, in Proceedings of the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence, pp. 1–6 (2018)

  62. Dutta, K., Bhattacharjee, D. and Nasipuri, M. Volumetric Density of Triangulated Range Images for Face Recognition, in Transactions on Computational Science XXXV: Special Issue on Signal Processing and Security in Distributed Systems, pp. 58–84 (2020)

  63. Li, H., Huang, D., Morvan, J.-M., Wang, Y., Chen, L.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. Comput. Vis. 113, 128–142 (2015)

    Article  MathSciNet  Google Scholar 

  64. Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: meshSIFT: local surface features for 3D face recognition under expression variations and partial data. Comput. Vis. Image Underst. Vis. Image Underst. 117, 158–169 (2013)

    Article  Google Scholar 

  65. Liu, P., Wang, Y., Huang, D., Zhang, Z., Chen, L.: Learning the spherical harmonic features for 3-D face recognition. IEEE Trans. Image Process. 22, 914–925 (2012)

    Article  MathSciNet  Google Scholar 

  66. Berretti, S., Werghi, N., del Bimbo, A., Pala, P.: Selecting stable keypoints and local descriptors for person identification using 3D face scans. Vis. Comput. Comput. 30, 1275–1292 (2014)

    Article  Google Scholar 

  67. Gilani, S.Z., Mian, A., Eastwood, P.: Deep, dense and accurate 3D face correspondence for generating population specific deformable models. Pattern Recogn. Recogn. 69, 238–250 (2017)

    Article  Google Scholar 

  68. Liang, Y., Zhang, Y., Zeng, X.-X.: Pose-invariant 3D face recognition using half face. Signal Process. Image Commun. 57, 84–90 (2017)

    Article  Google Scholar 

  69. Kim, D., Hernandez, M., Choi, J., Medioni, G.: Deep 3D face identification, in. IEEE Int. Joint Conf. Biom. IJCB 2017, 133–142 (2017)

    Google Scholar 

  70. Lin, S., Liu, F., Liu, Y. and Shen, L.: Local feature tensor based deep learning for 3d face recognition, in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1–5 (2019)

  71. Olivetti, E.C., Ferretti, J., Cirrincione, G., Nonis, F., Tornincasa, S. and Marcolin, F.: Deep CNN for 3D face recognition, in Design Tools and Methods in Industrial Engineering: Proceedings of the International Conference on Design Tools and Methods in Industrial Engineering, ADM 2019, September 9–10, 2019, Modena, Italy, pp. 665–674 (2020)

  72. Bhople, A.R., Prakash, S.: Learning similarity and dissimilarity in 3D faces with triplet network. Multimed. Tools Appl. 80, 35973–35991 (2021)

    Article  Google Scholar 

  73. Hariri, W., Farah, N. and Vishwakarma, D.K.: Deep and Shallow Covariance Feature Quantization for 3D Facial Expression Recognition, arXiv preprint arXiv:2105.05708, (2021)

  74. Zhang, Z., Da, F., Yu, Y.: Learning directly from synthetic point clouds for “in-the-wild” 3D face recognition. Pattern Recogn. Recogn. 123, 108394 (2022)

    Article  Google Scholar 

  75. Yu, Y., Da, F., Zhang, Z.: Few-data guided learning upon end-to-end point cloud network for 3D face recognition. Multimed. Tools Appl. 81, 12795–12814 (2022)

    Article  Google Scholar 

  76. Wang, Q., Lei, H., Qian, W.: Point CNN: 3D face recognition with local feature descriptor and feature enhancement mechanism. Sensors 23, 7715 (2023)

    Article  Google Scholar 

  77. Yu, C., Zhang, Z., Li, H., Sun, J., Xu, Z.: Meta-learning-based adversarial training for deep 3D face recognition on point clouds. Pattern Recogn. Recogn. 134, 109065 (2023)

    Article  Google Scholar 

Download references

Acknowledgments

Not applicable.

Author information

Authors and Affiliations

  1. ISA, Department of Computer Science, Faculty of Science and Technology, University Sidi Mohamed Ben Abdelah, Fez, Morocco

    Abdelghafour Abbad & Khalid Abbad

  2. LISAC, Department of Computer Science, Faculty of Sciences Dhar El Mahraz, University Sidi Mohamed Ben Abdelah, Fez, Morocco

    Soukaina El Idrissi El Kaitouni & Hamid Tairi

  3. Computer Science and Systems Engineering, Department of Computer Science, Faculty of Science and Technology, University Abdelmalek Essaadi, Tetouan, Morocco

    Adil Benhdech

Authors
  1. Abdelghafour Abbad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soukaina El Idrissi El Kaitouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adil Benhdech
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khalid Abbad
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hamid Tairi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AA and SE wrote the main manuscript text. AB checked the English grammar. KA and HT reviewed the manuscript.

Corresponding author

Correspondence to Abdelghafour Abbad.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbad, A., El Kaitouni, S.E.I., Benhdech, A. et al. 3D face recognition using image decomposition and POEM descriptor. SIViP (2024). https://doi.org/10.1007/s11760-024-03128-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03128-x

Keywords

Search

Navigation