Skip to main content
SpringerLink
Log in

Registration of 3D face images of JUDeiTy3DK database using POFaceReg8 algorithm

  • Published:
Sādhanā Aims and scope Submit manuscript

Abstract

The application of depth cameras for 3D visual data based measurements (VBM) is an active research area. 3D face recognition is one such niche area that is drawing the attention of researchers. This paper introduces a new 3D facial database named JUDeiTy3DK developed by a research group at Jadavpur University, Kolkata, India, using Kinect as a vision-based depth measuring instrument to capture 3D facial depth data. This database aims to provide the research community with a new 3D face database containing various challenging situations in the context of face recognition. This database consists of 3D face models of 200 male and female individuals with various poses, facial expressions, real-world occlusions, and combinations of various poses, expressions, and occlusions we encounter daily. In addition, ground truth face images are also provided, where 14 visible facial landmark points on each face are manually annotated. Also, we have formulated a new 3D face registration algorithm named POFaceReg8, which combines some selected features of landmark and holistic-based algorithms, followed by a new face recognition algorithm Diag_DCT. We have presented the registration and recognition performance results by comparing our proposed methods with other standard state-of-the-art techniques using pre-existing metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11

Similar content being viewed by others

Abbreviations

\({\Psi }_{\mathrm{ij}}\) :

Interpolated target point clouds

\({\mathrm{Point}}_{\mathrm{new}(\mathrm{x},\mathrm{y},\mathrm{z})}\) :

Interpolated 3D point

\({\Phi }_{\mathrm{ij}}\) :

Facial landmarks corresponding to each target model \({\Psi }_{\mathrm{ij}}\)

\({\Psi }_{\mathrm{ij}\_\mathrm{up}}\) :

Equal sized n target models in point cloud form

\({\Psi }_{\mathrm{ij}\_\mathrm{target}}\) :

Final averaged target model

Sij :

The source facial landmarked model corresponding to the source point cloud \({\mathrm{S}}_{\mathrm{pointij}}\)

CT :

Centroid of \({\Psi }_{\mathrm{ij}\_\mathrm{target}}\)

CS :

Centroid of Sij

H:

Covariance matrix

R, t:

The rotation and translation matrices after the singular value decomposition of H

\({\mathrm{S}}_{\mathrm{coarseij}}\) :

Transformed point cloud after coarse alignment by PCA

\({\mathrm{S}}_{\mathrm{ij}\_\mathrm{trans}}\) :

Transformed landmarks after coarse alignment by PCA

\({\mathrm{R}}_{1},{\mathrm{T}}_{1}\) :

The rotation and translation matrices after fine alignment

\({\mathrm{S}}_{\mathrm{ij}\_\mathrm{fine}}\) :

Transformed point cloud after fine alignment by ICP

\({\mathrm{S}}_{\mathrm{fineij}}\) :

Final registered point cloud

References

  1. Turk M and Pentland A 1991 Eigenfaces for recognition. Journal of Cognitive Neuroscience 3: 71–86

    Article  Google Scholar 

  2. Belhumeur P N, Hespanha J P and Kriegman D J 1997 Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19: 711–720

    Article  Google Scholar 

  3. Ahonen T, Hadid A and Pietikainen M 2006 Face description with local binary patterns: application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 28: 2037–2041

    Article  MATH  Google Scholar 

  4. Phillips P J, Moon H, Rizvi S A and Rauss P J 2000 The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22: 1090–1104

    Article  Google Scholar 

  5. Beveridge J R, Givens G H, Phillips P J, Draper B A, Bolme D S and Lui Y M 2010 FRVT 2006: quo vadis face quality. Image and Vision Computing 28: 732–743

    Article  Google Scholar 

  6. Phillips P J, Flynn P J, Scruggs T, Bowyer K W, Chang J, Hoffman K and Worek W 2005 Overview of the face recognition grand challenge. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1: 947–954

    Google Scholar 

  7. Mohan K, Seal A, Krejcar O and Yazidi A 2020 Facial expression recognition using local gravitational force descriptor-based deep convolution neural networks. IEEE Transactions on Instrumentation and Measurement 70: 1–12

    Article  Google Scholar 

  8. Yu H, Zhen W, Yang W and Scherer S 2020 Line-based 2-D–3-D registration and camera localization in structured environments. IEEE Transactions on Instrumentation and Measurement 69: 8962–8972

    Article  Google Scholar 

  9. Chiodini S, Giubilato R, Pertile M and Debei S 2020 Retrieving scale on monocular visual odometry using low-resolution range sensors. IEEE Transactions on Instrumentation and Measurement 69: 5875–5889

    Article  Google Scholar 

  10. Hui B, Wen G, Zhang P and Li D 2013 A novel line scan camera calibration technique with an auxiliary frame camera. IEEE Transactions on Instrumentation and Measurement 62: 2567–2575

    Article  Google Scholar 

  11. Doumanoglou A, Asteriadis S, Alexiadis D S, Zarpalas D and Daras P 2013 A dataset of Kinect-based 3D scans. In: IVMSP, pp. 1–4

  12. Mao Q R, Pan X Y, Zhan Y Z and Shen X J 2015 Using Kinect for real-time emotion recognition via facial expressions. Frontiers of Information Technology and Electronic Engineering 16: 272–282

    Article  Google Scholar 

  13. Zhang J, Huang D, Wang Y and Sun J 2016 Lock3dface: a large-scale database of low-cost Kinect 3d faces. In: International Conference on Biometrics, pp. 1–8

  14. Li B Y, Mian A S, Liu W and Krishna A 2013 Using Kinect for face recognition under varying poses, expressions, illumination and disguise. In: IEEE Workshop on Applications of Computer Vision, pp. 186–192

  15. Min R, Kose N and Dugelay J L 2014 Kinectfacedb: a kinect database for face recognition. IEEE Transactions on Systems, Man, and Cybernetics: Systems 44: 1534–1548

    Article  Google Scholar 

  16. Hg R I, Jasek P, Rofidal C, Nasrollahi K, Moeslund T B and Tranchet G 2012 An RGB-d database using Microsoft’s Kinect for Windows for face detection. In: Eighth International Conference on Signal Image Technology and Internet-Based Systems, pp. 42–46

  17. Gupta S, Markey M K and Bovik A C 2010 Anthropometric 3D face recognition. International Journal of Computer Vision 90: 331–349

    Article  Google Scholar 

  18. Lusi I, Escarela S and Anbarjafari G 2016 Sase: Rgb-depth database for human head pose estimation. In: Computer Vision–ECCV 2016 Workshops. Part III, pp. 325–336

  19. Goswami G, Vatsa M and Singh R 2014 RGB-D face recognition with texture and attribute features. IEEE Transactions on Information Forensics and Security 9: 1629–1640

    Article  Google Scholar 

  20. Cao C, Weng Y, Zhou S, Tong Y and Zhou K 2013 Facewarehouse: a 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics 20: 413–425

    Google Scholar 

  21. Deng J, Dong W, Socher R, Li L J, Li K and Fei-Fei L 2009 Imagenet: a large-scale hierarchical image database. In: IEEE Conference On Computer Vision and Pattern Recognition, pp. 248–255

  22. Piemontez R A and Comunello E 2020 RAP3DF-One shoot 3D face dataset. Data in Brief 32: 106281

    Article  Google Scholar 

  23. Zhang Z 2000 A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22: 1330–1334

    Article  Google Scholar 

  24. Su Y, Gao W, Liu Z, Sun S and Fu Y 2020 Hybrid marker-based object tracking using Kinect v2. IEEE Transactions on Instrumentation and Measurement 69: 6436–6445

    Article  Google Scholar 

  25. Bagchi P, Dutta K, Bhattacharjee D and Nasipuri M 2016 Face range image dataset: acquisition, processing and evaluation. In: 5th International Congress on Engineering and Information (ICEAI), Osaka, Japan

  26. Bagchi P, Nasipuri M, Bhattacharjee D and Basu D K 2012 A novel approach for nose tip detection using smoothing by weighted median filtering applied to 3D face images in variant poses. In: International Conference on Pattern Recognition, Informatics and Medical Engineering, pp. 272–277

  27. Bagchi P, Bhattacharjee D and Nasipuri M 2019 Reg3DFacePtCd: registration of 3D point clouds using a common set of landmarks for alignment of human face images. KI-Künstliche Intelligenz 33: 369–387

    Article  Google Scholar 

  28. Perakis P, Passalis G, Theoharis T and Kakadiaris I A 2012 3D facial landmark detection under large yaw and expression variations. IEEE Transactions on Pattern Analysis and Machine Intelligence 35: 1552–1564

    Article  Google Scholar 

  29. Conde C, Cipolla R, Rodríguez-Aragón L J, Serrano Á and Cabello E 2005 3D facial feature location with spin images. In: MVA, pp. 418–421

  30. Moreno A 2004 Gavab D B: a 3D face database. In: 2nd COST275 Workshop on Biometrics on the Internet, pp. 75–80

  31. Whitmarsh T, Veltkamp R C, Spagnuolo M, Marini S and ter Haar F B 2006 Landmark detection on 3d face scans by facial model registration. In: 1st International Symposium on Shapes and Semantics, pp. 71–75

  32. Gokberk B, Ali Salah A, Akarun L, Etheve R, Riccio D and Dugelay J L 2009 3D face recognition. In: Guide to Biometric Reference Systems and Performance Evaluation, pp. 263–295

  33. Vishwakarma V P, Dalal S and Sisaudia V 2018 Efficient feature extraction using DWT-DCT for robust face recognition under varying illuminations. In: 2nd International Conference on Power Electronics, Intelligent Control and Energy Systems, pp. 982–987

  34. Passalis G, Perakis P, Theoharis T and Kakadiaris I A 2011 Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 33: 1938–1951

    Article  Google Scholar 

  35. Tang H, Yin B, Sun Y and Hu Y 2013 3D face recognition using local binary patterns. Signal Processing 93: 2190–2198

    Article  Google Scholar 

  36. Shekar B H, Harivinod N, Kumari M S and Holla K R 2011 3D face recognition using significant point based SULD descriptor. In: International Conference on Recent Trends in Information Technology, pp. 981–986

  37. Alom M Z, Taha T M, Yakopcic C, Westberg S, Sidike P, Nasrin M S and Asari V K 2018 The history began from alexnet: a comprehensive survey on deep learning approaches. arXiv preprint https://arxiv.org/abs/1803.01164

  38. http://www.frav.es/index.php/en/

  39. Boughrara H, Chtourou M and Amar C B 2012 MLP neural network based face recognition system using constructive training algorithm. In: International Conference on Multimedia Computing and Systems, pp. 233–238

Download references

Acknowledgments

The work has been supported by a grant from DeiTy, MCIT, Govt. of India, a project entitled, “Development of three-Dimensional Face Recognition Techniques Based on Range Images”. The authors hereby thank all the participants who had readily agreed to be a part of this huge database. We are very grateful to the faculty members, staff members, students, full-time research scholars of the CSE Department of Jadavpur University as well as the faculties of some of the private engineering colleges and part-time Ph.D. students outside Jadavpur University who readily agreed to give us their precious time for the purpose of acquisition. The authors are thankful to the CMATER Lab, Department of Computer Science and Engineering, Jadavpur University, for providing the facilities where the acquisition took place. The authors also express special thanks to Mr. Koushik Dutta for his continuous support during the acquisition of the database.

Author information

Authors and Affiliations

  1. Department of CSE, RCC Institute of Information Technology, Kolkata, West Bengal, India

    Parama Bagchi

  2. Department of CSE, Jadavpur University, Kolkata, West Bengal, India

    Debotosh Bhattacharjee & Mita Nasipuri

  3. Center for Basic and Applied Science, Faculty of Informatics and Management, University of Hradec Kralove, Hradec Kralove, Czech Republic

    Ondrej Krejcar

Authors
  1. Parama Bagchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debotosh Bhattacharjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mita Nasipuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ondrej Krejcar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Parama Bagchi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bagchi, P., Bhattacharjee, D., Nasipuri, M. et al. Registration of 3D face images of JUDeiTy3DK database using POFaceReg8 algorithm. Sādhanā 48, 158 (2023). https://doi.org/10.1007/s12046-023-02225-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12046-023-02225-w

Keywords

Search

Navigation