Abstract
Face recognition using 3D cameras captures and processes explicit 3D shape information and is more resistant to spoofing security attacks than 2D face recognition. However, typical compact-baseline, triangulation-based 3D capture systems are generally only suitable for close-range applications. The 3D shape may be used alone, or in combination with simultaneously captured, standard 2D color images in multimodal 2D/3D face recognition. Like its 2D counterpart, modern 3D face recognition often uses a feature extraction phase followed by a classification phase. Traditionally, such features have been handcrafted, but recent approaches use the representational learning power of deep neural networks. Other approaches use shape alignment methods, or analysis-by-synthesis, which exploits the prior shape distribution information captured by 3D morphable models. Additionally, 3D face models have been used for pose correction and calculation of the facial albedo map, which is independent of illumination. Finally, 3D face recognition has also achieved significant success toward expression invariance by modeling nonrigid surface deformations, removing facial expressions or by using parts-based face recognition. This chapter is not intended to be a comprehensive survey. Rather, it gives a tutorial introduction to 3D face recognition and a representative sample of both well-established and more recent state-of-the-art 3D face recognition techniques in terms of their implementation and expected performance on benchmark datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This may be referred to as a 3D scan or a 3D image, depending on the mode of shape capture.
- 2.
Note that liveness detection, such as the detection of blinking, can be used to augment security [76] in both 2D and 3D modalities.
- 3.
3D points are structured in a rectangular array and since (x, y) values are included, strictly it is not a range image, which contains z-values only.
- 4.
Singular Inversions, Facegen Modeller, www.facegen.com.
References
Achermann, B., Jiang, X., Bunke, H.: Face recognition using range images. In: Proceedings of International Conference on Virtual Systems and MultiMedia VSMM ’97 (Cat. No.97TB100182), pp. 129–136 (1997)
Adini, Y., Moses, Y., Ullman, S.: Face recognition: the problem of compensating for changes in illumination direction. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 721–732 (1997)
Al-Osaimi, F., Bennamoun, M., Mian, A.: Integration of local and global geometrical cues for 3D face recognition. Pattern Recognit. 41(3), 1030–1040 (2008)
Angel, E.: Interactive Computer Graphics: A Top-Down Approach Using OpenGL, 5th edn. Addison-Wesley Publishing Company, Reading (2008)
Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3D point sets. IEEE Trans. Pattern Anal. Mach. Intell. 9(5), 698–700 (1987)
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)
Besl, P.J., McKay, N.D.: A method for registration of 3D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, Berlin (2006)
Blanz, V., Scherbaum, K., Seidel, H.: Fitting a morphable model to 3D scans of faces. In: Proceedings of 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co., USA (1999)
Bledsoe, W.W.: The model method in facial recognition, technical report pri 15. Technical Report, Panoramic Research, Inc. Palo Alto, California, USA (1964)
Bogazici University: The Bosphorus 3D face database. http://bosphorus.ee.boun.edu.tr/. Accessed 21 Jan 2020
Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Three-dimensional face recognition. Int. J. Comput. Vis. 64(1), 5–30 (2005)
Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20(3), 413–425 (2014)
Chang, J.K., Bowyer, K., Flynn, P.: Face recognition using 2D and 3D facial data. In: Proceedings of Multimodal User Authentication Workshop, pp. 25–32 (2004)
Chang, K.I., Bowyer, K.W., Flynn, P.J.: Multiple nose region matching for 3D face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)
Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multimodal 2D+3D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 619–624 (2005)
Chua, C.-S., Han, F., Ho, Y.-K.: 3D human face recognition using point signature. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 233–238 (2000)
Chua, C.S., Jarvis, R.: Point signatures: a new representation for 3D object recognition. Int. J. Comput. Vis. 25(1), 63–85 (1997)
Colombo, A., Cusano, C., Schettini, R.: UMB-DB: A database of partially occluded 3D faces. In: Proceedings of 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2113–2119 (2011)
Colombo, A., Cusano, C., Schettini, R.: 3D face detection using curvature analysis. Pattern Recognit. 39(3), 444–455 (2006)
Creusot, C., Pears, N.E., Austin, J.: 3D face landmark labelling. In: Proceedings of 1st ACM workshop on 3D object retrieval, pp. 27–32 (2010)
Creusot, C., Pears, N., Austin, J.: Automatic keypoint detection on 3D faces using a dictionary of local shapes. In: Proceedings of 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pp. 204–211 (2011)
Creusot, C.: Automatic landmarking for non-cooperative 3D face recognition. Ph.D. Thesis, Department of Computer Science, University of York, UK (2011)
Creusot, C., Pears, N., Austin, J.: A machine-learning approach to keypoint detection and landmarking on 3D meshes. Int. J. Comput. Vis. 102(1), 146–179 (2013)
Deng, J., Roussos, A., Chrysos, G., Ververas, E., Kotsia, I., Shen, J., Zafeiriou, S.: The menpo benchmark for multi-pose 2D and 3D facial landmark localisation and tracking. Int. J. Comput. Vis. 1–26 (2018)
Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2270–2283 (2013)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2001)
Emambakhsh, M., Evans, A.: Nasal patches and curves for expression-robust 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 995–1007 (2017)
Faltemier, T.C., Bowyer, K.W., Flynn, P.J.: Using a multi-instance enrollment representation to improve 3D face recognition. In: Proceedings of 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–6 (2007)
Faltemier, T.C., Bowyer, K.W., Flynn, P.J.: A region ensemble for 3D face recognition. IEEE Trans. Inf. Forensics Secur. 3(1), 62–73 (2008)
Farkas, L.G.: Anthropometry of the Head and Face. Raven Press, New York (1994)
Fisher, N.I., Lee, A.J.: Correlation coefficients for random variables on a unit sphere or hypersphere. Biometrika 73(1), 159–164 (1986)
Fitzgibbon, A.: Robust registration of 2D and 3D point sets. Image Vis. Comput. 21, 1145–1153 (2002)
Fleishman, S., Drori, I., Cohen-Or, D.: Bilateral mesh denoising. ACM Trans. Graph. 22(3), 950–953 (2003)
Gao, H., Davis, J.W.: Why direct lda is not equivalent to lda. Pattern Recognit. 39(5), 1002–1006 (2006)
Garland, M., Heckbert, P.: Surface simplification using quadric error metrics. In: Proceedings of the ACM SIGGRAPH Conference on Computer Graphics (1997)
Gilani, S.Z., Mian, A.: Learning from millions of 3D scans for large-scale 3D face recognition. In: Proceedings of International Conference Computer Vision and Pattern Recognition (2018)
Gilani, S.Z., Mian, A., Shafait, F., Reid, I.: Dense 3D face correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 40(7), 1584–1598 (2018)
Gökberk, B., İrfanoğlu, M.O., Akarun, L.: 3D shape-based face representation and feature extraction for face recognition. Image Vis. Comput. 24(8), 857–869 (2006)
Gordon, G.G.: Face recognition based on depth and curvature features. In: Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 808–810 (1992)
Grgic, M., Delac, K.: Face recognition homepage. http://www.face-rec.org. Accessed 21 Jan 2020
Günther, M., Cruz, S., Rudd, E.M., Boult, T.E.: Toward open-set face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRw), pp. 573–582 (2017)
Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J., Kwok, N.M.: A comprehensive performance evaluation of 3D local feature descriptors. Int. J. Comput. Vis. 116(1), 66–89 (2016)
Gupta, S., Markey, M.K., Bovik, A.C.: Anthropometric 3D face recognition. Int. J. Comput. Vis. 90(3), 331–349 (2010)
Heckbert, P., Garl, M.: Survey of polygonal surface simplification algorithms. In: SIGGRAPH, Course Notes: Multiresolution Surface Modeling (1997)
Heseltine, T., Pears, N.E., Austin, J.: Three-dimensional face recognition: a fishersurface approach. In: Proceedings of the International Conference Image Analysis and Recognition, pp. 684–691 (2004)
Heseltine, T., Pears, N.E., Austin, J.: Three-dimensional face recognition: an eigensurface approach. In: Proceedings of the IEEE International Conference Image Processing, Singapore, 2004 (ICIP’04), pp. 1421–1424 (2004)
Heseltine, T., Pears, N.E., Austin, J.: Three-dimensional face recognition using combinations of surface feature map subspace components. Image Vis. Comput. 26(3), 382–396 (2008)
Hesher, C., Srivastava, A., Erlebacher, G.: A novel technique for face recognition using range imaging. In: Proceedings of Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings., vol. 2, pp. 201–204(2003)
Horn, B.: Robot Vision, chapter 16. MIT Press, Cambridge (1986)
Huang, G., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07–49 (2007)
Institute of Automation, Chinese Academy of Sciences (CASIA): biometrics ideal test: CASIA-3D FaceV1. http://biometrics.idealtest.org. Accessed 21 Jan 2020
Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 4–20 (2004)
Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)
Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)
Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The megaface benchmark: 1 million faces for recognition at scale. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4873–4882 (2016)
Kim, D., Hernandez, M., Choi, J., Medioni, G.: Deep 3D face identification. In: Proceedings of 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 133–142 (2017)
Klassen, E., Srivastava, A., Mio, M., Joshi, S.H.: Analysis of planar shapes using geodesic paths on shape spaces. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 372–383 (2004)
Koenderink, J.J., van Doorn, A.J.: Surface shape and curvature scales. Image Vis. Comput. 10(8), 557–565 (1992)
Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. Neural Inf. Process. Syst. 25 (2012)
Lee, J., Milios, E.: Matching range images of human faces. In: Proceedings of Third International Conference on Computer Vision, pp. 722–726. IEEE Computer Society (1990)
Lee, Y.-H., Shim, J.-C.: Curvature based human face recognition using depth weighted hausdorff distance. In: Proceedings of 2004 International Conference on Image Processing, 2004, vol. 3, pp. 1429–1432 (2004)
Li, H., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113(2), 128–142 (2015)
Lo, T.W.R., Siebert, J.P.: Local feature extraction and matching on range images: 2.5D SIFT. Comput. Vis. Image Underst. 113(12), 1235–1250 (2009)
Lu, X., Jain, A.K., Colbry, D.: Matching 2.5d face scans to 3D models. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 31–43 (2006)
Lu, X., Jain, A.: Deformation modeling for robust 3D face matching. IEEE Trans. Pattern Anal. Mach. Intell. 30(8), 1346–1357 (2008)
Max, N.: Weights for computing vertex normals from facet normals. J. Graph. Tools 4(2), 1–6 (1999)
MeshLab: MeshLab. Open source system for processing and editing 3D triangular meshes. http://www.meshlab.net. Accessed 22 Jan 2020
Mian, A., Bennamoun, M., Owens, R.: An efficient multimodal 2D–3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)
Mian, A., Bennamoun, M., Owens, R.: Keypoint detection and local feature matching for textured 3D face recognition. Int. J. Comput. Vis. 79, 1–12 (2008)
Mohammadzade, H., Hatzinakos, D.: Iterative closest normal point for 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 381–397 (2013)
Ng, H., Winkler, S.: A data-driven approach to cleaning large face datasets. In: Proceedings of 2014 IEEE International Conference on Image Processing, pp. 343–347 (2014)
Padia, C., Pears, N.E.: A review and characterization of icp-based symmetry plane localisation in 3D face data. Technical Report 463, Department of Computer Science, University of York (2011)
Pan, G., Han, S., Wu, Z., Wang, Y.: 3D face recognition using mapped depth images. In: Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) - Workshops, p. 175 (2005)
Pan, G., Wu, Z., Sun, L.: Recent Advances in Face Recognition (chap.: Liveness detection for face recognition), pp. 236–254. Intech Open, London (2008)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference 2015, pp. 41.1–41.12. BMVA Press (2015)
Passalis, G., Kakadiaris, I.A., Theoharis, T., Toderici, G., Murtuza, N.: Evaluation of the ur3d algorithm using the frgc v2 data set. In: Proceedings of the IEEE Workshop on Face Recognition Grand Challenge Experiments (2005)
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Proceedings of 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 296–301 (2009)
Pears, N.E., Heseltine, T.: Isoradius contours: New representations and techniques for 3D face registration and matching. In: Proceedings of 3rd International Symposium on 3D Data Processing, Visualization, and Transmission, Chapel Hill, NC, USA, pp. 176–183 (2006)
Pears, N.E., Heseltine, T., Romero, M.: From 3D point clouds to pose normalised depth maps. Int. J. Comput. Vis. 89(2), 152–176 (2010)
Phillips, P.J., Flynn, P., Scruggs, T., Bowyer, K., Chang, J.K., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 947–954 (2005)
Phillips, P.J., Yates, A., Hu, Y., Hahn, C., Noyes, E., Jackson, K., Cavazos, J., Jeckeln, G., Ranjan, R., Sankar, S., Chen, J.C., Castillo, C., Chellappa, R., White, D., O’Toole, A.: Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. Proc. Natl. Acad. Sci. 115, 201721,355 (2018)
Piegl, L., Tiller, W.: The NURBS Book. Monographs in Visual Communication, 2nd edn. Springer, Berlin (1997)
Queirolo, C.C., Silva, L., Bellon, O.R.P., Pamplona Segundo, M.: 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 206–219 (2010)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the icp algorithm. In: Proceedings of Third International Conference on 3D Digital Imaging and Modeling, pp. 145–152 (2001)
Samir, C., Srivastava, A., Daoudi, M.: Three-dimensional face recognition using shapes of facial curves. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1858–1863 (2006)
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3d face analysis. In: B. Schouten, N.C. Juul, A. Drygajlo, M. Tistarelli (eds.) Proceedings of European Workshop on Biometrics and Identity Management, pp. 47–56. Springer, Berlin (2008)
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2013)
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)
Silva, L., Bellon, O., Boyer, K.: Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 27, 762–776 (2005)
Sirovich, L., Kirby, M.: Low-dimensional procedure for the characterization of human faces. J. Opt. Soc. Amer. A, Opt. Image Sci. 4, 519–524 (1987)
Soltanpour, S., Boufama, B., Wu, Q.J.: A survey of local feature methods for 3D face recognition. Pattern Recognit. 72, 391–406 (2017)
Spreeuwers, L.: Fast and accurate 3D face recognition. Int. J. Comput. Vis. 93(3), 389–414 (2011)
Spreeuwers, L.J.: Breaking the 99% barrier: optimisation of three-dimensional face recognition. IET Biometrics 4, 169–178 (2015)
Sun, X., Rosin, P.L., Martin, R.R., Langbein, F.C.: Noise analysis and synthesis for 3D laser depth scanners. Graph. Models 71(2), 34–48 (2009)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
Tanaka, H.T., Ikeda, M., Chiaki, H.: Curvature-based face surface recognition using spherical correlation. Principal directions for curved object recognition. In: Proceedings of Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 372–377 (1998)
The Blender Foundation: Open source 3D suite. https://www.blender.org/. Accessed 21 Jan 2020
The University of Texas at Austin: Texas 3D face recognition database. http://live.ece.utexas.edu/research/texas3dfr. Accessed 22 Jan 2020
Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognit. Neurosci. 3(1), 71–86 (1991)
Vijayan, V., Bowyer, K.W., Flynn, P.J., Huang, D., Chen, L., Hansen, M., Ocegueda, O., Shah, S.K., Kakadiaris, I.A.: Twins 3D face recognition challenge. In: Proceedings of 2011 International Joint Conference on Biometrics (IJCB), pp. 1–7 (2011)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Xu, C., Wang, Y., Tan, T., Quan, L.: Automatic 3D face recognition combining global geometric features with local shape variation information. In: Proceedings of Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings., pp. 308–313 (2004)
Yan, P., Bowyer, K.W.: A fast algorithm for icp-based 3D shape biometrics. In: Proceedings of Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID’05), pp. 213–218 (2005)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: Proceedings of 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, pp. 1–6 (2008)
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: Proceedings of 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 211–216 (2006)
Yu, H., Yang, J.: A direct lda algorithm for high-dimensional data - with application to face recognition. Pattern Recognit. 34, 2067–2070 (2001)
Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: High-resolution capture for modeling and animation. In: Deng, Z., Neumann, U. (eds.) Data-Driven 3D Facial Animation, pp. 248–276. Springer, London (2008)
Zhou, S., Xiao, S.: 3D face recognition: a survey. Human-Centric Comput. Inf. Sci. 8(1), (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pears, N., Mian, A. (2020). 3D Face Recognition. In: Liu, Y., Pears, N., Rosin, P.L., Huber, P. (eds) 3D Imaging, Analysis and Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-44070-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-44070-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44069-5
Online ISBN: 978-3-030-44070-1
eBook Packages: Computer ScienceComputer Science (R0)