Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors
- 1.7k Downloads
Registration algorithms performed on point clouds or range images of face scans have been successfully used for automatic 3D face recognition under expression variations, but have rarely been investigated to solve pose changes and occlusions mainly since that the basic landmarks to initialize coarse alignment are not always available. Recently, local feature-based SIFT-like matching proves competent to handle all such variations without registration. In this paper, towards 3D face recognition for real-life biometric applications, we significantly extend the SIFT-like matching framework to mesh data and propose a novel approach using fine-grained matching of 3D keypoint descriptors. First, two principal curvature-based 3D keypoint detectors are provided, which can repeatedly identify complementary locations on a face scan where local curvatures are high. Then, a robust 3D local coordinate system is built at each keypoint, which allows extraction of pose-invariant features. Three keypoint descriptors, corresponding to three surface differential quantities, are designed, and their feature-level fusion is employed to comprehensively describe local shapes of detected keypoints. Finally, we propose a multi-task sparse representation based fine-grained matching algorithm, which accounts for the average reconstruction error of probe face descriptors sparsely represented by a large dictionary of gallery descriptors in identification. Our approach is evaluated on the Bosphorus database and achieves rank-one recognition rates of 96.56, 98.82, 91.14, and 99.21 % on the entire database, and the expression, pose, and occlusion subsets, respectively. To the best of our knowledge, these are the best results reported so far on this database. Additionally, good generalization ability is also exhibited by the experiments on the FRGC v2.0 database.
KeywordsRegistration-free 3D face recognition Expression, pose and occlusion 3D keypoint descriptors Fine-grained matching
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 11401464, 61202237 and 61273263; the China Postdoctoral Science Foundation (No. 2014M560785); the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20121102120016); the French research agency, Agence Nationale de la Recherche (ANR) under Grant ANR-07-SESU-004, ANR-2010-INTB-0301-01 and ANR-13-INSE-0004-02; the joint project by the LIA 2MCSI lab between the group of Ecoles Centrales and Beihang University; and the Fundamental Research Funds for the Central Universities. We would like to thank the Bosphorus (Savran et al. 2008) and the FRGC (Phillips et al. 2005) organizers for the face data, Peyré for the Toolbox Fast Marching.
- Alyüz, N., Gökberk. B., & Akarun. L. (2008). A 3d face recognition system for expression and occlusion invariance. In: 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) (pp 1–7).Google Scholar
- Alyüz, N., Gokberk, B., & Akarun, L. (2013). 3-d face recognition under occlusion using masked projection. IEEE Transactions on Information Forensics and Security, 8(5), 789–802.Google Scholar
- Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., & Slama, R. (2013). 3d face recognition under expressions, occlusions, and pose variations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9), 2270–2283.Google Scholar
- Kakadiaris, I. A., Passalis, G., Toderici, G., Murtuza, M. N., Lu, Y., Karampatziakis, N., et al. (2007). Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 640–649.CrossRefGoogle Scholar
- Li, H., Huang, D., Lemaire, P., Morvan, J., & Chen, L. (2011). Expression robust 3d face recognition via mesh-based histograms of multiple order surface differential quantities. In: Proceedings of IEEE International Conference on Image Processing (ICIP) (pp. 4019–4023).Google Scholar
- Li, X., Jia, T., & Zhang, H. (2009). Expression-insensitive 3d face recognition using sparse representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2575–2582).Google Scholar
- Maes, C., Fabry, T., Keustermans, J., Smeets, D., Suetens, P., & Vandermeulen, D. (2010). Feature detection on 3d face surfaces for pose normalisation and recognition. In: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp 1–6.Google Scholar
- Masi, I., Lisanti, G., Bagdanov, A., Pala, P., & Del Bimbo, A. (2013). Using 3d models to recognize 2d faces in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (pp. 775–780).Google Scholar
- Ocegueda, O., Passalis, G., Theoharis, T., Shah, S., & Kakadiaris, I. (2011). Ur3d-c: Linear dimensionality reduction for efficient 3d face recognition. In: Proceedings of IEEE International Joint Conference on Biometrics (IJCB).Google Scholar
- Pati, Y.C., Rezaiifar, R., & Krishnaprasad, P.S. (1993). Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proceedings of 27th Asilomar Conference on Signals, Systems and Computers (ACSSC).Google Scholar
- Phillips, P., Flynn, P. J., Scruggs, T., Bowyer, K., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005). Overview of the face recognition grand challenge. In: Proeedings of of IEEE conference of Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Savran, A., Alyüz, N., Dibeklioglu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008). 3d face recognition benchmarks on the bosphorus database with focus on facial expressions. In: Proceedings of Workshop on Biometrics and Identity Management (WBIM).Google Scholar
- Skellya, L. J., & Sclaroffb, S. (2007). Improved feature descriptors for 3-d surface matching. Proceedings of of SPIE Two- and Three-Dimensional Methods for Inspection and Metrology, 6762, 1–12.Google Scholar
- Smeets, D., Claes, P., Hermans, J., Vandermeulen, D., & Suetens, P. (2012). A comparative study of 3-d face recognition under expression variations. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 42(5), 710–727.Google Scholar
- Smeets, D., Keustermans, J., Vandermeulen, D., & Suetens, P. (2013). meshsift: Local surface features for 3d face recognition under expression variations and partial data. Computer Vision and Image Understanding, 117(2), 158–169.Google Scholar
- Szeptycki, P., Ardabilian, M., & Chen, L. (2009). A coarse-to-fine curvature analysis-based rotation invariant 3d face landmarking. In: International Conference on Biometrics: Theory, Applications and Systems (ICB) (pp. 3206–3211).Google Scholar
- Veltkamp, R.C., van Jole, S., Drira, H., Amor, B.B., Daoudi, M., Li, H., Chen, L., Claes, P., Smeets, D., Hermans, J., Vandermeulen, D., & Suetens, P. (2011). Shrec ’11 track: 3d face models retrieval. In: Euro-graphics Workshop on 3D Object Retrieval (3DOR) (pp. 89–95).Google Scholar
- Yang, A.Y., Wright, J., Ma, Y., & Sastry, S.S. (2007). Feature selection in face recognition: A sparse representation perspective. Tech. Rep. UCB/EECS-2007-99, EECS Department, University of California, Berkeley. http://www.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-99.html.
- Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 373–380).Google Scholar