International Journal of Computer Vision

, Volume 113, Issue 2, pp 128–142 | Cite as

Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors

  • Huibin Li
  • Di Huang
  • Jean-Marie Morvan
  • Yunhong Wang
  • Liming Chen
Article

Abstract

Registration algorithms performed on point clouds or range images of face scans have been successfully used for automatic 3D face recognition under expression variations, but have rarely been investigated to solve pose changes and occlusions mainly since that the basic landmarks to initialize coarse alignment are not always available. Recently, local feature-based SIFT-like matching proves competent to handle all such variations without registration. In this paper, towards 3D face recognition for real-life biometric applications, we significantly extend the SIFT-like matching framework to mesh data and propose a novel approach using fine-grained matching of 3D keypoint descriptors. First, two principal curvature-based 3D keypoint detectors are provided, which can repeatedly identify complementary locations on a face scan where local curvatures are high. Then, a robust 3D local coordinate system is built at each keypoint, which allows extraction of pose-invariant features. Three keypoint descriptors, corresponding to three surface differential quantities, are designed, and their feature-level fusion is employed to comprehensively describe local shapes of detected keypoints. Finally, we propose a multi-task sparse representation based fine-grained matching algorithm, which accounts for the average reconstruction error of probe face descriptors sparsely represented by a large dictionary of gallery descriptors in identification. Our approach is evaluated on the Bosphorus database and achieves rank-one recognition rates of 96.56, 98.82, 91.14, and 99.21 % on the entire database, and the expression, pose, and occlusion subsets, respectively. To the best of our knowledge, these are the best results reported so far on this database. Additionally, good generalization ability is also exhibited by the experiments on the FRGC v2.0 database.

Keywords

Registration-free 3D face recognition Expression, pose and occlusion 3D keypoint descriptors Fine-grained matching 

References

  1. Al-Osaimi, F., Bennamoun, M., & Mian, A. (2009). An expression deformation approach to non-rigid 3d face recognition. International Journal of Computer Vision, 81(3), 302–316.CrossRefGoogle Scholar
  2. Alyüz, N., Gökberk. B., & Akarun. L. (2008). A 3d face recognition system for expression and occlusion invariance. In: 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) (pp 1–7).Google Scholar
  3. Alyüz, N., Gökberk, B., & Akarun, L. (2010). Regional registration for expression resistant 3-d face recognition. IEEE Transactions on Information Forensics and Security, 5(3), 425–440.CrossRefGoogle Scholar
  4. Alyüz, N., Gokberk, B., & Akarun, L. (2013). 3-d face recognition under occlusion using masked projection. IEEE Transactions on Information Forensics and Security, 8(5), 789–802.Google Scholar
  5. Berretti, S., Werghi, N., del Bimbo, A., & Pala, P. (2013). Matching 3d face scans using interest points and local histogram descriptors. Computers and Graphics, 37(5), 509–525.CrossRefGoogle Scholar
  6. Bowyer, K. W., Chang, K., & Flynn, P. (2006). A survey of approaches and challenges in 3d and multi-modal 3d+2d face recognition. Computer Vision and Image Understanding, 101, 1–15.CrossRefGoogle Scholar
  7. Colombo, A., Cusano, C., & Schettini, R. (2011). Three-dimensional occlusion detection and restoration of partially occluded faces. Journal of Mathematical Imaging and Vision, 40(1), 105–119.CrossRefMATHMathSciNetGoogle Scholar
  8. Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., & Slama, R. (2013). 3d face recognition under expressions, occlusions, and pose variations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(9), 2270–2283.Google Scholar
  9. Faltemier, T. C., Bowyer, K. W., & Flynn, P. J. (2008). A region ensemble for 3d face recognition. IEEE Transactions on Information Forensics and Security, 3(1), 62–73.CrossRefGoogle Scholar
  10. Goldfeather, J., & Interrante, V. (2004). A novel cubic-order algorithm for approximating principal direction vectors. ACM Transactions Graphics, 23(1), 45–63.CrossRefGoogle Scholar
  11. Guo, Z., Zhang, Y., Xia, Y., Lin, Z., Fan, Y., & Feng, D. (2013). Multi-pose 3d face recognition based on 2d sparse representation. Journal of Visual Communication and Image Representation, 24(2), 1047–3203.CrossRefGoogle Scholar
  12. Huang, D., Ardabilian, M., Wang, Y., & Chen, L. (2012). 3-d face recognition using elbp-based facial description and local feature hybrid matching. IEEE Transactions on Information Forensics and Security, 7(5), 1551–1565.CrossRefGoogle Scholar
  13. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cats visual cortex. The Journal of Physiology, 160, 106–154.CrossRefGoogle Scholar
  14. Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.CrossRefGoogle Scholar
  15. Kakadiaris, I. A., Passalis, G., Toderici, G., Murtuza, M. N., Lu, Y., Karampatziakis, N., et al. (2007). Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 640–649.CrossRefGoogle Scholar
  16. Li, H., Huang, D., Lemaire, P., Morvan, J., & Chen, L. (2011). Expression robust 3d face recognition via mesh-based histograms of multiple order surface differential quantities. In: Proceedings of IEEE International Conference on Image Processing (ICIP) (pp. 4019–4023).Google Scholar
  17. Li, H., Huang, D., Morvan, J. M., Chen, L., & Wang, Y. (2014). Expression-robust 3d face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing, 133, 179–193.CrossRefGoogle Scholar
  18. Li, S. Z., & Jain, A. K. (2005). Handbook of Face Recognition. Secaucus, NJ: Springer-Verlag New York Inc.MATHGoogle Scholar
  19. Li, X., Jia, T., & Zhang, H. (2009). Expression-insensitive 3d face recognition using sparse representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2575–2582).Google Scholar
  20. Liao, S., Jain, A., & Li, S. (2013). Partial face recognition: Alignment-free approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(5), 1193–1205.CrossRefGoogle Scholar
  21. Lo, T., & Siebert, J. (2009). Local feature extraction and matching on range images: 2.5d sift. Computer Vision and Image Understanding, 113(12), 1235–1250.CrossRefGoogle Scholar
  22. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.CrossRefGoogle Scholar
  23. Maes, C., Fabry, T., Keustermans, J., Smeets, D., Suetens, P., & Vandermeulen, D. (2010). Feature detection on 3d face surfaces for pose normalisation and recognition. In: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp 1–6.Google Scholar
  24. Masi, I., Lisanti, G., Bagdanov, A., Pala, P., & Del Bimbo, A. (2013). Using 3d models to recognize 2d faces in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (pp. 775–780).Google Scholar
  25. Meyer, T., Eriksson, M., & Maggio, R. (2001). Gradient estimation from irregularly spaced data sets. Mathematical Geology, 33, 693–717.CrossRefMATHGoogle Scholar
  26. Mian, A. S., Bennamoun, M., & Owens, R. A. (2007). An efficient multimodal 2d–3d hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 1927–1943.CrossRefGoogle Scholar
  27. Mian, A. S., Bennamoun, M., & Owens, R. A. (2008). Keypoint detection and local feature matching for textured 3d face recognition. International Journal of Computer Vision, 79(1), 1–12.CrossRefGoogle Scholar
  28. Mohammadzade, H., & Hatzinakos, D. (2013). Iterative closest normal point for 3d face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2), 381–397.CrossRefGoogle Scholar
  29. Ocegueda, O., Passalis, G., Theoharis, T., Shah, S., & Kakadiaris, I. (2011). Ur3d-c: Linear dimensionality reduction for efficient 3d face recognition. In: Proceedings of IEEE International Joint Conference on Biometrics (IJCB).Google Scholar
  30. Passalis, G., Perakis, P., Theoharis, T., & Kakadiaris, I. (2011). Using facial symmetry to handle pose variations in real-world 3d face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(10), 1938–1951.CrossRefGoogle Scholar
  31. Pati, Y.C., Rezaiifar, R., & Krishnaprasad, P.S. (1993). Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proceedings of 27th Asilomar Conference on Signals, Systems and Computers (ACSSC).Google Scholar
  32. Phillips, P., Flynn, P. J., Scruggs, T., Bowyer, K., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005). Overview of the face recognition grand challenge. In: Proeedings of of IEEE conference of Computer Vision and Pattern Recognition (CVPR).Google Scholar
  33. Queirolo, C., Silva, L., Bellon, O., & Segundo, M. (2010). 3d face recognition using simulated annealing and the surface interpenetration measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2), 206–219.CrossRefGoogle Scholar
  34. Savran, A., Alyüz, N., Dibeklioglu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008). 3d face recognition benchmarks on the bosphorus database with focus on facial expressions. In: Proceedings of Workshop on Biometrics and Identity Management (WBIM).Google Scholar
  35. Skellya, L. J., & Sclaroffb, S. (2007). Improved feature descriptors for 3-d surface matching. Proceedings of of SPIE Two- and Three-Dimensional Methods for Inspection and Metrology, 6762, 1–12.Google Scholar
  36. Smeets, D., Claes, P., Hermans, J., Vandermeulen, D., & Suetens, P. (2012). A comparative study of 3-d face recognition under expression variations. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 42(5), 710–727.Google Scholar
  37. Smeets, D., Keustermans, J., Vandermeulen, D., & Suetens, P. (2013). meshsift: Local surface features for 3d face recognition under expression variations and partial data. Computer Vision and Image Understanding, 117(2), 158–169.Google Scholar
  38. Spreeuwers, L. (2011). Fast and accurate 3d face recognition using registration to an intrinsic coordinate system and fusion of multiple region classifiers. International Journal of Computer Vision, 93(3), 389–414.CrossRefMATHGoogle Scholar
  39. Szeptycki, P., Ardabilian, M., & Chen, L. (2009). A coarse-to-fine curvature analysis-based rotation invariant 3d face landmarking. In: International Conference on Biometrics: Theory, Applications and Systems (ICB) (pp. 3206–3211).Google Scholar
  40. Tola, E., Lepetit, V., & Fua, P. (2010). Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(5), 815–830.CrossRefGoogle Scholar
  41. Tombari, F., Salti, S., & Di Stefano, L. (2013). Performance evaluation of 3d keypoint detectors. International Journal of Computer Vision, 102(1–3), 198–220.CrossRefGoogle Scholar
  42. Veltkamp, R.C., van Jole, S., Drira, H., Amor, B.B., Daoudi, M., Li, H., Chen, L., Claes, P., Smeets, D., Hermans, J., Vandermeulen, D., & Suetens, P. (2011). Shrec ’11 track: 3d face models retrieval. In: Euro-graphics Workshop on 3D Object Retrieval (3DOR) (pp. 89–95).Google Scholar
  43. Wang, Y., Liu, J., & Tang, X. (2010). Robust 3d face recognition by local shape difference boosting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(10), 1858–1870.CrossRefGoogle Scholar
  44. Wright, J., Yang, A. Y., Ganesh, A., Sastry, S. S., & Ma, Y. (2009). Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), 210–227.CrossRefGoogle Scholar
  45. Yang, A.Y., Wright, J., Ma, Y., & Sastry, S.S. (2007). Feature selection in face recognition: A sparse representation perspective. Tech. Rep. UCB/EECS-2007-99, EECS Department, University of California, Berkeley. http://www.eecs.berkeley.edu/Pubs/TechRpts/2007/EECS-2007-99.html.
  46. Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 373–380).Google Scholar
  47. Zaharescu, A., Boyer, E., & Horaud, R. P. (2012). Keypoints and local descriptors of scalar functions on 2d manifolds. International Journal of Computer Vision, 100(1), 78–98.CrossRefMATHGoogle Scholar
  48. Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Computing Surveys, 35, 399–458.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Huibin Li
    • 1
    • 2
  • Di Huang
    • 3
  • Jean-Marie Morvan
    • 4
    • 5
  • Yunhong Wang
    • 3
  • Liming Chen
    • 6
  1. 1.School of Mathematics and StatisticsXi’an Jiaotong UniversityShaanxiPeople‘s Republic of China
  2. 2.Beijing Center for Mathematics and Information Interdisciplinary Sciences (BCMIIS)BeijingPeople‘s Republic of China
  3. 3.Laboratory of Intelligent Recognition and Image Processing, School of Computer Science and EngineeringBeihang UniversityBeijingPeople‘s Republic of China
  4. 4.Département de MathématiquesUniversité Claude Bernard Lyon 1LyonFrance
  5. 5.Geometric Modeling and Scientific Visualization CenterKing Abdullah University of Science and TechnologyMakkahSaudi Arabia
  6. 6.Département de Mathématiques et InformatiqueUMR CNRS 5205, Ecole Centrale LyonLyonFrance

Personalised recommendations