Skip to main content
SpringerLink
Log in

Blending 2D and 3D Face Recognition

  • Chapter
  • First Online:
Face Recognition Across the Imaging Spectrum

Abstract

Over the last decade, performance of face recognition algorithms systematically improved. This is particularly impressive when considering very large or challenging datasets such as the FRGC v2 or Labelled Faces in the Wild . A better analysis of the structure of the facial texture and shape is one of the main reasons of improvement in recognition performance. Hybrid face recognition methods , combining holistic and feature-based approaches, also allowed to increase efficiency and robustness. Both photometric information and shape information allow to extract facial features which can be exploited for recognition. However, both sources, grey levels of image pixels and 3D data , are affected by several noise sources which may impair the recognition performance. One of the main difficulties in matching 3D faces is the detection and localization of distinctive and stable points in 3D scans. Moreover, the large amount of data (tens of thousands of points) to be processed make the direct one-to-one matching a very time-consuming process. On the other hand, matching algorithms based on the analysis of 2D data alone are very sensitive to variations in illumination, expression and pose. Algorithms, based on the face shape information alone, are instead relatively insensitive to these sources of noise. These mutually exclusive features of 2D- and 3D-based face recognition algorithm call for a cooperative scheme which may take advantage of the strengths of both, while coping for their weaknesses. We envisage many real and practical applications where 2D data can be used to improve 3D matching and vice versa. Towards this end, this chapter highlights both the advantages and disadvantages of 2D- and 3D-based face recognition algorithms . It also explores the advantages of blending 2D- and 3D data -based techniques, also proposing a novel approach for a fast and robust matching. Several experimental results, obtained from publicly available datasets, currently at the state of the art, demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Inverse problems most often do not fulfil Hadamard’s postulates of well-posedness: they may not have a solution in the strict sense, and solutions may not be unique and/or may not depend continuously on the data.

  2. 2.

    The difference of Gaussian is defined as the difference of two successive scale-space representations of the image, divided by the scale difference.

References

  1. Abate, A.F., Nappi, M., Riccio, D., Sabatino, G.: 2d and 3d face recognition: a survey. Pattern Recogn. Lett. 28(14), 1885–1906 (2007)

    Article  Google Scholar 

  2. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Article  MATH  Google Scholar 

  3. Al-Osaimi, F., Bennamoun, M., Mian, A.: An expression deformation approach to non-rigid 3d face recognition. Int. J. Comput. Vis. 81(3), 302–316 (2009)

    Article  Google Scholar 

  4. Alyuz, N., Gokberk, B., Akarun, L.: A 3d face recognition system for expression and occlusion invariance. In: Proceedings of IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–7. IEEE (2008)

    Google Scholar 

  5. Alyuz, N., Gokberk, B., Akarun, L.: Regional registration for expression resistant 3-d face recognition. IEEE Trans. Inf. Forensics Secur. 5(3), 425–440 (2010)

    Article  Google Scholar 

  6. Bartlett, M.S., Movellan, J.R., Sejnowski, T.J.: Face recognition by independent component analysis. IEEE Trans. Neural Networks 13(6), 1450–1464 (2002)

    Article  Google Scholar 

  7. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 404–417 (2006)

    Google Scholar 

  8. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997). doi 10.1109/34.598228. URL http://dx.doi.org/10.1109/34.598228

    Google Scholar 

  9. Berretti, S., Del Bimbo, A., Pala, P.: 3d face recognition using isogeodesic stripes. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2162–2177 (2010)

    Article  Google Scholar 

  10. Berretti, S., Werghi, N., Del Bimbo, A., Pala, P.: Matching 3d face scans using interest points and local histogram descriptors. Comput. Graph. 37(5), 509–525 (2013)

    Article  Google Scholar 

  11. Bicego, M., Lagorio, A., Grosso, E., Tistarelli, M.: On the use of sift features for face authentication. In: Proceedings of Computer Vision and Pattern Recognition Workshop (CVPR), pp. 35–35 (2006)

    Google Scholar 

  12. Boehnen, C., Flynn, P.: Impact of involuntary subject movement on 3d face scans. In: Proceedings of IEEE Computer Vision and Pattern Recognition Workshops (CVPR), pp. 1–6. IEEE (2009)

    Google Scholar 

  13. Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challenges in 3d and multimodal 3d + 2d face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006)

    Article  Google Scholar 

  14. Cadoni, M., Bicego, M., Grosso, E.: Iconic methods for multimodal face recognition: a comparative study. In: Proceedings of IAPR International Conference on Biometrics (ICB), vol. LNCS 5558, pp. 279–288. Springer (2009)

    Google Scholar 

  15. Cadoni, M., Lagorio, A., Grosso, E.: Iconic methods for multimodal face recognition: a comparative study. In: Proceedings of 22nd International Conference on Pattern Recognition (ICPR), pp. 4612–4617 (2014)

    Google Scholar 

  16. Cartoux, J.Y., LaPresté, J.T., Richetin, M.: Face authentification or recognition by profile extraction from range images. In: Proceedings of Workshop on Interpretation of 3D Scenes, 1989, pp. 194–199. IEEE (1989)

    Google Scholar 

  17. Chang, K., Bowyer, K., Flynn, P.: Face recognition using 2d and 3d facial data. In: ACM Workshop on Multimodal User Authentication, pp. 25–32 (2003)

    Google Scholar 

  18. Chang, K.I., Bowyer, K.W., Flynn, P.J.: Adaptive rigid multi-region selection for handling expression variation in 3d face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition-Workshops (CVPR), pp. 157–157. IEEE (2005)

    Google Scholar 

  19. Chang, K.I., Bowyer, W., Flynn, P.J.: Multiple nose region matching for 3d face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)

    Article  Google Scholar 

  20. Colombo, A., Cusano, C., Schettini, R.: Three-dimensional occlusion detection and restoration of partially occluded faces. J. Math. Imaging Vis. 40(1), 105–119 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  21. Cook, J., McCool, C., Chandran, V., Sridharan, S.: Combined 2d/3d face recognition using log-Gabor templates. In: Proceedings of IEEE International Conference on Video and Signal Based Surveillance (AVSS), pp. 83–83. IEEE (2006)

    Google Scholar 

  22. Cottrell, G.W., Fleming, M.: Face recognition using unsupervised feature extraction. In: Proceedings of the Intelligence Neural Network Conference, pp. 322–325 (1990)

    Google Scholar 

  23. Drira, H., Ben Amor, B., Srivastava, A., Daoudi, M., Slama, R.: 3d face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2270–2283 (2013)

    Article  Google Scholar 

  24. Er, M.J., Wu, S., Lu, J., Toh, H.L.: Face recognition with radial basis function (rbf) neural networks. IEEE Trans. Neural Networks 13(3), 697–710 (2002)

    Article  Google Scholar 

  25. Faltemier, T.C., Bowyer, K.W., Flynn, P.J.: A region ensemble for 3-d face recognition. IEEE Trans. Inf. Forensics Secur. 3(1), 62–73 (2008)

    Article  Google Scholar 

  26. Hadamard, J.: Lectures on the Cauchy Problem in Linear Partial Differential Equations. Yale University Press, New Haven (1923)

    MATH  Google Scholar 

  27. He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using Laplacian faces. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 328–340 (2005)

    Article  Google Scholar 

  28. He, X., Yan, S., Hu, Y., Zhang, H.J.: Learning a locality preserving subspace for visual recognition. In: Proceedings of Ninth IEEE International Conference on Computer Vision, 2003, pp. 385–392. IEEE (2003)

    Google Scholar 

  29. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst (2007)

    Google Scholar 

  30. Husken, M., Brauckmann, M., Gehlen, S., Von der Malsburg, C.: Strategies and benefits of fusion of 2d and 3d face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition-Workshops (CVPR), pp. 174–174. IEEE (2005)

    Google Scholar 

  31. Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. JIPS 5(2), 41–68 (2009)

    Google Scholar 

  32. Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)

    Article  Google Scholar 

  33. Kirby, M., Sirovich, L.: Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans. Pattern Anal. Mach. Intell. 12(1), 103–108 (1990)

    Article  Google Scholar 

  34. Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Networks 8(1), 98–113 (1997)

    Article  Google Scholar 

  35. Lee, J.C., Milios, E.: Matching range images of human faces. In: Proceedings of 3rd International Conference on Computer Vision, pp. 722–726. IEEE (1990)

    Google Scholar 

  36. Li, H., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: Towards 3d face recognition in the real: a registration-free approach using fine-grained matching of 3d keypoint descriptors. Int. J. Comput. Vis. 1–15 (2014)

    Google Scholar 

  37. Lin, S.H., Kung, S.Y., Lin, L.J.: Face recognition/detection by probabilistic decision-based neural network. IEEE Trans. Neural Networks 8(1), 114–132 (1997)

    Article  Google Scholar 

  38. Lin, W.Y., Wong, K.C., Boston, N., Hu, Y.H.: 3d face recognition under expression variations using similarity metrics fusion. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 727–730. IEEE (2007)

    Google Scholar 

  39. Liu, C.: Gabor-based kernel PCA with fractional power polynomial models for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 572–581 (2004)

    Article  Google Scholar 

  40. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). doi 10.1023/B:VISI.0000029664.99615.94. URL http://dx.doi.org/10.1023/B%3AVISI.0000029664.99615.94

    Google Scholar 

  41. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  42. Lu, J., Plataniotis, K.N., Venetsanopoulos, A.N.: Face recognition using lda-based algorithms. IEEE Trans. Neural Networks 14(1), 195–200 (2003)

    Article  Google Scholar 

  43. Lu, X., Jain, A.K., Colbry, D.: Matching 2.5 d face scans to 3d models. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 31–43 (2006)

    Article  Google Scholar 

  44. Martínez, A.M., Kak, A.C.: Pca versus lda. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 228–233 (2001)

    Article  Google Scholar 

  45. Maurer, T., Guigonis, D., Maslov, I., Pesenti, B., Tsaregorodtsev, A., West, D., Medioni, G.: Performance of geometrix activeidˆ tm 3d face recognition engine on the FRGC data. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition-Workshops (CVPR), pp. 154–154. IEEE (2005)

    Google Scholar 

  46. Mian, A.S., Bennamoun, M., Owens, R.: An efficient multimodal 2d-3d hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)

    Article  Google Scholar 

  47. Mian, A.S., Bennamoun, M., Owens, R.: An efficient multimodal 2d-3d hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 1584–1601 (2007)

    Google Scholar 

  48. Ocegueda, O., Fang, T., Shah, S.K., Kakadiaris, I.A.: 3d face discriminant analysis using Gauss-Markov posterior marginals. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 728–739 (2013)

    Article  Google Scholar 

  49. Ocegueda, O., Passalis, G., Theoharis, T., Shah, S.K., Kakadiaris, I.A.: Ur3d-c: linear dimensionality reduction for efficient 3d face recognition. In: Proceedings of IEEE International Joint Conference on Biometrics (IJCB), pp. 1–6. IEEE (2011)

    Google Scholar 

  50. Olver, P.: Joint invariant signatures. Found. Comput. Math. 1, 3–67 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  51. Queirolo, C.C., Silva, L., Bellon, O.R.P., Segundo, M.P.: 3d face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 206–219 (2010). doi http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.14

    Google Scholar 

  52. Shashua, A., Levin, A., Avidan, S.: Manifold pursuit: a new approach to appearance based recognition. In: Proceedings of 16th International Conference on Pattern Recognition (ICPR), vol. 3, pp. 590–594. IEEE (2002)

    Google Scholar 

  53. Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: meshsift: local surface features for 3d face recognition under expression variations and partial data. Comput. Vis. Image Underst. 117(2), 158–169 (2013)

    Article  Google Scholar 

  54. Spreeuwers, L.: Fast and accurate 3d face recognition. Int. J. Comput. Vis. 93(3), 389–414 (2011)

    Article  MATH  Google Scholar 

  55. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)

    Article  Google Scholar 

  56. Turk, M.A., Pentland, A.P.: Face recognition using eigenfaces. In: Proceedings CVPR’91., IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991, pp. 586–591. IEEE (1991)

    Google Scholar 

  57. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  58. Wang, Y., Liu, J., Tang, X.: Robust 3d face recognition by local shape difference boosting. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1858–1870 (2010)

    Article  Google Scholar 

  59. Wiskott, L., Fellous, J.M., Kuiger, N., Von Der Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 775–779 (1997)

    Article  Google Scholar 

  60. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  61. Wu, Y., Chan, K.L., Wang, L.: Face recognition based on discriminative manifold learning. In: Proceedings of 17th International Conference on Pattern Recognition (ICPR), vol. 4, pp. 171–174. IEEE (2004)

    Google Scholar 

  62. Zhang, B., Gao, Y., Zhao, S., Liu, J.: Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans. Image Process. 19(2), 533–544 (2010)

    Article  MathSciNet  Google Scholar 

  63. Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv (CSUR) 35(4), 399–458 (2003)

    Article  Google Scholar 

Download references

Acknowledgements

This research has been partially funded by the European Union COST Action IC1106 “Integrating Biometrics and Forensics for the Digital Age” and by funds from the Italian Ministry of Research.

Author information

Authors and Affiliations

  1. University of Sassari, Sassari, Italy

    M. Tistarelli, M. Cadoni, A. Lagorio & E. Grosso

Authors
  1. M. Tistarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Cadoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Lagorio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. Grosso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Tistarelli .

Editor information

Editors and Affiliations

  1. Multispectral Imagery Lab—MILab, LCSEE, West Virginia University, MORGANTOWN, West Virginia, USA

    Thirimachos Bourlai

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Tistarelli, M., Cadoni, M., Lagorio, A., Grosso, E. (2016). Blending 2D and 3D Face Recognition. In: Bourlai, T. (eds) Face Recognition Across the Imaging Spectrum. Springer, Cham. https://doi.org/10.1007/978-3-319-28501-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28501-6_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28499-6

  • Online ISBN: 978-3-319-28501-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation