Advertisement

International Journal of Computer Vision

, Volume 64, Issue 1, pp 5–30 | Cite as

Three-Dimensional Face Recognition

  • Alexander M. Bronstein
  • Michael M. Bronstein
  • Ron Kimmel
Article

Abstract

An expression-invariant 3D face recognition approach is presented. Our basic assumption is that facial expressions can be modelled as isometries of the facial surface. This allows to construct expression-invariant representations of faces using the bending-invariant canonical forms approach. The result is an efficient and accurate face recognition algorithm, robust to facial expressions, that can distinguish between identical twins (the first two authors). We demonstrate a prototype system based on the proposed algorithm and compare its performance to classical face recognition methods.

The numerical methods employed by our approach do not require the facial surface explicitly. The surface gradients field, or the surface metric, are sufficient for constructing the expression-invariant representation of any given face. It allows us to perform the 3D face recognition task while avoiding the surface reconstruction stage.

Keywords

expression-invariant 3D face recognition isometry invariant facial expressions multidimensional scaling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Achermann, B. and Bunke, H. 2000. Classifying range images of humanfaces with Hausdorff distance. In Proc. ICPR, pp. 809–813.Google Scholar
  2. Achermann, B., Jiang, X., and Bunke, H. 1997. Face recognition using range images. In Int’l Conf. Virtual Systems and Multimedia, pp. 129–136.Google Scholar
  3. Arnoldi, W. 1951. The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math, 9:17–29.Google Scholar
  4. Ashbourn, J. 2002. Biometrics: Advanced Identity Verification. Springer-Verlag: Berlin Heidelberg, New York.Google Scholar
  5. Bai, Z., Demmel, J., Dongarra, J., Ruhe, A., and van der Vorst, H. 2000. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide. Philadelphia: SIAM, third edition. Online: http://www.cs.utk.edu/dongarra/etemplates/index.html.
  6. Besl, P.J. and McKay, N.D. 1992. A method for registration of 3D shapes. IEEE Trans. PAMI, 14:239–256.Google Scholar
  7. Beumier, C. and Acheroy, M.P. 1998. Automatic face authentication from 3D surface. In Proc. British Machine Vision Conf, pp. 449–458.Google Scholar
  8. Bledsoe, W.W. 1966. Man-machine face recognition. Technical Report PRI 22, Panoramic Research Inc., Palo Alto (CA) USA.Google Scholar
  9. Borg, I. and Groenen, P. 1997. Modern Multidimensional Scaling—Theory and Applications. Springer-Verlag: Berlin Heidelberg New York.Google Scholar
  10. Bowyer, K.W., Chang, K., and Flynn, P. 2004. A survey of 3D and multi-modal 3D + 2D face recognition. Dept. of Computer Science and Electrical Engineering Technical Report, University of Notre Dame.Google Scholar
  11. Bronstein, M.M. 2004. Three-dimensional face recognition. Master’s Thesis, Department of Computer Science, Technion—Israel Institute of Technology.Google Scholar
  12. Bronstein, A.M., Bronstein, M.M., Gordon, E., and Kimmel, R. 2003a. High-resolution structured light range scanner with automatic calibration. Technical Report CIS-2003-06, Dept. of Computer Science, Technion, Israel.Google Scholar
  13. Bronstein, A.M., Bronstein, M.M., Gordon, E., and Kimmel, R. 2004a. Fusion of 3D and 2D information in face recognition. In Proc. ICIP, pp. 87–90.Google Scholar
  14. Bronstein, A.M., Bronstein, M.M., and Kimmel, R. 2003b. Expression-invariant 3D face recognition. In Proc. Audio and Video-based Biometric Person Authentication, pp. 62–69.Google Scholar
  15. Bronstein, A.M., Bronstein, M.M., and Kimmel, R. 2005a. Expression-invariant face recognition via spherical embedding. In Proc. ICIP, to appear.Google Scholar
  16. Bronstein, A.M., Bronstein, M.M., and Kimmel, R. 2005b. On isometric embedding of facial surfaces into \(\mathbb S\)3. In Proc. Int’l Conf. Scale Space and PDE Methods in Computer Vision, Lecture Notes in Comp. Science 3459, Springer, pp. 622–631.Google Scholar
  17. Bronstein, A.M., Bronstein, M.M., Kimmel, R., and Spira, A. 2004b. Face recognition from facial surface metric. In Proc. ECCV, pp. 225–237.Google Scholar
  18. Bronstein, M.M., Bronstein, A.M., Kimmel, R., and Yavneh, I. 2005c. A multigrid approach for multi-dimensional scaling. In Proc. Copper Mountain Conf. Multigrid Methods (submitted).Google Scholar
  19. Bronstein, M.M., Bronstein, A.M., and Kimmel, R. 2005. Expression-invariant representations for human faces. Technical Report CIS-2005-01, Dept. of Computer Science, Technion, Israel.Google Scholar
  20. Brunelli, R. and Poggio, T. 1993. Face recognition: Features vs. templates. IEEE Trans. PAMI, 15(10):1042–1053.Google Scholar
  21. Carrihill, B. and Hummel, R. 1985. Experiments with the intensity ratio depth sensor. Computer Vision, Graphics and Image Processing, 32:337–358.Google Scholar
  22. Cartoux, J.Y., La Preste, J.T., and Richetin, M. 1989. Face authentication or recognition by profile extraction from range images. In Proc. Workshop on Interpretation of 3D Scenes, pp. 194–199.Google Scholar
  23. Chang, K., Bowyer, K., and Flynn, P. 2003. Face recognition using 2D and 3D facial data. In Proc. Multimodal User Authentication Workshop, pp. 25–32.Google Scholar
  24. Cox, I., Ghosn, J., and Yianilos, P. 1996. Feature-based face recognition using mixture distance. In Proc. CVPR, pp. 209–216.Google Scholar
  25. De Leeuw, J. 1977. Recent Developments in Statistics, Chapt. Applications of convex analysis to multidimensional scaling, Amsterdam: North-Holland, pp. 133–145.Google Scholar
  26. De Leeuw, J. and Stoop, I. 1984. Upper bounds on Kruskal’s stress. Psychometrika 49:391–402.Google Scholar
  27. Eckart, C. and Young, G. 1936. Approximation of one matrix by another of lower rank. Psychometrika, 1:211–218.Google Scholar
  28. Ekman, P. 1973. Darwin and Facial Expression; A Century of Research in Review. Academic Press: New York.Google Scholar
  29. Elad, A. and Kimmel, R. 2001. Bending invariant representations for surfaces. In Proc. CVPR, pp. 168–174.Google Scholar
  30. Elad, A. and Kimmel, R. 2002. Geometric Methods in Bio-Medical Image Processing, Vol. 2191, Chapt. Spherical flattening of the cortex surface. Springer-Verlag: Berlin Heidelberg New York, pp. 77–89.Google Scholar
  31. Elad, A. and Kimmel, R. 2003. On bending invariant signatures for surfaces. IEEE Trans. PAMI, 25(10):1285–1295.Google Scholar
  32. Fleishman, S., Drori, I., and Cohen-Or, D. 2003. Bilateral mesh denoising. In Proc. SIGGRAPH, pp. 950–953.Google Scholar
  33. Gauss, C.F. 1827. Disquisitiones generales circa superficies curva. Commentationes Societatis Regiæ Scientiarum Gottingensis Recentiores, 6:99–146.Google Scholar
  34. Georghiades, A.S., Belhumeur, P.N., and Kriegman, D. 1998. Illumination cones for recognition under variable lighting: Faces. In Proc. CVPR, pp. 52–58.Google Scholar
  35. Gheorghiades, A.S., Belhumeur, P.N., and Kriegman, D.J. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. PAMI, 23(6):643–660.Google Scholar
  36. Goldstein, A., Harmon, L., and Lesk, A. 1971. Identification of human faces. Proc. IEEE, 59(5):748–760.Google Scholar
  37. Golub, G.H. and Saunders, M. 2004. Personal communication.Google Scholar
  38. Golub, G.H. and van Loan, C.F. 1996. Matrix Computations. 3rd ed. The John Hopkins University Press.Google Scholar
  39. Gordon, G. 1992. Face recognition based on depth and curvature features. In Proc. CVPR, pp. 108–110.Google Scholar
  40. Gordon, G. 1997. Face recognition from frontal and profile views. In Proc. Int’l Workshop on Face and Gesture Recognition, pp. 47–52.Google Scholar
  41. Gower, J.C. 1966. Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika, 53:325–338.Google Scholar
  42. Grossman, R., Kiryati, N., and Kimmel, R. 2002. Computational surface flattening: A voxel-based approach. IEEE Trans. PAMI, 24(4):433–441.Google Scholar
  43. Gruen, A. and Akca, D. 2004. Least squares 3D surface matching. In Proc. ISPRS Working Group V/1 Panoramic Photogrammetry Workshop, pp. 19–22.Google Scholar
  44. Gudmundsson, S. 2004. An introduction to riemannian geometry (lecture notes). Online. Available at http://www.matematik.lu.se/matematiklu/personal/sigma/Riemann.pdf.
  45. Guttman, L. 1968. A general nonmetric technique for finding the smallest coordinate space for a configuration of points. Psychometrika 33, pp. 469–506.Google Scholar
  46. Hallinan, P. 1994. A low-dimensional representation of human faces for arbitrary lighting conditions. In Proc. CVPR, pp. 995–999.Google Scholar
  47. Hesher, C., Srivastava, A., and Erlebacher, G. 2003. A novel technique for face recognition using range images. In Int’l Symp. Signal Processing and Its Applications, Vol. 2, pp. 201–204.Google Scholar
  48. Horn, E. and Kiryati, N. 1999. Toward optimal structured light patterns. Image and Vision Computing, 17(2):87–97.CrossRefGoogle Scholar
  49. Huang, J., Blanz, V., and Heisele, V. 2002. Face recognition using component-based SVM classification and morphable models. SVM, pp. 334–341.Google Scholar
  50. Hugli, H. and Maitre, G. 1989. Generation and use of color pseudo random sequences for coding structured light in active ranging. In Proc. Industrial Inspection, Vol. 1010, pp. 75–82.Google Scholar
  51. Hung, Y.-P., Chen, C.-S., Hsieh, I.-B., and Fuh, C.-S. 1999. Reconstruction of complete 3D object model from multiview range images. In Proc. SPIE Int’l Symposium on Electronic Imaging, pp. 138–145.Google Scholar
  52. Kanade, T. 1973. Picture processing by computer complex and recognition of human faces. Technical report, Kyoto University, Dept. of Information Science.Google Scholar
  53. Kearsley, A., Tapia, R., and Trosset, M. 1998. The solution of the metric STRESS and SSTRESS problems in multidimensional scaling using Newton’s method. Computational Statistics, 13(3):369–396.Google Scholar
  54. Kimmel, R. 2003. Numerical Geometry of Images. Springer-Verlag: Berlin Heidelberg New York.Google Scholar
  55. Kimmel, R. and Sethian, J.A. 1998. Computing geodesic on manifolds. In Proc. US National Academy of Science, Vol. 95, pp. 8431–8435.Google Scholar
  56. Kreyszig, E. 1991. Differential Geometry. Dover Publications Inc.: New York.Google Scholar
  57. Lee, J.C. and Milios, E. 1990. Matching range images of human faces. In Proc. ICCV, pp. 722–726.Google Scholar
  58. Linial, N., London, E., and Rabinovich, Y. 1995. The geometry of graphs and some its algorithmic applications. Combinatorica, 15:333–344.CrossRefGoogle Scholar
  59. Mansfield, T., Kelly, G., Chandler, D., and Kane, J. 2001. Biometric product testing final report. Technical report, Centre for Mathematics and Scientific Computing, National Physical Laboratory, UK.Google Scholar
  60. Mavridis, N., Tsalakanidou, F., Pantazis, D., Malassiotis, S., and Strintzis, M.G. 2001. The HISCORE face recognition application: Affordable desktop face recognition based on a novel 3D camera. In Proc. Int’l Conf. Augmented Virtual Environments and 3D Imaging, pp. 157–160.Google Scholar
  61. Medioni, G. and Waupotitsch, R. 2003. Face recognition and modeling in 3D. In Proc. AMFG, pp. 232–233.Google Scholar
  62. Mémoli, F. and Sapiro, G. 2004. Comparing point clouds, IMA preprint series no. 1978, University of Minnesota.Google Scholar
  63. Nagamine, T., Uemura, T., and Masuda, I. 1992. 3D facial image analysis for human identification. In Proc. ICPR, pp. 324–327.Google Scholar
  64. Nash, S. 2000. A multigrid approach to discretized optimization problems. Journal of Optimization Methods and Software, 14:99–116.Google Scholar
  65. Ortega-Garcia, J., Bigun, J., Reynolds, D., and Gonzalez-Rodriguez, J. 2004. Authentication gets personal with biometrics. IEEE Signal Processing magazine, 21(2):50–62.CrossRefGoogle Scholar
  66. Pentland, A., Moghaddam, B., and Starner, T. 1994. View-based and modular eigenspaces for face recognition. In Proc. CVPR, pp. 84–91.Google Scholar
  67. Phillips, P.J., Grother, P., Michaels, R.J., Blackburn, D.M., Tabassi, E., and Bone, J. 2003. FRVT 2002: Overview and summary. Online. Available at www.frvt.org.Google Scholar
  68. Posdamer, J.L. and Altschuler, M.D. 1982. Surface measurement by space-encoded projected beam systems. Computer Graphics and Image Processing, 18(1):1–17.CrossRefGoogle Scholar
  69. Schwartz, E.L., Shaw, A., and Wolfson, E. 1989. A numerical solution to the generalized mapmaker’s problem: Flattening nonconvex polyhedral surfaces. IEEE Trans. PAMI, 11:1005–1008.Google Scholar
  70. Sethian, J.A. 1996. A review of the theory, algorithms, and applications of level set method for propagating surfaces. Acta numerica, pp. 309–395.Google Scholar
  71. Sirovich, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. JOSA A, 2:519–524.Google Scholar
  72. Sochen, N., Kimmel, R., and Malladi, R. 1998. A general framework for low level vision. IEEE Trans. Image Proc, 7(3):310–318.CrossRefGoogle Scholar
  73. Spira, A. and Kimmel, R. 2003. An efficient solution to the eikonal equation on parametric manifolds. In INTERPHASE 2003 meeting.Google Scholar
  74. Spira, A. and Kimmel, R. 2004. An efficient solution to the Eikonal equation on parametric manifolds. Interfaces and Free Boundaries, 6(3):315–327.Google Scholar
  75. Stewart, G.W. and Sun, J.-G. 1990. Matrix Perturbation Theory. Academic Press.Google Scholar
  76. Tajima, J. and Iwakawa, M. 1990. 3D data acquisition by rainbow range finder. In Proc. Int’l Conf. Pattern Recognition, pp. 309–313.Google Scholar
  77. Tal, A., Elad, M., and Ar, S. 2001. Content based retrieval of VRML objects—an iterative and interactive approach. In Proc. Eurographics Workshop on Multimedia.Google Scholar
  78. Tanaka, H.T., Ikeda, M., and Chiaki, H. 1998. Curvature-based face surface recognition using spherical correlation principal directions for curved object recognition. In Proc. Int’l Conf. Automated Face and Gesture Recognition, pp. 372–377.Google Scholar
  79. Tasdizen, T., Whitaker, R., Burchard, P., and Osher, S. 2002. Geometric surface smoothing via anisotropic diffusion of normals. In Proc. IEEE Conf. Visualization, pp. 125–132.Google Scholar
  80. Torgerson, W.S. 1952. Multidimensional scaling I—Theory and methods. Psychometrika, 17:401–419.Google Scholar
  81. Tsalakanidou, F., Tzocaras, D., and Strintzis, M. 2003. Use of depth and colour eigenfaces for face recognition. Pattern Recognition Letters, 24:1427–1435.CrossRefGoogle Scholar
  82. Tsitsiklis, J.N. 1995. Efficient algorithms for globally optimal trajectories. IEEE Trans. Automatic Control, 40(9):1528–1538.CrossRefGoogle Scholar
  83. Turk, M. and Pentland, A. 1991. Face recognition using eigenfaces. In Proc. CVPR, pp. 586–591.Google Scholar
  84. Vuylsteke, P. and Oosterlinck, A. 1990. Range image acquisition with a single binary-encoded light pattern. IEEE Trans. PAMI, 12(2):148–163.Google Scholar
  85. Walter, J. and Ritter, H. 2002. On interactive visualization of high-dimensional data using the hyperbolic plane. In Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, pp. 123–131.Google Scholar
  86. Wang, Y., Chua, C., and Ho, Y. 2002. Facial feature detection and face recognition from 2D and 3D images. Pattern Recognition Letters, 23:1191–1202.CrossRefGoogle Scholar
  87. Wiskott, L. 1995. Labeled Graphs and Dynamic Link Matching for Face Recognition and Scene Analysis, No. 53 in Reihe Physik. Thun—Frankfurt am Main: Verlag Harri Deutsch, PhD Thesis.Google Scholar
  88. Wiskott, L., Fellous, J., Kruger, N., and von der Malsburg, C. 1997. Face recognition by elastic bunch graph matching. IEEE Trans. PAMI, 19(7):733–742.Google Scholar
  89. Young, G. and Householder, A.S. 1938. Discussion of a set of point in terms of their mutual distances. Psychometrika, 3:19–22.Google Scholar
  90. Zhang, Z.Y. 1994. Terative point matching for registration of free-form curves and surfaces. IJCV, 13:119–152.CrossRefGoogle Scholar
  91. Zigelman, G., Kimmel, R., and Kiryati, N. 2002. Texture mapping using surface flattening via multi-dimensional scaling. IEEE Trans. Visualization and computer graphics, 9(2):198–207.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  • Alexander M. Bronstein
    • 1
  • Michael M. Bronstein
    • 1
  • Ron Kimmel
    • 1
  1. 1.Department of Computer ScienceTechnion–Israel Institute of TechnologyHaifaIsrael

Personalised recommendations