International Journal of Computer Vision

, Volume 93, Issue 3, pp 389–414 | Cite as

Fast and Accurate 3D Face Recognition

Using Registration to an Intrinsic Coordinate System and Fusion of Multiple Region Classifiers
  • Luuk SpreeuwersEmail author
Open Access


In this paper we present a new robust approach for 3D face registration to an intrinsic coordinate system of the face. The intrinsic coordinate system is defined by the vertical symmetry plane through the nose, the tip of the nose and the slope of the bridge of the nose. In addition, we propose a 3D face classifier based on the fusion of many dependent region classifiers for overlapping face regions. The region classifiers use PCA-LDA for feature extraction and the likelihood ratio as a matching score. Fusion is realised using straightforward majority voting for the identification scenario. For verification, a voting approach is used as well and the decision is defined by comparing the number of votes to a threshold. Using the proposed registration method combined with a classifier consisting of 60 fused region classifiers we obtain a 99.0% identification rate on the all vs first identification test of the FRGC v2 data. A verification rate of 94.6% at FAR=0.1% was obtained for the all vs all verification test on the FRGC v2 data using fusion of 120 region classifiers. The first is the highest reported performance and the second is in the top-5 of best performing systems on these tests. In addition, our approach is much faster than other methods, taking only 2.5 seconds per image for registration and less than 0.1 ms per comparison. Because we apply feature extraction using PCA and LDA, the resulting template size is also very small: 6 kB for 60 region classifiers.


3D Face recognition Registration Fusion Region classifiers FRGC 


  1. 3DFace (2009). 3D face project web page.
  2. Achermann, B., Jiang, X., & Bunke, H. (1997). Face recognition using range data. In Proceedings of the international conference on virtual systems and multimedia (pp. 129–136). Geneva: IEEE Press. Google Scholar
  3. Al-Osaimi, F., Bennamoun, M., & Mian, A. (2009). An expression deformation approach to non-rigid 3D face recognition. International Journal of Computer Vision, 81(3), 302–316. doi: 10.1007/s11263-008-0174-0. CrossRefGoogle Scholar
  4. Alyüz, N., Gökberk, B., & Akarun, L. (2009). Regional registration and curvature descriptors for expression resistant 3D face recognition. In Proceedings of the IEEE 17th signal processing and communications applications conference (SIU-2009) (pp. 544–547). CrossRefGoogle Scholar
  5. Bazen, A. M., & Veldhuis, R. N. J. (2004). Likelihood ratio-based biometric verification. IEEE Transactions on Circuits and Systems for Video Technology, 14(1), 86–94. CrossRefGoogle Scholar
  6. Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256. doi: 10.1109/34.121791. CrossRefGoogle Scholar
  7. Beumer, G. M., Tao, Q., Bazen, AM, & Veldhuis, R. N. J. (2006). A landmark paper in face recognition. In Proceedings of the 7th international conference on automatic face and gesture recognition (FGR 2006) (pp. 73–78). Los Alamitos: IEEE Computer Society Press. CrossRefGoogle Scholar
  8. Boehnen, C., Peters, T., & Flynn, P. J. (2009). 3d signatures for fast 3D face recognition. In Proceedings of the third international conference on advances in biometrics (ICB ’09) (pp. 12–21). Berlin: Springer. Google Scholar
  9. Boom, B. J., Beumer, G. M., Spreeuwers, L. J., & Veldhuis, R. N. J. (2006). The effect of image resolution on the performance of a face recognition system. In Proceedings of the 9th international conference on control, automation, robotics and vision (ICARCV), Singapore, Malaysia (pp. 409–414). Google Scholar
  10. Boom, B. J., Spreeuwers, L. J., & Veldhuis, R. N. J. (2007). Automatic face alignment by maximizing similarity score. In A. Fred & A. K. Jain (Eds.), Proceedings of the 7th international workshop on pattern recognition in information systems (Madeira, Portugal, 2007) (pp. 221–230). Madeira: INSTICC Press. Biosignals. Google Scholar
  11. Bowyer, K. W., Chang, K., & Flynn, P. (2006). A survey of approaches and challenges in 3D and multi-modal 3d+2d face recognition. Computer Vision and Image Understanding, 101(1), 1–15. doi: 10.1016/j.cviu.2005.05.005. CrossRefGoogle Scholar
  12. Brent, R. P. (1973). Algorithms for minimization without derivatives. Englewood Cliffs: Prentice Hall, Chap. 5. zbMATHGoogle Scholar
  13. Buhan, I. R., Doumen, J. M., Hartel, P. H., Tang, Q., & Veldhuis, R. N. J. (2010). Embedding renewable cryptographic keys into continuous noisy data. International Journal of Information Security, 9(3), 193–208. CrossRefGoogle Scholar
  14. Cartoux, J. Y., Lapreste, J. T., & Richetin, M. (1989). Face authentification or recognition by profile extraction from range images. In Proceedings of the workshop on interpretation of 3D scenes (pp. 194–199). Google Scholar
  15. Chen, C., Veldhuis, R. N. J., Kevenaar, TAM, & Akkermans, A. H. M. (2009). Biometric quantization through detection rate optimized bit allocation. EURASIP Journal on Advances in Signal Processing, 2009, 784–834. Google Scholar
  16. Colombo, A., Cusano, C., & Schettini, R. (2006). Detection and restoration of occlusions for 3D face recognition. In IEEE international conference on multimedia and expo (pp. 1541–1544). doi: 10.1109/ICME.2006.262837. CrossRefGoogle Scholar
  17. Faltemier, T., Bowyer, K., & Flynn, P. (2008a). A region ensemble for 3-d face recognition. IEEE Transactions on Information Forensics and Security, 3(1), 62–73. doi: 10.1109/TIFS.2007.916287. CrossRefGoogle Scholar
  18. Faltemier, T. C., Bowyer, K. W., & Flynn, P. J. (2008b). Using multi-instance enrollment to improve performance of 3D face recognition. Computer Vision and Image Understanding, 112(2), 114–125. doi: 10.1016/j.cviu.2008.01.004. CrossRefGoogle Scholar
  19. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395. CrossRefMathSciNetGoogle Scholar
  20. Gokberk, B., Irfanoglu, M. O., & Akarun, L. (2006). 3D shape-based face representation and feature extraction for face recognition. Image and Vision Computing, 24(8), 857–869. doi: 10.1016/j.imavis.2006.02.009. CrossRefGoogle Scholar
  21. Gonzalez-Rodriguez, J., Fiérrez-Aguilar, J., Ortega-Garcia, J., & Lucena-Molina, J. J. (2002). Biometric identification in forensic cases according to the bayesian approach. In Proceedings of the international ECCV 2002 workshop Copenhagen on biometric authentication (pp. 177–185). London: Springer. Google Scholar
  22. van der Heijden, F., & Spreeuwers, L. J. (2007). Image processing. In Multimedia retrieval (pp. 163–165). Berlin: Springer. Google Scholar
  23. Hesher, C., Srivastava, A., & Erlebacher, G. (2003). A novel technique for face recognition using range imaging. In 7th international symposium on signal processing and its applications (Vol. 2, pp. 201–204). doi: 10.1109/ISSPA.2003.1224850. CrossRefGoogle Scholar
  24. Jain, A. K., Duin, R. P. W., & Mao, J. (2000). Statistical pattern recognition: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 4–37. doi: 10.1109/34.824819. CrossRefGoogle Scholar
  25. Kakadiaris, I. A., Passalis, G., Toderici, G., Murtuza, M. N., Lu, Y., Karampatziakis, N., & Theoharis, T. (2007). Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 640–649. doi: 10.1109/TPAMI.2007.1017. CrossRefGoogle Scholar
  26. Kelkboom, E. J. C., Zhou, X., Breebaart, J., Veldhuis, R. N. J., & Busch, C. (2009). Multi-algorithm fusion with template protection. In IEEE 3rd international conference on biometrics: theory, applications, and systems, 2009 (BTAS ’09) (pp. 1–8). Washington: IEEE Computer Society Press. CrossRefGoogle Scholar
  27. Kelkboom, E. J. C., Garcia-Molina, G., Breebaart, J., Veldhuis, R. N. J., Kevenaar, T. A. M., & Jonker, W. (2010). Binary biometrics: an analytic framework to estimate the performance curves under gaussian assumption. IEEE Transactions on Systems, Man and Cybernetics. Part A. Systems and Humans, 40(3), 555–571. CrossRefGoogle Scholar
  28. Kuncheva, L. I., Whitaker, C. J., & Duin, R. P. W. (2003). Limits on the majority vote accuracy in classifier fusion. Pattern Analysis and Applications, 6, 22–31. CrossRefzbMATHMathSciNetGoogle Scholar
  29. Maurer, T., Guigonis, D., Maslov, I., Pesenti, B., Tsaregorodtsev, A., West, D., & Medioni, G. (2005). Performance of geometrix activeid™ 3d face recognition engine on the frgc data. In Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05)—workshops (p. 154). Washington: IEEE Computer Society. doi: 10.1109/CVPR.2005.581. Google Scholar
  30. Medioni, G., & Waupotitsch, R. (2003). Face modeling and recognition in 3-d. In Proceedings of the IEEE international workshop on analysis and modeling of faces and gestures (AMFG ’03) (p. 232). Washington: IEEE Computer Society. Google Scholar
  31. Mian, A., Bennamoun, M., & Owens, R. (2007). An efficient multimodal 2D–3D hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 1927–1943. doi: 10.1109/TPAMI.2007.1105. CrossRefGoogle Scholar
  32. Papatheodorou, T., & Rueckert, D. (2005). Evaluation of 3D face recognition using registration and pca. In T. Kanade, A. K. Jain, & N. K. Ratha (Eds.), Lecture notes in computer science: Vol. 3546. Proceedings of the 5th international conference on audio- and video-based biometric person authentication (AVBPA 2005) (pp. 997–1009). Hilton Rye Town: Springer. CrossRefGoogle Scholar
  33. Papatheodorou, T., & Rueckert, D. (2007). 3D face recognition. Vienna: I-Tech Education and Publishing. Google Scholar
  34. Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., Marques, J., Min, J., & Worek, W. (2005). Overview of the face recognition grand challenge. In Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05) (pp. 947–954). Washington: IEEE Computer Society. doi: 10.1109/CVPR.2005.268. CrossRefGoogle Scholar
  35. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1988). Numerical recipes in C. The art of scientific computing. Cambridge: Press Syndicate of the University of Cambridge. Chap. 10. zbMATHGoogle Scholar
  36. Queirolo, C. C., Silva, L., Bellon, O. R., & Segundo, M. P. (2010). 3D face recognition using simulated annealing and the surface interpenetration measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 206–219. CrossRefGoogle Scholar
  37. Ross, A. A., Jain, A. K., & Nandakumar, K. (2006a). Levels of fusion in biometrics. In International series on biometrics (Vol. 6, pp. 59–90). Berlin: Springer. Chap 4. doi: 10.1007/0-387-33123-9. Google Scholar
  38. Ross, A. A., Jain, A. K., & Nandakumar, K. (2006b). Score level fusion. In International series on biometrics (Vol. 6, pp. 91–142). Berlin: Springer. Chap. 5. doi: 10.1007/0-387-33123-9. Google Scholar
  39. Salah, A. A., Alyüz, N., & Akarun, L. (2007). Alternative face models for 3D face registration. In Proceedings of the SPIE conference on vision geometry XV, San Jose, CA, USA (Vol. 6499). Berlin: Springer. doi: 10.1117/12.705860. Google Scholar
  40. Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008). Bosphorus database for 3D face analysis. In B. Schouten, N. C. Juul, A. Drygajlo, & M. Tistarelli (Eds.), Lecture notes in computer science: Vol. 5372. Biometrics and identity management (pp. 47–56). Berlin: Springer. Chap 6. doi: 10.1007/978-3-540-89991-4_6. CrossRefGoogle Scholar
  41. Scheenstra, A., Ruifrok, A., & Veltkamp, R. C. (2005). A survey of 3D face recognition methods. In Lecture notes in computer science (pp. 891–899). Berlin: Springer. Google Scholar
  42. Silva, L., Bellon, O. R. P., & Boyer, K. L. (2005). Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 762–776. doi: 10.1109/TPAMI.2005.108. CrossRefGoogle Scholar
  43. Spreeuwers, L. J., Boom, B. J., & Veldhuis, R. N. J. (2007). Better than best: matching score based face registration. In R. N. J. Veldhuis & H. S. Cronie (Eds.), Proceedings of the 28th symposium on information theory in the Benelux, Enschede, The Netherlands (pp. 125–132). Eindhoven: Werkgemeenschap voor Informatie- en Communicatietheorie. Google Scholar
  44. Tang, X., Chen, J., & Moon, Y. (2008). Accurate 3D face registration based on the symmetry plane analysis on nose regions. In Proceedings of the 16th European signal processing conference (EUSIPCO 2008), Lausanne, Switzerland. Google Scholar
  45. Tao, Q., & Veldhuis, R. N. J. (2007). Optimal decision fusion for a face verification system. In L. Seong-Whan & Z. L. Stan (Eds.), Image processing, computer vision, pattern recognition. Proceedings of the 2nd international conference on biometrics. Seoul, Korea (pp. 958–967). Berlin: Springer. Google Scholar
  46. Tao, Q., & Veldhuis, R. N. J. (2008). Hybrid fusion for biometrics: Combining score-level and decision-level fusion. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition, workshop on biometrics (pp. 1–6). Anchorage: IEEE Computer Society Press. Google Scholar
  47. Tao, Q., & Veldhuis, R. N. J. (2009). Threshold-optimized decision-level fusion and its application to biometrics. Pattern Recognition, 42(5), 823–836. CrossRefGoogle Scholar
  48. Tao, Q., van Rootseler, R. T. A., Veldhuis, R. N. J., Gehlen, S., & Weber, F. (2007). Optimal decision fusion and its application on 3D face recognition. In A. Bromme, C. Busch, & D. Huhnlein (Eds.), Proceedings of the special interest group on biometrics and electronic signatures, Darmstadt, Germany, Gesellschaft fur Informatik e.V. (pp. 15–24). Berlin: GI-Edition. Google Scholar
  49. Torr, P. H. S., & Zisserman, A. (2000). Mlesac: a new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78, 138–156. doi: 10.1006/cviu.1999.0832. CrossRefGoogle Scholar
  50. Veldhuis, R. N. J., Bazen, A. M., Booij, W. D. T., & Hendrikse, A. J. (2006). Hand-geometry recognition based on contour landmarks. In M. Spiliopoulou, R. Kruse, C. Borgelt, A. Nurnberger, & W. Gaul (Eds.), Studies in classification, data analysis, and knowledge organisation. Proceedings of the 29th annual conference of the Gesellschaft fur Klassifikation e.V., Magdeburg, Germany (pp. 646–653). Berlin: Springer. Google Scholar

Copyright information

© The Author(s) 2011

Authors and Affiliations

  1. 1.Chair of Signals and Systems, Department of EEMCSUniversity of TwenteTwenteThe Netherlands

Personalised recommendations