Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels
2D intensity images and 3D shape models are both useful for face recognition, but in different ways. While algorithms have long been developed using 2D or 3D data, recently has seen work on combining both into multi-modal face biometrics to achieve higher performance. However, the fusion of the two modalities has mostly been at the decision level, based on scores obtained from independent 2D and 3D matchers.
In this paper, we propose a systematic framework for fusing 2D and 3D face recognition at both feature and decision levels, by exploring synergies of the two modalities at these levels. The novelties are the following. First, we propose to use Local Binary Pattern (LBP) features to represent 3D faces and present a statistical learning procedure for feature selection and classifier learning. This leads to a matching engine for 3D face recognition. Second, we propose a statistical learning approach for fusing 2D and 3D based face recognition at both feature and decision levels. Experiments show that the fusion at both levels yields significantly better performance than fusion at the decision level.
KeywordsFace Recognition Face Image Local Binary Pattern Decision Level Principal Component Analysis Method
Unable to display preview. Download preview PDF.
- 3.Bartlett, M.S., Lades, H.M., Sejnowski, T.J.: Independent component representations for face recognition. In: Proceedings of the SPIE, Conference on Human Vision and Electronic Imaging III, vol. 3299, pp. 528–539 (1998)Google Scholar
- 7.Ahonen, T., Hadid, A., Pietikainen, M.: Face recognition with local binary patterns. In: Proceedings of the European Conference on Computer Vision, Prague, Czech, pp. 469–481 (2004)Google Scholar
- 8.Hadid, A., Pietikinen, M., Ahonen, T.: A discriminative feature space for detecting and recognizing faces. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 797–804 (2004)Google Scholar
- 9.Jones, M., Viola, P.: Face recognition using boosted local features. Tech. Report TR2003-025, MERL (2003)Google Scholar
- 10.Zhang, G., Huang, X., Li, S.Z., Wang, Y.: Boosting local binary pattern (LBP)-based face recognition. In: Li, S.Z., Lai, J.-H., Tan, T., Feng, G.-C., Wang, Y. (eds.) SINOBIOMETRICS 2004. LNCS, vol. 3338, pp. 180–187. Springer, Heidelberg (2004)Google Scholar
- 11.Kanade, T.: Picture Processing by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University (1973)Google Scholar
- 12.Cartoux, J.Y., LaPreste, J.T., Richetin, M.: Face authentication or recognition by profile extraction from range images. In: Proceedings of the Workshop on Interpretation of 3D Scenes (1989)Google Scholar
- 13.Besl, P.J., Jain, R.C.: Intrinsic and extrinsic surface characteristics. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, California, pp. 226–233 (1985)Google Scholar
- 14.Bowyer, K.W., Chang, K., Flynn, P.: A survey of 3D and multi-modal 3d+2d face recognition. In: Proceedings of International Conference Pattern Recognition, pp. 358–361 (2004)Google Scholar
- 15.Lu, X., Jain, A.K.: Integrating range and texture information for 3d face recognition. In: Proc. 7th IEEE Workshop on Applications of Computer Vision (WACV 2005), Breckenridge, CO. (2005)Google Scholar
- 16.Tsalakanidou, F., Malassiotis, S., Strintzis, M.G.: Face localization and authentication using color and depth images 14, 152–168 (2005)Google Scholar
- 17.Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multi-modal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence (2005) (to appear)Google Scholar
- 19.Schapire, R.: A brief introduction to boosting. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (1999)Google Scholar
- 20.Viola, P., Jones, M.: Robust real time object detection. In: IEEE ICCV Workshop on Statistical and Computational Theories of Vision, Vancouver, Canada (2001)Google Scholar
- 21.Moghaddam, B., Nastar, C., Pentland, A.: A Bayesain similarity measure for direct image matching. Media Lab Tech. Report No.393, MIT (1996)Google Scholar