Abstract
In this paper, we develop a novel framework for 3D tracking of the non-rigid face deformation from a single camera. The difficulty of the problem lies in the fact that 3D deformation parameter estimation becomes unstable when there are few reliable facial features correspondences. Unfortunately, this often occurs in real tracking scenario when there is significant illumination change, motion blur or large pose variation. In order to extract more information of feature correspondences, the proposed framework integrates three types of features which discriminate face deformation across different views: 1) the semantic features which provide constant correspondences between 3D model points and major facial features; 2) the silhouette features which provide dynamic correspondences between 3D model points and facial silhouette under varying views; 3) the online tracking features that provide redundant correspondences between 3D model points and salient image features. The integration of these complementary features is important for robust estimation of the 3D parameters. In order to estimate the high dimensional 3D deformation parameters, we develop a hierarchical parameter estimation algorithm to robustly estimate both rigid and non-rigid 3D parameters. We show the importance of both features fusion and hierarchical parameter estimation for reliable tracking 3D face deformation. Experiments demonstrate the robustness and accuracy of the proposed algorithm especially in the cases of agile head motion, drastic illumination change, and large pose change up to profile view.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2D+3D active appearance models. In: Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 535–542 (2004)
Matthews, I., Baker, S.: Active Appearance Models Revisited. International Journal of Computer Vision 60(2), 135–164 (2004)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp. 187–194 (1999)
Zhou, Y., Zhang, W., Tang, X., Shum, H.: A Bayesian Mixture Model for Multi-view Face Alignment. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 741–746 (2005)
Gu, L., Kanade, T.: 3D Alignment of Face in a Single Image. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1305–1312 (2006)
Vogler, C., Li, Z., Kanaujia, A., Goldenstein, S., Metaxas, D.: The Best of Both Worlds: Combining 3D Deformable Models with Active Shape Models. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–7 (2007)
Wang, Q., Zhang, W., Tang, X., Shum, H.: Real-Time Bayesian 3-D Pose Tracking. IEEE Transactions on Circuits and Systems for Video Technology 16(12), 1533–1541 (2006)
Xiao, R., Zhu, L., Zhang, H.: Boosting chain learning for object detection. In: Proceedings. Ninth IEEE International Conference on Computer Vision, pp. 709–715 (2003)
Yan, S., Li, M., Zhang, H., Cheng, Q.: Ranking prior likelihood distributions for Bayesian shape localization framework. In: Ninth IEEE International Conference on Computer Vision, pp. 51–58 (2003)
Dementhon, D., Davis, L.: Model-based object pose in 25 lines of code. International Journal of Computer Vision 15(1), 123–141 (1995)
Xin, L., Wang, Q., Tao, J., Tang, X., Tan, T., Shum, H.: Automatic 3D Face Modeling from Video. In: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV 2005) (2005)
Martinez, A., Benavente, R.: The AR face database. CVC Technical Reportno 24 (1998)
Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTSDB: The Extended M2VTS Database. In: Second International Conference on Audio and Video-based Biometric Person Authentication, vol. 626 (1999)
Hu, Y., Jiang, D., Yan, S., Zhang, L., et al.: Automatic 3D reconstruction for face recognition. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 843–848 (2004)
Ostermann, J.: Face animation in mpeg-4 (2002)
Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: IEEE Conference on Computer Vision and Pattern Recognition (2005)
Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1(4), 321–331 (1988)
Brox, T., Rosenhahn, B., Cremers, D., Seidel, H.: High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In: European Conference on Computer Vision, ECCV, vol. 6, pp. 98–111 (2006)
Shi, J., Tomasi, C.: Good features to track. In: Proceedings of the 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600 (1994)
Torr, P., Davidson, C.: IMPSAC: Synthesis of importance sampling and random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(3), 354–364 (2003)
Hill, A., Cootes, T., Taylor, C.: Active Shape Models and the shape approximation problem. Image and Vision Computing 14(8), 601–607 (1996)
Lu, C., Hager, G., Mjolsness, E.: Fast and globally convergent pose estimation from video images. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(6), 610–622 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, W., Wang, Q., Tang, X. (2008). Real Time Feature Based 3-D Deformable Face Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_53
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)