Abstract
We propose a new technique for simultaneously executing face deformation modeling and 3D head pose estimation. Previous methods for estimating 3D head pose require a preliminary training stage for the head model, and cannot start tracking the head pose until this stage is complete. In contrast, our proposed method can acquire and refine a user’s deformable head model in parallel with tracking the user’s head pose. This allows progressive improvement in the accuracy of the estimation of head pose and face deformation.
Our technique consists of three main steps. In the first step we estimate the 3D head pose using a head model that is obtained automatically. The second step finds true positions of feature points by using the resulting poses of the first step. Finally, the basis vectors of face deformation are calculated from the true positions of feature points to acquire a new deformable head model as a linear combination of the basis vectors.
The iteration of the three steps refines the deformable head model, thus improving the accuracy of head pose estimation progressively. The improvement has been successfully demonstrated via experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Azarbayejani, A., Starner, T., Horowitz, B., Pentland, A.: Visually controlled graphics. IEEE Trans. Pattern Analysis and Machine Intelligence 15(6), 602–605 (1993)
Black, M., Yacoob, Y.: Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In: Proc. IEEE ICCV 1995, pp. 374–381 (1995)
De Carlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. Int. J. Computer Vision 38(2), 99–127 (2000)
Del Bue, A., Smeraldi, F., Agapito, L.: Non-rigid structure from motion using non-parametric tracking and non-linear optimization. In: Proc. IEEE CVPRW 2004. Articulated and Non-Rigid Motion, vol. 1 (2004)
Dornaika, F., Davoine, F.: Head and facial animation tracking using appearance-adaptive models and particle filters. In: Proc. IEEE CVPRW 2004, Real-Time Vision for Human-Computer Interaction, vol. 10 (2004)
Fua, P.: Using model-driven bundle-adjustment to model heads from raw video sequences. In: Proc. IEEE ICCV 1999, vol. 1, pp. 46–53 (1999)
Gee, A., Cipolla, R.: Fast visual tracking by temporal consensus. Image and Vision Computing 14, 105–114 (1996)
Gokturk, S., Bouguet, J., Grzeszczuk, R.: A data-driven model for monocular face tracking. In: Proc. IEEE ICCV, vol. 2, pp. 701–708 (2001)
Isard, M., Blake, A.: Condensation– conditional density propagation for visual tracking. Int. J. Computer Vision 29(1), 5–28 (1998)
Jebara, T., Pentland, A.: Parametrized structure from motion for 3D adaptive feedback tracking of faces. In: Proc. IEEE CVPR 1997, pp. 144–150 (1997)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Int. Joint Conf. Artificial Intelligence, pp. 674–679 (1981)
Lao, S., Kozuru, T., Okamoto, T., Yamashita, T., Tabata, N., Kawade, M.: A fast 360-degree rotation invariant face detection system. In: Demo session of IEEE ICCV 2003 (2003)
MacCormick, J., Isard, M.: Partitioned sampling, articulated objects, and interface-quality hand tracking. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 3–19. Springer, Heidelberg (2000)
Matsumoto, Y., Zelinsky, A.: An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement. In: Proc. IEEE FG 2000, pp. 499–504 (2000)
Matthews, I., Baker, S.: Active appearance models revisited. Int. J. Computer Vision 60(2), 135–164 (2004)
Morency, L., Rahimi, A., Darrell, T.: Adaptive view-based appearance models. In: Proc. IEEE CVPR 2003, vol. 1, pp. 803–810 (2003)
Oka, K., Sato, Y., Nakanishi, Y., Koike, H.: Head pose estimation system based on particle filtering with adaptive diffusion control. In: Proc. IAPR MVA 2005, pp. 586–589 (2005)
Shan, Y., Liu, Z., Zhang, Z.: Model-based bundle adjustment with application to face modeling. In: Proc. IEEE ICCV 2001, vol. 2, pp. 644–651 (2001)
Shi, J., Tomasi, C.: Good features to track. In: Proc. IEEE CVPR 1994, pp. 593–600 (1994)
Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3D tracking using online and offline information. IEEE Trans. Pattern Analysis and Machine Intelligence 26(10), 1380–1384 (2004)
Weng, J., Zhang, Y., Hwang, W.: Candid Covariance-Free Incremental Principal Component Analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 25(8), 1034–1040 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Oka, K., Sato, Y. (2005). Real-Time Modeling of Face Deformation for 3D Head Pose Estimation. In: Zhao, W., Gong, S., Tang, X. (eds) Analysis and Modelling of Faces and Gestures. AMFG 2005. Lecture Notes in Computer Science, vol 3723. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564386_24
Download citation
DOI: https://doi.org/10.1007/11564386_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29229-6
Online ISBN: 978-3-540-32074-6
eBook Packages: Computer ScienceComputer Science (R0)