Abstract
3D face tracking is an important component for many computer vision applications. Most state-of-the-art tracking algorithms can be characterized as being either intensity- or feature-based. The intensity-based tracker relies on the brightness constraint while the feature-based tracker utilizes 2D local feature correspondences. In this paper, we propose a hybrid tracker for robust 3D face tracking. Instead of relying on single source of information, the hybrid tracker integrates feature correspondence and brightness constraints within a nonlinear optimization framework. The proposed method can track the 3D face pose reliably in real-time. We have conducted a series of evaluations to compare the performance of the proposed tracker with other state-of-the-art trackers. The experiments consist of synthetic sequences with simulation of different environmental factors, real sequences with estimated ground truth, and sequences from a real-world HCI application. The proposed tracker is shown to be superior in both accuracy and robustness.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Fidaleo, D., Medioni, G., Fua, P., Lepetit, V.: An investigation of model bias in 3d face tracking. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, IEEE Computer Society Press, Los Alamitos (2005)
Black, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. IJCV 25(1), 23–48 (1997)
Basu, S., Essa, I., Pentland, A.: Motion regularization for model-based head tracking. In: ICPR 1996, vol. 3, pp. 611–616 (1996)
Cascia, M.L., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. PAMI 22(4), 322–336 (2000)
Xiao, J., Moriyama, T., Kanade, T., Cohn, J.: Robust full-motion recovery of head by dynamic templates and re-registration techniques. Internal Journal of Imaging Systems and Technology 13, 85–94 (2003)
Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. PAMI 26(10), 1385–1391 (2004)
Shan, Y., Liu, Z., Zhang, Z.: Model-based bundle adjustment with application to face modeling. In: ICCV 2001, vol. 2, pp. 644–651 (2001)
Schodl, A., Haro, A., Essa, I.: Head tracking using a textured polygonal model. In: Proceedings of Perceptual User Interfaces Workshop (held in Conjunction with ACM UIST 1998), ACM Press, New York (1998)
DeCarlo, D., Metaxas, D.: The integration of optical flow and deformable models with applications to human face shape and motion estimation. In: CVPR 1996, pp. 231–238 (1996)
Lu, L., Dai, X., Hager, G.: Efficient particle filtering using ransac with application to 3d face tracking. IVC 24(6), 581–592 (2006)
Baker, S., Matthews, I.: Lucas-kanade 20 years on: A unifying framework. IJCV 56(3), 221–255 (2004)
Baker, S., Patil, R., Cheung, K.M., Matthews, I.: Lucas-kanade 20 years on: Part 5. Technical Report CMU-RI-TR-04-64, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (November 2004)
Huber, P.J.: Robust Statistics. Wiley, New York (1981)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)
FaceVision200: Geometrix, http://www.geometrix.com
Photomodeler, http://www.photomodeler.com
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liao, WK., Fidaleo, D., Medioni, G. (2007). Integrating Multiple Visual Cues for Robust Real-Time 3D Face Tracking. In: Zhou, S.K., Zhao, W., Tang, X., Gong, S. (eds) Analysis and Modeling of Faces and Gestures. AMFG 2007. Lecture Notes in Computer Science, vol 4778. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75690-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-75690-3_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75689-7
Online ISBN: 978-3-540-75690-3
eBook Packages: Computer ScienceComputer Science (R0)