Rigid and Non-rigid Face Motion Tracking by Aligning Texture Maps and Stereo-Based 3D Models

  • Fadi Dornaika
  • Angel D. Sappa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4179)


Accurate rigid and non-rigid tracking of faces is a challenging task in computer vision. Recently, appearance-based 3D face tracking methods have been proposed. These methods can successfully tackle the image variability and drift problems. However, they may fail to provide accurate out-of-plane face motions since they are not very sensitive to out-of-plane motion variations. In this paper, we present a framework for fast and accurate 3D face and facial action tracking. Our proposed framework retains the strengths of both appearance and 3D data-based trackers. We combine an adaptive appearance model with an online stereo-based 3D model. We provide experiments and performance evaluation which show the feasibility and usefulness of the proposed approach.


Appearance Model Facial Action Face Motion Active Appearance Model Mesh Vertex 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Moreno, F., Tarrida, A., Andrade-Cetto, J., Sanfeliu, A.: 3D real-time tracking fusing color histograms and stereovision. In: IEEE International Conference on Pattern Recognition (2002)Google Scholar
  2. 2.
    Cascia, M., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 322–336 (2000)CrossRefGoogle Scholar
  3. 3.
    Ahlberg, J.: An active model for facial feature tracking. EURASIP Journal on Applied Signal Processing 2002, 566–571 (2002)CrossRefGoogle Scholar
  4. 4.
    Matthews, I., Baker, S.: Active appearance models revisited. International Journal of Computer Vision 60, 135–164 (2004)CrossRefGoogle Scholar
  5. 5.
    Malassiotis, S., Strintzis, M.G.: Robust real-time 3D head pose estimation from range data. Pattern Recognition 38, 1153–1165 (2005)CrossRefGoogle Scholar
  6. 6.
    Besl, P., McKay, N.: A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 239–256 (1992)CrossRefGoogle Scholar
  7. 7.
    Ahlberg, J.: CANDIDE-3 - an updated parametrized face. Technical Report LiTH-ISY-R-2326, Department of Electrical Engineering, Linköping University, Sweden (2001)Google Scholar
  8. 8.
    Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2D+3D active appearance models. In: IEEE Int. Conference on Computer Vision and Pattern Recognition (2004)Google Scholar
  9. 9.
    Jepson, A., Fleet, D., El-Maraghi, T.: Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 1296–1311 (2003)CrossRefGoogle Scholar
  10. 10.
    Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Transactions on Circuits and Systems for Video Technology (in press)Google Scholar
  11. 11.
    Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 681–684 (2001)CrossRefGoogle Scholar
  12. 12.
    Dornaika, F., Sappa, A.: Appearance-based tracker: An evaluation study. In: IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (2005)Google Scholar
  13. 13.
    Chetverikov, D., Stepanov, D., Kresk, P.: Robust Euclidean alignment of 3D point sets: the trimmed iterative closet point algorithm. Image and Vision Computing 23, 299–309 (2005)CrossRefGoogle Scholar
  14. 14.
    Fitzgibbon, A.: Robust registration of 2D and 3D point sets. Image and Vision Computing 21, 1145–1153 (2003)CrossRefGoogle Scholar
  15. 15.
    Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communication ACM 24, 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Rousseeuw, P., Leroy, A.: Robust Regression and Outlier Detection. John Wiley & Sons, New York (1987)MATHCrossRefGoogle Scholar
  17. 17.
    Horn, B.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Amer. A. 4, 629–642 (1987)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Fadi Dornaika
    • 1
  • Angel D. Sappa
    • 1
  1. 1.Computer Vision Center, Campus UABBellaterra, BarcelonaSpain

Personalised recommendations