Automatic and Self-adaptive Facial Expression Tracking

  • Jing Chi
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 62)


We present a novel automatic and self-adaptive technique for facial expression tracking. The face point clouds are acquired at video rate with 3D scanner or reconstructed from 2D images. A single time-varying deformable mesh model is computed with our new metric to track these point clouds. A normal constraint is presented and introduced in the new metric to measure the direction consistency of vertex normal and vertex motion in the tracking process. Combined with other constraints in the new metric, it can automatically ensure the vertices move to optimal position. The normal constraint also works effectively where the model is very different from the point clouds in geometry. Experiment results show that the new technique can well track facial expression without manual aid.


Facial Expression Point Cloud Optical Flow Iterative Close Point Normal Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. Lien, T. Kanade, A. Zlochower, J. Cohn, and C. Li. 1998. Subtly different facial expression recognition and expression intensity estimation. In CVPR’98, 853–859.Google Scholar
  2. 2.
    Y. Yacoob and L. Davis. 1994. Computing spatio-temporal representations of human faces. In CVPR’94, 70–75.Google Scholar
  3. 3.
    Pighin, F., Salesin, D. H., and Szeliski, R. 1999. Resynthesizing facial animation through 3D model-based tracking. In Proc. Int. Conf. on Computer Vision, 143–150.Google Scholar
  4. 4.
    Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In Proceedings of EUROGRAPHICS, vol. 22, 641–650.Google Scholar
  5. 5.
    Y. Wang, X. Huang, C-S. Lee, S. Zhang, Z. Li, D. Samaras, D. Metaxas, A. Elgammal, and P. Huang. 2004. High Resolution Acquisition, Learning and Transfer of Dynamic 3-D Facial Expressions. Computer Graphics Forum, 23(3), 677–686.CrossRefGoogle Scholar
  6. 6.
    Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for nonrigid registration of depth scans. In Symposium on Geometry Processing, 27(5).Google Scholar
  7. 7.
    M. Wand, B. Adams, M. Ovsjanikov, A. Berner, M. Bokeloh, P. Jenke, L. Guibas, H.-P. Seidel, A. Schilling. 2009. Efficient Reconstruction of Nonrigid Shape and Motion from Real-Time 3D Data. ACM Transactions on Graphics 28(2), #15.CrossRefGoogle Scholar
  8. 8.
    J. Feldmar and N. Ayache. 1996. Rigid, affine and locally affine registration of free-form surfaces. Int. Journal of Computer Vision, 18(2), 99–119.CrossRefGoogle Scholar
  9. 9.
    Xiaolei Huang, Song Zhang, Yang Wang, D. Metaxas, D., Samaras. 2004. A Hierarchical Framework For High Resolution Facial Expression Tracking. In proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop, 22–29.Google Scholar
  10. 10.
    B. Amberg, S. Romdhani, T.Vetter. 2007. Optimal Step Nonrigid ICP Algorithms for Surface Registration. IEEE Conference on Computer Vision and Pattern Recognition, 1–8.Google Scholar
  11. 11.
    Yang Wang, Mohit Gupta, Song Zhang, Sen Wang, Xianfeng Gu. 2007. dimistris Samaras and Peisen Huang. High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps, international journal of computer vision, 76(3), 283–300.CrossRefGoogle Scholar
  12. 12.
    B. Bickel, M. Botsch, R. Angst, W. Matusik, M. Otaduy, H., Pfister, M. Gross. 2007. Multi-Scale Capture of Facial Geometry and Motion. ACM Tansactions on Graphics (TOG), 26(3), #33.CrossRefGoogle Scholar
  13. 13.
    Decarlo, D., and Metaxas, D. 2002. Adjusting shape parameters using model based optical flow residuals. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(6), 814–823.CrossRefGoogle Scholar
  14. 14.
    D. DeCarlo and D. Metaxas. 2000. Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision, 38(2), 99–127.zbMATHCrossRefGoogle Scholar
  15. 15.
    Li Zhang, Noah Snavely, Brain Curless, Steven M. Seitz. 2004. Spacetime Faces: High Resolution Capture for Modeling and Animation. ACM Transaction on Graphics, 23(3), 548–558.CrossRefGoogle Scholar
  16. 16.
    Zhu, C., Byrd, R. H., Lu, P., and Nocedal, J. 1997. Algorithm 778. L-BFGS-B: Fortran subroutines for Large-Scale bound constrained optimization. ACM Transactions on Mathematical Software 23, 4 (Dec.), 550–560.CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Jing Chi
    • 1
  1. 1.Department of Computer Science and TechnologyShandong Economic UniversityJi’nanChina

Personalised recommendations