Multimedia Tools and Applications

, Volume 74, Issue 21, pp 9427–9447 | Cite as

A framework for automatic and perceptually valid facial expression generation

  • Hui YuEmail author
  • Oliver Garrod
  • Rachael Jack
  • Philippe Schyns


Facial expressions are facial movements reflecting the internal emotional states of a character or in response to social communications. Realistic facial animation should consider at least two factors: believable visual effect and valid facial movements. However, most research tends to separate these two issues. In this paper, we present a framework for generating 3D facial expressions considering both the visual the dynamics effect. A facial expression mapping approach based on local geometry encoding is proposed, which encodes deformation in the 1-ring vector. This method is capable of mapping subtle facial movements without considering those shape and topological constraints. Facial expression mapping is achieved through three steps: correspondence establishment, deviation transfer and movement mapping. Deviation is transferred to the conformal face space through minimizing the error function. This function is formed by the source neutral and the deformed face model related by those transformation matrices in 1-ring neighborhood. The transformation matrix in 1-ring neighborhood is independent of the face shape and the mesh topology. After the facial expression mapping, dynamic parameters are then integrated with facial expressions for generating valid facial expressions. The dynamic parameters were generated based on psychophysical methods. The efficiency and effectiveness of the proposed methods have been tested using various face models with different shapes and topological representations.


Facial expression mapping Facial animation FACS Psychophysical Perceptually valid Face dynamics 


  1. 1.
    Asthana A, de la Hunty M, Dhall A, Goecke R (2012) Facial performance transfer via deformable models and parametric correspondence. IEEE Trans Vis Comput Graph 18(9):1511–1519, 460CrossRefGoogle Scholar
  2. 2.
    Chin S, Kim K (2009) Emotional intensity-based facial expression cloning for low polygonal applications. IEEE Trans Syst Man Cybern-Part C: Appl Rev 39(3)Google Scholar
  3. 3.
    Choe B, Lee H, Ko H (2001) Performance-driven muscle-based facial animation. J Vis Comput Animat 12:67–79CrossRefzbMATHGoogle Scholar
  4. 4.
    Cosker D, Borkett R, Marshall D, Rosin PL (2008) Towards automatic performance driven animation between multiple types of facial model”. IET Comput Vis 2(3):129–141CrossRefGoogle Scholar
  5. 5.
    Curio C, Breidt M, Kleiner M, Vuong Q, Giess M, Bulthoff H (2007) Semantic 3D motion retargeting for facial animation. In: Proceedings of the ACMAPGV 59–64Google Scholar
  6. 6.
    Deng Z, Noh JY (2007) Computer facial animation: a survey. Springer press, data-driven 3d facial animation 1–28Google Scholar
  7. 7.
    Do M, Carmo P (1976) Differential geometry of curves and surfaces. Prentice-hallGoogle Scholar
  8. 8.
    Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo AltoGoogle Scholar
  9. 9.
    Ersotelos N, Dong F (2008) Building highly realistic facial animation: a survey. Vis Comput 24(1):13CrossRefGoogle Scholar
  10. 10.
    Gao W, Chen Y, Wang R, Shan S, Jiang D (2003) Learning and synthesizing Mpeg-4 compatible 3d face animation from video sequence. IEEE Trans Circ Syst Video Technol 13(11):1119–1128CrossRefGoogle Scholar
  11. 11.
    Hartley R, Zisserman A (2004) Multiple view geometry. In computer vision 2nd Ed. Cambridge University PressGoogle Scholar
  12. 12.
    Kalogerakis E, Simari P, Nowrouzezahrai E, Singh K (2007) Robust statistical estimation of curvature on discretized surfaces proceedings of the fifth eurographics symposium on geometry processing 13–22Google Scholar
  13. 13.
    Kalra P, Mangili A, Magnenatthalmann N, Thalmann D (1992) Simulation of facial muscle actions based on rational free form deformations. In Eurographics’92, 59–69Google Scholar
  14. 14.
    Lee I, Mahmood MT, Shim S, Choi T Optimizing image focus for 3D shape recovery through genetic algorithm, multimedian tools and applications, doi: 10.1007/s11042-013-1433-9
  15. 15.
    Ma J, Cole R, Pellom B, Ward W, Wise B (2004) Accurate automatic visible speech synthesis of arbitrary 3d model based on concatenation of diviseme motion capture data. Comput Animat Virtual World 15:1–17CrossRefGoogle Scholar
  16. 16.
    Meyer M, Desbrun M, Schröder P, Barr AH (2003) Discrete differential-geometry operators for triangulated 2-manifolds. In: Proc. of visualization and mathematics 35–57Google Scholar
  17. 17.
    Noh J, Neumann U (2001) Expression cloning. Proc Siggraph 277–288Google Scholar
  18. 18.
    Parke F (1972) Computer generated animation of faces. In Acm National Conferences Vol 1. Acm Press 451–457Google Scholar
  19. 19.
    Parke FI (1982) Parameterized models for facial animation. IEEE Comput Graph Appl 2(9):61–68CrossRefGoogle Scholar
  20. 20.
    Parke FI, Waters K (1996) Computer facial animation. A K Peters, WellesleyGoogle Scholar
  21. 21.
    Popovic Z,Witkin A (1999) Physically based motion transformation. In proceedings of siggraph 99 11–20Google Scholar
  22. 22.
    Saracchini RFV, Jorge S, Leitão HCG, Atkinson GA, Smith ML (2013) Robust 3D face capture using example-based photometric stereo. Comput Ind 64(9):1399–1410CrossRefGoogle Scholar
  23. 23.
    Shin HJ, Lee Y (2009) Expression synthesis and transfer in parameter spaces. Comput Graph Forum 28(7):1829–1835CrossRefGoogle Scholar
  24. 24.
    Sifakis E, Selle A, Robinson-Mosher A, Fedkiw R (2006) Simulating speech with a physics-based facial muscle model. Eurographics 261–270Google Scholar
  25. 25.
    Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Ossl CR¨, Seidel H-P (2004) Laplacian surface editing. In Proc. of eurographics symposium on geometry processing 04; 179–188Google Scholar
  26. 26.
    Sumner R, P J (2004) Deformation transfer for triangle meshes. In: Proceedings of the Acm siggraph 04; 399–405Google Scholar
  27. 27.
    Sun Y, Chen X, Rosato M, Yin LJ (2010) Tracking vertex flow and model adaptation for three-dimensional sp2atiotemporal face analysis. IEEE Trans Syst Man Cybern Part A Syst Hum 40(3):461–474CrossRefGoogle Scholar
  28. 28.
    Taubin G (2000) Geometric signal processing on polygonal meshes. In: Proc. of eurographics’2000: star-state of the art reportGoogle Scholar
  29. 29.
    Terzopoulos D, And K (1990) Waters. physically-based facial modelling, analysis, and animation. J Vis Comput Animat 1:73–80CrossRefGoogle Scholar
  30. 30.
    Vezzetti F, Marcolin F (2013) Geometry-based 3D face morphology analysis: soft-tissue landmark formalization, multimedia tools and applications, 10.1007/s11042-012-1091-3
  31. 31.
    Weng Y, Xu W, Wu Y, Zhou K, Guo BN (2006) 2D shape deformation using nonlinear least squares optimization the visual computer. 22(9)Google Scholar
  32. 32.
    Williams L (1990) Performance-driven facial animation. In Siggraph’90, Acm press 235–242Google Scholar
  33. 33.
    Ying S, Peng J, Du S, Qiao H (2009) A scale stretch method based on icp for 3d data registration. IEEE Trans Autom Sci Eng 6(3):559–565CrossRefGoogle Scholar
  34. 34.
    Yu H, Garrod O, Schyns P (2012) Perception-driven facial expression synthesis. Comput Graph 36(3):152–162CrossRefGoogle Scholar
  35. 35.
    Zhang Q, Liu Z, Guo B, Terzopoulos D, Shum H (2006) Geometry-driven photorealistic facial expression synthesis’. IEEE Trans Vis Comput Graph 12(1):48–60CrossRefGoogle Scholar
  36. 36.
    Zhang L, Snavely N, Curless B, Seitz SM (2004) Spacetime faces: high resolution capture for modeling and animation. ACM Trans Graph 23(3):548–558CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Hui Yu
    • 1
    Email author
  • Oliver Garrod
    • 2
  • Rachael Jack
    • 2
  • Philippe Schyns
    • 2
  1. 1.University of PortsmouthPortsmouthUK
  2. 2.University of GlasgowGlasgowUK

Personalised recommendations