Animated Faces for Robotic Heads: Gaze and Beyond

  • Samer Al Moubayed
  • Jonas Beskow
  • Jens Edlund
  • Björn Granström
  • David House
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6800)


We introduce an approach to using animated faces for robotics where a static physical object is used as a projection surface for an animation. The talking head is projected onto a 3D physical head model.

In this chapter we discuss the different benefits this approach adds over mechanical heads. After that, we investigate a phenomenon commonly referred to as the Mona Lisa gaze effect. This effect results from the use of 2D surfaces to display 3D images and causes the gaze of a portrait to seemingly follow the observer no matter where it is viewed from. The experiment investigates the perception of gaze direction by observers. The analysis shows that the 3D model eliminates the effect, and provides an accurate perception of gaze direction. We discuss at the end the different requirements of gaze in interactive systems, and explore the different settings these findings give access to.


Facial Animation Talking Heads Shader Lamps Robotic Heads Gaze Mona Lisa Effect Avatar Dialogue System Situated Interaction 3D Projection Gaze Perception 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Beskow, J., et al.: Face-to-face interaction and the KTH Cooking Show. In: Esposito, A., et al. (eds.) Development of Multimodal Interfaces: Active Listening and Synchrony, pp. 157–168. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Ruttkay, Z., Pelachaud, C. (eds.): From Brows till Trust: Evaluating Embodied Conversational Agents. Kluwer, Dordrecht (2004)zbMATHGoogle Scholar
  3. 3.
    Pelachaud, C.: Modeling Multimodal Expression of Emotion in a Virtual Agent. Philosophical Transactions of Royal Society B Biological Science, B 364, 3539–3548 (2009)CrossRefGoogle Scholar
  4. 4.
    Granstrm, B., House, D.: Modeling and evaluating verbal and non-verbal communication in talking animated interface agents. In: Dybkjaer, l., Hemsen, H., Minker, W. (eds.) Evaluation of Text and Speech Systems, pp. 65–98. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Shinozawa, K., Naya, F., Yamato, J., Kogure, K.: Differences in effect of robot and screen agent recommendations on human decision-making. International Journal of Human Computer Studies 62(2), 267–279 (2005)CrossRefGoogle Scholar
  6. 6.
    Todorovi, D.: Geometrical basis of perception of gaze direction. Vision Research 45(21), 3549–3562 (2006)CrossRefGoogle Scholar
  7. 7.
    Gockley, R., Simmons, J., Wang, D., Busquets, C., DiSalvo, K., Caffrey, S., Rosenthal, J., Mink, S., Thomas, W., Adams, T., Lauducci, M., Bugajska, D., Perzanowski, Schultz, A.: Grace and George: Social Robots at AAAI. In: Proceedings of AAAI 2004. Mobile Robot Competition Workshop, pp. 15–20. AAAI Press, Menlo Park (2004)Google Scholar
  8. 8.
    Sosnowski, S., Mayer, C., Kuehnlenz, K., Radig, B.: Mirror my emotions! Combining facial expression analysis and synthesis on a robot. In: Proceedings of the Thirty Sixth Annual Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour, AISB 2010 (2010)Google Scholar
  9. 9.
    Raskar, R., Welch, G., Low, K.-L., Bandyopadhyay, D.: Shader lamps: animating real objects with image-based illumination. In: Proc. of the 12th Eurographics Workshop on Rendering Techniques, pp. 89–102 (2001)Google Scholar
  10. 10.
    Lincoln, P., Welch, G., Nashel, A., Ilie, A., State, A., Fuchs, H.: Animatronic shader lamps avatars. In: Proc. of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2009). IEEE Computer Society, Washington, DC (2009)Google Scholar
  11. 11.
    Beskow, J.: Talking heads – Models and applications for multimodal speech synthesis. Doctoral dissertation, KTH (2003)Google Scholar
  12. 12.
    Parke, F.I.: Parameterized Models for Facial Animation. IEEE Computer Graphics and Applications 2(9), 61–68 (1982)CrossRefGoogle Scholar
  13. 13.
    Kendon, A.: Some functions of gaze direction in social interaction. Acta Psychologica 26, 22–63 (1967)CrossRefGoogle Scholar
  14. 14.
    Argyle, M., Cook, M.: Gaze and mutual gaze. Cambridge University Press, Cambridge (1976) ISBN: 978-0521208659Google Scholar
  15. 15.
    Kleinke, C.L.: Gaze and eye contact: a research review. Psychological Bulletin 100, 78–100 (1986)CrossRefGoogle Scholar
  16. 16.
    Takeuchi, A., Nagao, K.: Communicative facial displays as a new conversational modality. In: Proc. of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems (1993)Google Scholar
  17. 17.
    Bilvi, M., Pelachaud, C.: Communicative and statistical eye gaze predictions. In: Proc. of International Conference on Autonomous Agents and Multi-Agent Systems, Melbourne, Australia (2003)Google Scholar
  18. 18.
    Gregory, R.: Eye and Brain: The Psychology of Seeing. Princeton University Press, Princeton (1997)Google Scholar
  19. 19.
    Delaunay, F., de Greeff, J., Belpaeme, T.: A study of a retro-projected robotic face and its effectiveness for gaze reading by humans. In: Proc. of the 5th ACM/IEEE International Conference on Human-robot Interaction, pp. 39–44. ACM, New York (2010)Google Scholar
  20. 20.
    Edlund, J., Nordstrand, M.: Turn-taking gestures and hour-glasses in a multimodal dialogue system. In: Proc. of ISCA Workshop on Multi-Modal Dialogue in Mobile Environments, Kloster Irsee, Germany (2002)Google Scholar
  21. 21.
    Edlund, J., Beskow, J.: MushyPeek - a framework for online investigation of audiovisual dialogue phenomena. Language and Speech 52(2-3), 351–367 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Samer Al Moubayed
    • 1
  • Jonas Beskow
    • 1
  • Jens Edlund
    • 1
  • Björn Granström
    • 1
  • David House
    • 1
  1. 1.Department of Speech, Music and HearingKTH Royal Institute of TechnologyStockholmSweden

Personalised recommendations