Modeling Emotional Expressions as Sequences of Behaviors

  • Radosław Niewiadomski
  • Sylwia Hyniewska
  • Catherine Pelachaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5773)


In this paper we present a system which allows a virtual character to display multimodal sequential expressions i.e. expressions that are composed of different signals partially ordered in time and belonging to different nonverbal communicative channels. It is composed of a language for the description of such expressions from real data and of an algorithm that uses this description to automatically generate emotional displays. We explain in detail the process of creating multimodal sequential expressions, from the annotation to the synthesis of the behavior.


virtual characters emotional expressions multimodality 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Haidt, J., Keltner, D.: Culture and facial expression: Open-ended methods find more expressions and a gradient of recognition. Cognition and Emotion 13(3), 225–266 (1999)CrossRefGoogle Scholar
  2. 2.
    Keltner, D.: Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology 68, 441–454 (1995)CrossRefGoogle Scholar
  3. 3.
    Wallbott, H.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)CrossRefGoogle Scholar
  4. 4.
    Ruttkay, Z.: Constraint-based facial animation. International Journal of Constraints 6, 85–113 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Paleari, M., Lisetti, C.: Psychologically grounded avatars expressions. In: First Workshop on Emotion and Computing at KI 2006, 29th Annual Conference on Artificial Intelligence, Bremen, Germany (2006)Google Scholar
  6. 6.
    Malatesta, L., Raouzaiou, A., Karpouzis, K., Kollias, S.D.: Towards modeling embodied conversational agent character profiles using appraisal theory predictions in expression synthesis. Appl. Intell. 30(1), 58–64 (2009)CrossRefGoogle Scholar
  7. 7.
    Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Scherer, K., Schorr, A., Johnstone, T. (eds.) Appraisal Processes in Emotion: Theory, Methods, Research, pp. 92–119. Oxford University Press, Oxford (2001)Google Scholar
  8. 8.
    Pan, X., Gillies, M., Sezgin, T.M., Loscos, C.: Expressing complex mental states through facial expressions. In: Second International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 745–746. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Lance, B., Marsella, S.: Emotionally expressive head and body movements during gaze shifts. In: Proceedings of the 7th International Conference on Intelligent Virtual Agents (IVA), pp. 72–85. Springer, Heidelberg (2007)Google Scholar
  10. 10.
    Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press (1978)Google Scholar
  11. 11.
    Ekman, P., Friesen, W.: Unmasking the Face. A guide to recognizing emotions from facial clues. Prentice-Hall, Inc., Englewood Cliffs (1975)Google Scholar
  12. 12.
    Bevacqua, E., Mancini, M., Niewiadomski, R., Pelachaud, C.: An expressive ECA showing complex emotions. In: Proceedings of the AISB Annual Convention, Newcastle, UK, pp. 208–216 (2007)Google Scholar
  13. 13.
    Niewiadomski, R., Hyniewska, S., Pelachaud, C.: Evaluation of multimodal sequential expressions of emotions in ECA. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII), Amsterdam, Holland. Springer, Heidelberg (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Radosław Niewiadomski
    • 1
  • Sylwia Hyniewska
    • 1
  • Catherine Pelachaud
    • 2
  1. 1.Telecom ParisTechParisFrance
  2. 2.CNRS-LTCIParisFrance

Personalised recommendations