Model of Facial Expressions Management for an Embodied Conversational Agent

  • Radosław Niewiadomski
  • Catherine Pelachaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4738)


In this paper we present a model of facial behaviour encompassing interpersonal relations for an Embodied Conversational Agent (ECA). Although previous solutions of this problem exist in ECA’s domain, in our approach a variety of facial expressions (i.e. expressed, masked, inhibited, and fake expressions) is used for the first time. Moreover, our rules of facial behaviour management are consistent with the predictions of politeness theory as well as the experimental data (i.e. annotation of the video-corpus). Knowing the affective state of the agent and the type of relations between interlocutors the system automatically adapts the facial behaviour of an agent to the social context. We present also the evaluation study we have conducted of our model. In this experiment we analysed the perception of interpersonal relations from the facial behaviour of our agent.


Embodied conversational agents social context facial expressions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Albrecht, I., Schröder, M., Haber, J., Seidel, H.: Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Reality 8(4), 201–212 (2005)CrossRefGoogle Scholar
  2. 2.
    André, E., Rehm, M., Minker, W., Buhler, D.: Endowing spoken language dialogue systems with emotional intelligence. In: André, E., Dybkjaer, L., Minker, W., Heisterkamp, P. (eds.) Affective Dialogue Systems, pp. 178–187. Springer, Heidelberg (2004)Google Scholar
  3. 3.
    Ballin, D., Gillies, M.F., Crabtree, I.B.: A framework for interpersonal attitude and non-verbal communication in improvisational visual media production, First European Conference on Visual Media Production (CVMP), pp. 203–210 (2004)Google Scholar
  4. 4.
    Bevacqua, E., Mancini, M., Niewiadomski, R., Pelachaud, C.: An expressive ECA showing complex emotions. In: Proceedings of the AISB Annual Convention, Newcastle, UK, pp. 208–216 (2007)Google Scholar
  5. 5.
    Brown, P., Levinson, S.C.: Politeness: some universals on language usage. Cambridge University Press, Cambridge (1987)Google Scholar
  6. 6.
    Buck, R., Losow, J., Murphy, M., Costanzo, P.: Social facilitation and inhibition of emotional expression and communication. Journal of Personality and Social Psychology 63(6), 962–968 (1992)CrossRefGoogle Scholar
  7. 7.
    Cassell, J., Bickmore, T.: Negotiated Collusion. Modeling Social Language and its Relationship Effects in Intelligent Agents 13(1-2), 89–132 (2003)Google Scholar
  8. 8.
    De Carolis, B., Pelachaud, C., Poggi, I., De Rosis, F.: Behavior Planning for a Reflexive Agent. In: Proceedings of IJCAI 2001, Oporto, Portugal (April 2001)Google Scholar
  9. 9.
    Ekman, P., Friesen, W.V.: The Repertoire of Nonverbal Behavior’s: Categories, Origins, Usage and Coding. Semiotica 1, 49–98 (1969)Google Scholar
  10. 10.
    Goleman, D.: Inteligencja Emocjonalna, Media Rodzina, Poznan (1997)Google Scholar
  11. 11.
    Gross, J.J., Oliver, P., Richards, J.M.: The Dissociation of Emotion Expression from Emotion Experience: a Personality Perspective, Personality and Social Psychology Bulletin 26(6), 712–726 (2000)Google Scholar
  12. 12.
    Johnson, W.L., Rizzo, P., Bosma, W., Kole, S., Ghijsen, M., van Welbergen, H.: Generating Socially Appropriate Tutorial Dialog. In: ISCA Workshop on Affective Dialogue Systems, pp. 254–264 (2004)Google Scholar
  13. 13.
    La France, M., Hecht, M.A.: Option or Obligation to Smile: The Effects of Power and Gender and Facial Expression. In: Philippot, P., Feldman, R.S., Coats, E.J. (eds.) The Social Context of Nonverbal Behavior (Studies in Emotion and Social Interaction), pp. 45–70. Cambridge University Press, Cambridge (2005)Google Scholar
  14. 14.
    Manstead, A.S.R., Fischer, A.H., Jakobs, E.B.: The Social and Emotional Functions of Facial Displays. In: Philippot, P., Feldman, R.S., Coats, E.J. (eds.) The Social Context of Nonverbal Behavior (Studies in Emotion and Social Interaction), pp. 287–316. Cambridge University Press, Cambridge (2005)Google Scholar
  15. 15.
    Niewiadomski, R.: A model of complex facial expressions in interpersonal relations for animated agents, Ph.D. dissertation, University of Perugia (2007)Google Scholar
  16. 16.
    Prendinger, H., Ishizuka, M.: Social role awareness in animated agents. In: Proceedings of the fifth international conference on Autonomous agents, Montreal, Quebec, Canada, pp. 270–277 (2001)Google Scholar
  17. 17.
    Reeves, B., Nass, C.: The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge (1996)Google Scholar
  18. 18.
    Rehm, M., André, E.: Informing the Design of Embodied Conversational Agents by Analysing Multimodal Politeness Behaviors in Human-Human Communication, In: Workshop on Conversational Informatics for Supporting Social Intelligence and Interaction (2005)Google Scholar
  19. 19.
    Wagner, H., Lee, V.: Facial Behavior Alone and in the Presence of Others. In: Philippot, P., Feldman, R.S., Coats, E.J. (eds.) The Social Context of Nonverbal Behavior (Studies in Emotion and Social Interaction), pp. 262–286. Cambridge University Press, Cambridge (2005)Google Scholar
  20. 20.
    Wagner, H.L., Smith, J.: Facial expression in the presence of friends and strangers. Journal of Nonverbal Behavior 15(4), 201–214 (1991)CrossRefGoogle Scholar
  21. 21.
    Walker, M., Cahn, J., Whittaker, S.: Linguistic style improvisation for lifelike computer characters. In: Proceedings of the AAAI Workshop on AI, Alife and Entertainment (1996)Google Scholar
  22. 22.
    Wiggins, J.S., Trapnell, P., Phillips, N.: Psychometric and geometric characteristics of the Revised Interpersonal Adjective Scales (IAS-R). Multivariate Behavioral Research 23(3), 517–530 (1988)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Radosław Niewiadomski
    • 1
  • Catherine Pelachaud
    • 1
  1. 1.IUT de Monreuil, Université Paris 8France

Personalised recommendations