Abstract
In this paper we present an Embodied Conversational Agent (ECA) model able to display rich verbal and non-verbal behaviors. The selection of these behaviors should depend not only on factors related to her individuality such as her culture, her social and professional role, her personality, but also on a set of contextual variables (such as her interlocutor, the social conversation setting), and other dynamic variables (belief, goal, emotion). We describe the representation scheme and the computational model of behavior expressivity of the Expressive Agent System that we have developed. We explain how the multi-level annotation of a corpus of emotionally rich TV video interviews can provide context-dependent knowledge as input for the specification of the ECA (e.g. which contextual cues and levels of representation are required for enabling the proper recognition of the emotions).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abrilian, S., Devillers, L., Buisine, S., Martin, J.-C.: Emotv1: Annotation of real-life emotions for the specification of multimodal affective interfaces. In: HCI International 2005, Las Vegas, USA (2005)
Abrilian, S., Martin, J.-C., Devillers, L.: A corpus-based approach for the modeling of multimodal emotional behaviors for the specification of embodied agents. In: HCI International 2005, Las Vegas, USA (2005)
Allwood, J., Cerrato, L., Dybkær, L., Paggio, P.: The mumin multimodal coding scheme. In: Workshop on Multimodal Corpora and Annotation, Stockholm (2004)
Allwood, J.: Cooperation and flexibility in multimodal communication. In: Bunt, H., Beun, R.-J. (eds.) CMC 1998. LNCS (LNAI), vol. 2155, pp. 113–124. Springer, Heidelberg (2001)
Atifi, H., Marcoccia, M.: L’expression et la mise en scéne des émotions á la télévision: l’articulation des émotions vécues, racontées et attribuées. In: Oralité et gestualité (ORAGE 2001): Interactions et comportements multimodaux dans la communication, Aix-en-Provence, pp. 179–182. L’Harmattan (2001)
Badam, V.R.K., Gharpure, C.P.: A stochastic and multi-layered model for personality in computational agents. In: Personality in Computational Agents, Logan, UT, USA, Department of Computer Science of Utah State University (2002)
Batliner, A., Fisher, K., Huber, R., Spilker, J., Noth, E.: Desperately seeking emotions or: Actors, wizards, and human beings. In: SpeechEmotion 2000, pp. 195–200 (2000)
Boone, R.T., Cunningham, J.G.: Children’s understanding of emotional meaning in expressive body movement. Poster presented at Biennial Meeting of the Society for Research in Child Development, Washington, DC (1996)
Boone, R.T., Cunningham, J.G.: Children’s decoding of emotion in expressive body movement: The development of cue attunement. Developmental Psychology 34(5), 1007–1016 (1998)
Calbris, G.: The semiotics of French gestures. University Press, Bloomington (1990)
Cassell, J., Stone, M., Hao, Y.: Coordination and context-dependence in the generation of embodied conversation. In: INLG, pp. 171–178 (2000)
Cowie, R.: Emotion recognition in human-computer interaction. IEEE Signal processing Magazine (18) (2001)
Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schroeder, M.: ’feeltrace’: An instrument for recording perceived emotion in real time. In: ISCA Workshop on Speech & Emotion, Northern Ireland, pp. 19–24 (2000)
Craggs, R., Wood, M.M.: A categorical annotation scheme for emotion in the linguistic content. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 89–100. Springer, Heidelberg (2004)
DeCarolis, B., Pelachaud, C., Poggi, I., de Rosis, F.: Behavior planning for a reflexive agent. In: IJCAI 2001, Seattle, USA (August 2001)
DeCarolis, B., Pelachaud, C., Poggi, I., Steedman, M.: APML, a mark-up language for believable behavior generation. In: Prendinger, H., Ishizuka, M. (eds.) Life-like Characters. Tools, Affective Functions and Applications, pp. 65–85. Springer, Heidelberg (2004)
deMeijer, M.: The contribution of general features of body movements to the attibution of emotions. Journal of Nonverbal behavior 13, 247–268 (1989)
deMeijer, M.: The attribution of agression and grief to body movements: the effect of sex-stereotypes. European Journal of Social Psychology 21, 249–259 (1991)
Douglas-Cowie, E., Campbell, N., Cowie, R., Roach, P.: Emotional speech; towards a new generation of databases. Speech Communication (40) (2003)
Ekman, P.: Basic emotions. In: Dalgleish, T., Power, M.J. (eds.) Handbook of Cognition & Emotion, pp. 301–320. John Wiley, New York (1999)
Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press, Inc., Palo Alto (1978)
Ekman, P., Rosenberg, E. (eds.): What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (Facs) (Series in Affective Science). Oxford Univ. Press, Oxford (1998)
Feyereisen, P., de Lannoy, J.D.: Gestures and speech: Psychological investigations. Cambridge University Press, Cambridge (1991)
Gallaher, P.E.: Individual differences in nonverbal behavior: Dimensions of style. Journal of Personality and Social Psychology 63(1), 133–145 (1992)
Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis. In: Computer Animation 2002, Geneva, Switzerland. IEEE Computer Society Press, Los Alamitos (2002)
Kendon, A.: Human gesture. In: Ingold, T., Gibson, K. (eds.) Tools, Language and Intelligence. Cambridge University Press, Cambridge (1993)
Kipp, M.: Gesture Generation by Imitation. From Human Behavior to Computer Character Animation. PhD thesis, Universität des Saarlandes (2004)
Kshirsagar, S.: A multilayer personality model. In: SMARTGRAPH 2002: Proceedings of the 2nd international symposium on Smart graphics, pp. 107–115. ACM Press, New York (2002)
Maya, V., Lamolle, M., Pelachaud, C.: Influences on embodied conversational agent’s expressivity: Toward an individualization of the ecas. In: Proceedings of AISB 2004 (2004)
McNeill, D.: Hand and mind - what gestures reveal about thoughts. University of Chicago Press, Chicago (1992)
Montepare, J., Koff, E., Zaitchik, D., Albert, M.: The use of body movements and gestures as cues to emotions in younger and older adults. Journal of Nonverbal Behavior 23(2), 133–152 (1999)
Newlove, J.: Laban for actors and dancers. Routledge, New York (1993)
Pelachaud, C., Carofiglio, V., De Carolis, B., de Rosis, F.: Embodied contextual agent in information delivering application. In: First International Joint Conference on Autonomous Agents & Multi-Agent Systems (AAMAS), Bologna, Italy (July 2002)
Plutchik, R.: The psychology and Biology of Emotion. Harper Collins College, New York (1994)
Poggi, I., Pelachaud, C., de Rosis, F.: Eye communication in a conversational 3D synthetic agent. AI Communications 13(3), 169–181 (2000)
Prendinger, H., Descamps, S., Ishizuka, M.: Scripting affective communication with life-like characters in web-based interaction systems. Applied Artificial Intelligence 16(7-8), 519–553 (2002)
Rist, T., Schmitt, M.: Applying socio-psychological concepts of cognitive consistency to negotiation dialog scenarios with embodied conversational characters. In: Proc. of AISB 2002 Symposium on Animated Expressive Characters for Social Interactions, pp. 79–84 (2003)
Rousseau, D., Hayes-Roth, B.: Personality in synthetic agents. Technical report, Stanford Knowledge Systems Laboratory Report KSL-96-21 (1996)
Ruttkay, Z., van Moppes, V., Noot, H.: The jovial, the reserved and the robot. In: Proceedings of the AAMAS 2003 Ws on Embodied Conversational Characters as Individuals, Melbourne, Australia (2003)
Scherer, K.R.: Emotion. In: Hewstone, M., Stroebe, W. (eds.) Introduction to Social Psychology: A European perspective, pp. 151–191. Blackwell, Oxford (2000)
Scherer, K.R., Schorr, A., Johnstone, T. (eds.): Appraisal processes in emotion: Theory, Methods, Research. Oxford University Press, New York (2001)
Theune, M., Heylen, D., Nijholt, A.: Generating embodied information presentations. In: Stock, O., Zancanaro, M. (eds.) Multimodal Intelligent Information Presentation, pp. 47–69. Kluwer Academic Publishers, Dordrecht (2004) ISBN=1-4020-3049-5
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lamolle, M., Mancini, M., Pelachaud, C., Abrilian, S., Martin, JC., Devillers, L. (2005). Contextual Factors and Adaptative Multimodal Human-Computer Interaction: Multi-level Specification of Emotion and Expressivity in Embodied Conversational Agents. In: Dey, A., Kokinov, B., Leake, D., Turner, R. (eds) Modeling and Using Context. CONTEXT 2005. Lecture Notes in Computer Science(), vol 3554. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11508373_17
Download citation
DOI: https://doi.org/10.1007/11508373_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26924-3
Online ISBN: 978-3-540-31890-3
eBook Packages: Computer ScienceComputer Science (R0)