’You are Sooo Cool, Valentina!’ Recognizing Social Attitude in Speech-Based Dialogues with an ECA

  • Fiorella de Rosis
  • Anton Batliner
  • Nicole Novielli
  • Stefan Steidl
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4738)


We propose a method to recognize the ’social attitude’ of users towards an Embodied Conversational Agent (ECA) from a combination of linguistic and prosodic features. After describing the method and the results of applying it to a corpus of dialogues collected with a Wizard of Oz study, we discuss the advantages and disadvantages of statistical and machine learning methods if compared with other knowledge-based methods.


Social Attitude Social Presence Acoustic Analysis Negative Comment Segment Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andersen, P.A., Guerrero, L.K.: Handbook of Communication and Emotions. Research, theory, applications and contexts. Academic Press, New York (1998)Google Scholar
  2. Ang, J., Dhillon, R., Krupsky, A., Shriberg, E., Stolcke, A.: Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: ICSLP, pp. 2037–2040 (2002)Google Scholar
  3. Bailenson, J.N., Swinth, K.R., Hoyt, C.L., Persky, S., Dimov, A., Blascovich, J.: The independent and interactive effects of embodied agents appearance and behavior on self-report, cognitive and behavioral markers of copresence in Immersive Virtual Environments. PRESENCE 14(4), 379–393 (2005)CrossRefGoogle Scholar
  4. Batliner, A., Fischer, K., Huber, R., Spilker, J., Nöth, E.: How to Find Trouble in Communication. Speech Communication 40, 117–143 (2003)zbMATHCrossRefGoogle Scholar
  5. Bickmore, T., Cassell, J.: Social Dialogue with Embodied Conversational Agents. In: van Kuppevelt, J., Dybkjaer, L., Bernsen, N. (eds.) Advances in Natural, Multimodal Dialogue Systems, pp. 1–32. Kluwer Academic, New York (2005)Google Scholar
  6. Blascovich, J.: Social influences within immersive virtual environments. In: Schroeder, R. (ed.) The social life of avatars, pp. 127–145. Springer, London (2002)Google Scholar
  7. Carberry, S., Lambert, L., Schroeder, L.: Towards recognizing and conveying an attitude of doubt via natural language. Applied Artificial Intelligence 16(7), 495–517 (2002)CrossRefGoogle Scholar
  8. de Rosis, F., Novielli, N.: From language to thought: inferring opinions and beliefs from verbal behavior. In: AISB 2007, Mindful Environments Workshop (2007)Google Scholar
  9. de Rosis, F., Novielli, N., Carofiglio, V., Cavalluzzi, A., De Carolis, B.: User modeling and adaptation in health promotion dialogs with an animated character. In: Journal of Biomedical Informatics, Special Issue on ’Dialog systems for health communications 39(5), 514–531 (2006)Google Scholar
  10. de Rosis, F., Novielli, N., Mazzotta, I.: Factors affecting the social attitude of users towards an ECA and how it is Worded. Submitted.Google Scholar
  11. Devillers, L., Vidrascu, L.: Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs. In: INTERSPEECH, pp. 801–804 (2006)Google Scholar
  12. Di Eugenio, B.: On the usage of Kappa to evaluate agreement on coding tasks. In: LREC 2000: Second International Conference on Language Resources and Evaluation, pp. 441–444 (2000)Google Scholar
  13. Hoorn, J.F., Konijn, E.A.: Perceiving and Experiencing Fictional Characters: An integrative account. Japanese Psychological Research 45(4), 250–268 (2003)CrossRefGoogle Scholar
  14. Lee, C.M., Narayanan, S.S., Pieraccini, R.: Combining acoustic and language information for emotion recognition. In: ICSPL, pp. 873–876 (2002)Google Scholar
  15. Liscombe, J., Riccardi, G., Hakkani-Tür, D.: Using context to improve emotion detection in spoken dialogue systems. In: Interspeech (2005)Google Scholar
  16. Litman, D., Forbes-Riley, K., Silliman, S.: Towards emotion prediction in spoken tutoring dialogues. In: HLT/NAACL, pp. 52–54 (2003)Google Scholar
  17. Nicholas, G., Rotaru, M., Litman, D.J.: Exploiting word-level features for emotion prediction. In: IEEE/ACL Workshop on Spoken Language Technology (SLT) (2006)Google Scholar
  18. Paiva, A. (ed.): Empathic Agents. In: Workshop in conjunction with AAMAS (2004)Google Scholar
  19. Polhemus, L., Shih, L.-F., Swan, K.: Virtual interactivity: the representation of social presence in an on line discussion. In: Annual Meeting of the American Educational Research Association (2001)Google Scholar
  20. Rettie, R.: Connectedness, awareness and social presence. In: PRESENCE, online proceedings (2003)Google Scholar
  21. Yu, C., Aoki, P.M., Woodruff, A.: Detecting user engagement in everyday conversations. In: ICSLP, pp. 1329–1332 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Fiorella de Rosis
    • 1
  • Anton Batliner
    • 2
  • Nicole Novielli
    • 1
  • Stefan Steidl
    • 2
  1. 1.Intelligent Interfaces, Department of Informatics, University of Bari, Via Orabona 4, 70126 BariItaly
  2. 2.Lehrstuhl für Mustererkennung, Universität Erlangen-Nürnberg, Martensstrasse 3, 91058 ErlangenF.R. of Germany

Personalised recommendations