Cognitive Computation

, Volume 6, Issue 4, pp 872–891 | Cite as

Applying a Text-Based Affective Dialogue System in Psychological Research: Case Studies on the Effects of System Behaviour, Interaction Context and Social Exclusion

  • Marcin SkowronEmail author
  • Stefan Rank
  • Aleksandra Świderska
  • Dennis Küster
  • Arvid Kappas


This article presents two studies conducted with an affective dialogue system in which text-based system–user communication was used to model, generate and present different affective and social interaction scenarios. We specifically investigated the influence of interaction context and roles assigned to the system and the participants, as well as the impact of pre-structured social interaction patterns that were modelled to mimic aspects of “social exclusion” scenarios. The results of the first study demonstrate that both the social context of the interaction and the roles assigned to the system influence the system evaluation, interaction patterns, textual expressions of affective states, as well as emotional self-reports. The results observed for the second study show the system’s ability to partially exclude a participant from a triadic conversation without triggering significantly different affective reactions or a more negative system evaluation. The experimental evidence provides insights on the perception, modelling and generation of affective and social cues in artificial systems that can be realized in different modalities, including the text modality, thus delivering valuable input for applying affective dialogue systems as tools for studying affect and social aspects in online communication.


Affective dialogue system Human–computer interaction Structuring affective and social interaction context Socially believable ICT interfaces 


  1. 1.
    Andre E, Dybkjaer L, Minker W, Heisterkamp PE. Affective dialogue systems, tutorial and research workshop (ads 2004). In: Lecture notes in computer science. Berlin: Springer; 2004.Google Scholar
  2. 2.
    Andre E, Rehm M, Minker W, Bühler. Endowing spoken language dialogue systems with emotional intelligence. Affect Dialogue Syst. 2004;3068:178–87.Google Scholar
  3. 3.
    Arnold J. Affect in language learning. Cambridge: Cambridge University Press; 1999.Google Scholar
  4. 4.
    Bevacqua E, De Sevin E, Hyniewska SJ, Pelachaud C. A listener model: introducing personality traits. J Multimodal User Interfaces. 2012;6:27–38.CrossRefGoogle Scholar
  5. 5.
    Bhatt K, Argamon S, Evens M. Hedged responses and expressions of affect in human/human and human/computer tutorial interactions. In: Proceedings of the 26th annual conference of the cognitive science society (CogSci04), 2004, pp. 114–119.Google Scholar
  6. 6.
    Blankenship V, Hnat SM, Hess TG, Brown DR. Reciprocal interaction and similarity of personality attributes. J Soc Pers Relat. 1984;1:415–32.Google Scholar
  7. 7.
    Blascovich J. A theoretical model of social influence for increasing the utility of collaborative virtual environments. In: Proceedings of the 4th International conference on collaborative virtual environments. 2002; doi: 10.1145/571878.571883.
  8. 8.
    Blascovich J, Loomis J, Beall AC, Swinth KR, Hoyt CL, Bailenson JN. Immersive virtual environment technology as a methodological tool for social psychology. Psychol Inq. 2002;13:103–24.CrossRefGoogle Scholar
  9. 9.
    Bradley M, Lang P. Affective norms for English words (anew): stimuli, instruction manual and affective ratings. University of Florida; 1999.Google Scholar
  10. 10.
    Buschmeier H, Kopp S. Co-constructing grounded symbols—feedback and incremental adaptation in human–agent dialogue. Künstliche Intelligenz. 2013;27:137–43.CrossRefGoogle Scholar
  11. 11.
    Calvo R, D’Mello S. Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput. 2010;1:18–37.CrossRefGoogle Scholar
  12. 12.
    Calvo RA, Kim S. Emotions in text: dimensional and categorical models. Comput Intell. 2012;29(3)527–43.Google Scholar
  13. 13.
    Chang C, Lin C. LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol. 2011; 2:27:1–27:27.Google Scholar
  14. 14.
    Cohen J. A power primer. Quantitative methods in psychology. Psychol Bull. 1992;112:155–9.Google Scholar
  15. 15.
    Davis MA. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology 1980;10:85.Google Scholar
  16. 16.
    Epley N, Akalis S, Waytz A, Cacioppo JT. Creating social connection through inferential reproduction. Psychol Sci. 2008;19:114–20.PubMedCrossRefGoogle Scholar
  17. 17.
    Fast L, Funder D. Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. J Pers Soc Psychol. 2008;94(2):334–46.PubMedCrossRefGoogle Scholar
  18. 18.
    Forsyth E, Martell C. Lexical and discourse analysis of online chat dialogue. In: Proceedings of the first IEEE international conference on semantic computing, 2007, pp. 19–26.Google Scholar
  19. 19.
    Garas A, Garcia D, Skowron M, Schweitzer F. Emotional persistence in online chatting communities. Sci Rep. 2012;2(402).Google Scholar
  20. 20.
    Gligorijevic V, Skowron M, Tadic B. Structure and stability of online chat network built on emotion-carrying links. Phys A Stat Mech Appl. 2012; 392: 538–543.Google Scholar
  21. 21.
    Gobron S, Ahn J, Quentin S, Thalmann D, Rank S, Skowron M, Paltoglou G, Thelwall M. An interdisciplinary VR-architecture for 3d chatting with non-verbal communication. In: Proceedings of joint virtual reality conference of EuroVR, 2011.Google Scholar
  22. 22.
    Gros C. Cognition and emotion: perspectives of a closing gap. Cogn Comput. 2010;2:78–85.CrossRefGoogle Scholar
  23. 23.
    Isbister K, Nass C. Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. Int J Hum Comput Stud. 2000;53:251–67.CrossRefGoogle Scholar
  24. 24.
    Jin B. How lonely people use and perceive Facebook. Comput Human Behav. 2013;29:2463–70.Google Scholar
  25. 25.
    Kappas A. Appraisals are direct, immediate, intuitive, and unwitting… and some are reflective. Cogn Emot. 2006;20:952–75.CrossRefGoogle Scholar
  26. 26.
    Kappas A, Kuester D, Theunis M, Tsankova E. Cyberemotions: subjective and physiological responses to reading online discussion forums. Psychophysiology 2010;47:S101.Google Scholar
  27. 27.
    Kassner MP, Wesselmann ED, Law AT, Williams KD. Virtually ostracized: studying ostracism in immersive virtual environments. Cyberpsychol Behav Soc Netw. 2012;15(8):399–403.PubMedCentralPubMedCrossRefGoogle Scholar
  28. 28.
    Minear M, Park DC. A lifespan database of adult facial stimuli. Behav Res Methods Instrum Comput. 2004;36:630–3.PubMedCrossRefGoogle Scholar
  29. 29.
    Moon Y. Intimate exchanges: using computers to elicit self-disclosure from consumers. J Consum Res. 2000;26:324–40.CrossRefGoogle Scholar
  30. 30.
    Moon Y, Nass C. How real are computer personalities? Psychological responses to personality types in human-computer interaction. Comm. Res. 1996;23:351–74.Google Scholar
  31. 31.
    Nass C, Isbister K, Lee EJ. Truth is beauty: researching embodied conversational agents. In: Embodied conversational agents, 2000, pp. 374–402.Google Scholar
  32. 32.
    Nass C, Moon Y. Machines and mindlessness: social responses to computers. J Soc Issues. 2000;56(1):81–103.CrossRefGoogle Scholar
  33. 33.
    Nass C, Moon Y, Carney P. Are respondents polite to computers? Social desirability and direct responses to computers. J Appl Soci Psychol. 1999;29:1093–110.CrossRefGoogle Scholar
  34. 34.
    Nass C, Moon Y, Fogg BJ, Reeves B, Dryer DC. Can computer personalities be human personalities? Int J Hum Comput Stud. 1995;1995(43):223–39.CrossRefGoogle Scholar
  35. 35.
    Nass C, Moon Y, Green N. Are computers gender-neutral? Social desirability and direct responses to computers. J Appl Soci Psychol. 1997;27:864–76.CrossRefGoogle Scholar
  36. 36.
    Nijholt A. Conversational agents and the construction of humorous acts, conversational informatics: an engineering approach. 2007, pp. 21–47.Google Scholar
  37. 37.
    Paiva A, Dias J, Sobral D, Aylett R, Woods S, Hall L, Zoll C. Learning by feeling: evoking empathy with synthetic characters. Appl Artif Intell. 2005;19:235–66.CrossRefGoogle Scholar
  38. 38.
    Paltoglou G, Gobron S, Skowron M, Thelwall M, Thalmann D. Sentiment analysis of informal textual communication in cyberspace. In: Proceedings of engage 2010, Springer LNCS State-of-the-Art Survey, 2010, pp. 13–25.Google Scholar
  39. 39.
    Paltoglou G, Theunis M, Kappas A, Thelwall M. Predicting emotional responses to long informal text. IEEE Trans Affect Comput. 2013;4:106–22.CrossRefGoogle Scholar
  40. 40.
    Pennebaker J, King L. Linguistic styles: language use as an individual difference. J Pers Soc Psychol. 1999;77(6):1296–312.PubMedCrossRefGoogle Scholar
  41. 41.
    Pennebaker J, Francis M, Booth R. Linguistic inquiry and word count: LIWC 2001. Lawrence: Erlbaum Publishers; 2001.Google Scholar
  42. 42.
    Pennebaker J, Mehl M, Niederhoffer K. Psychological aspects of natural language use: our words, our self. Annu Rev Psychol. 2003;54:547–77.PubMedCrossRefGoogle Scholar
  43. 43.
    Pillutla M, Murnighan J. Unfairness, anger, and spite: emotional rejections of ultimatum offers. Organ Behav Hum Decis Process. 1996;68(3):208–24.CrossRefGoogle Scholar
  44. 44.
    Pittermann J, Pittermann A, Minker W. Handling emotions in human-computer dialogues. Dordrecht: Springer; 2009.Google Scholar
  45. 45.
    Reeves B, Nass C. The media equation: how people treat computers, television, and new media like real people and places. New York: Cambridge University Press; 1996.Google Scholar
  46. 46.
    Sanfey AG, Rilling JK, Aronson JA, Nystrom LE, Cohen JD. The neural basis of economic decision-making in the Ultimatum Game. Science. 2003;300:1755–8.PubMedCrossRefGoogle Scholar
  47. 47.
    Schröder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, Maat M, McKeown G, Pammi S, Pantic M, Pelachaud C, Schuller B, Sevin E, Valstar M, Wöllmer M. Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput. 2012;3:165–83.CrossRefGoogle Scholar
  48. 48.
    Sienkiewicz J, Skowron M, Paltoglou G, Hołyst JA. Entropy growth in emotional online dialogues. J Phys: Conf Ser. 2013;410(1):012096.Google Scholar
  49. 49.
    Skowron M. Affect listeners. Acquisition of affective states by means of conversational systems. In: Proceedings of COST 2102 training school, LNCS; 2010, pp. 169–181.Google Scholar
  50. 50.
    Skowron M. Affect listeners. Acquisition of affective states by means of conversational systems. In: Development of multimodal interfaces—active listening and synchrony, Springer, Lecture Notes in Computer Science, 2010; pp. 169–181.Google Scholar
  51. 51.
    Skowron M, Paltoglou G. Affect bartender—affective cues and their application in a conversational agent. In: IEEE symposium series on computational intelligence 2011, IEEE workshop on affective computational intelligence, 2011.Google Scholar
  52. 52.
    Skowron M, Irran J, Krenn B. Computational framework for and the realization of cognitive agents providing intelligent assistance capabilities. In: ECAI. The 6th international cognitive robotics workshop, 2008, pp. 88–96.Google Scholar
  53. 53.
    Skowron M, Pirker H, Rank S, Paltoglou G, Ahn J, Gobron S. No peanuts! Affective cues for the virtual bartender. In: Proceedings of the Florida artificial intelligence research society conference. USA: AAAI Press; 2011.Google Scholar
  54. 54.
    Skowron M, Rank S. Interacting with collective emotions in e-Communities. In: Von Scheve C, Salmela M editors Collective emotions, perspectives from psychology, philosophy, and sociology. Oxford: Oxford University Press; 2014.Google Scholar
  55. 55.
    Skowron M, Rank S, Theunis M, Sienkiewicz J. The good, the bad and the neutral: affective profile in dialog system-user communication. In: Proceedings of fourth bi-annual affective computing and intelligent interaction conference—ACII 2011, LNCS, 2011, pp. 347–356.Google Scholar
  56. 56.
    Skowron M, Theunis M, Rank S, Borowiec A. Effect of affective profile on communication patterns and affective expressions in interactions with a dialog system. In: Proceedings of fourth bi-annual affective computing and intelligent interaction conference—ACII 2011, LNCS, 2011, pp. 337–346.Google Scholar
  57. 57.
    Skowron M, Theunis M, Rank S, Kappas A. Affect and social processes in online communication—experiments with affective dialogue system. IEEE Trans Affect Comput. 2013;4:267–79.CrossRefGoogle Scholar
  58. 58.
    Squartini S, Schuller B, Hussain A. Cognitive and emotional information processing for human-machine interaction. Cogn Comput. 2012;4(4):383-5Google Scholar
  59. 59.
    Stone P, Dexter C, Marshall S, Ogilvie D. The general inquirer: a computer approach to content analysis. Cambridge: MIT Press; 1966.Google Scholar
  60. 60.
    Tadic B, Gligorijevic V, Skowron M, Suvakov M. The dynamics of emotional chats with bots: experiment and agent-based simulations. ScienceJet. 2014;3:50.Google Scholar
  61. 61.
    Thelwall M, Buckley K, Paltoglou G, Cai D, Kappas A. Sentiment strength detection in short informal text. J Am Soc Inf Sci Technol. 2010;61:2544–58.CrossRefGoogle Scholar
  62. 62.
    Thelwall M, Buckley K, Paltoglou G, Skowron M, Garcia D, Gobron S, Ahn J, Kappas A, Kuster D, Holyst JA. Damping sentiment analysis in online communication: discussions, monologs and dialogs. In: Computational linguistics and intelligent text processing, 2013, pp. 1–12..Google Scholar
  63. 63.
    Tatai G, Laufer L. Extraction of affective components from texts and their use in natural language dialogue systems. Acta Cybernetica. 2004;16:625–42.Google Scholar
  64. 64.
    ter Maat M, Truong KP, Heylen D. How agents’ turn-taking strategies influence impressions and response behaviors. Presence. 2011;20(5):412–30.CrossRefGoogle Scholar
  65. 65.
    Traum D, DeVault D, Lee J, Wang Z, Marsella S. Incremental dialogue understanding and feedback for multiparty, multimodal conversation. In: Proceedings of the 12th international conference on intelligent virtual agents, 2012, pp. 275–288.Google Scholar
  66. 66.
    Valdesolo P, DeSteno D. Moral hypocrisy: social groups and the flexibility of virtue. Psychol Sci. 2007;18:689–90.PubMedCrossRefGoogle Scholar
  67. 67.
    van’t Wout M, Kahn RS, Sanfey AG, Aleman A. Affective state and decision making in the ultimatum game. Exp Brain Res. 2006;169:564–568.Google Scholar
  68. 68.
    Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schröder M. Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput. 2012;3:69–87.CrossRefGoogle Scholar
  69. 69.
    von der Pütten AM, Krämer NC, Gratch J, Kang S. It doesn’t matter what you are! - Explaining social effects ofagents and avatars. Comput Human Behav. 2010;26:1641–50.Google Scholar
  70. 70.
    Ward AF, Olsen AS, Wegner DM. The harm-made mind: observing victimization augments attribution of minds to vegetative patients, robots, and the dead. Psychol Sci. 2013;24:1437–45.PubMedCrossRefGoogle Scholar
  71. 71.
    Wang Z, Lee J, Marsella S. Towards more comprehensive listening behavior: beyond the bobble head. In: IVA, 2006, pp. 216–227.Google Scholar
  72. 72.
    Waytz A, Epley N, Cacioppo JT. Social cognition unbound: insights into anthropomorphism and dehumanization. Curr Dir Psychol Sci. 2010;19:58–62.PubMedCentralPubMedCrossRefGoogle Scholar
  73. 73.
    Waytz A, Epley N. Social connection enables dehumanization. J Exp Soc Psychol. 2012;48:70–6.CrossRefGoogle Scholar
  74. 74.
    Wilks Y. Is there progress on talking sensibly to machines? Science. 2007;318(9):927–8.PubMedCrossRefGoogle Scholar
  75. 75.
    Williams KD, Cheung CKZ, Choi. Cyberostracism: effects of being ignored over the Internet. J Pers Soc Psychol. 2000; 79/5: 748–762.Google Scholar
  76. 76.
    Williams KD, Govan CL, Croker V, Tynan D, Cruickshank M, Lam A. Investigations into the differences between social- and cyberostracism. Group Dyn Theor Res Pract. 2002;6:65–77.Google Scholar
  77. 77.
    Williams KD, Yaeger DS, Cheung CKT, Choi W. Cyberball (version 4.0) [Software], 2012. Available from
  78. 78.
    Yarkoni T. Personality in 100,000 words: a large-scale analysis of personality and word use among bloggers. J Res Pers. 2010;44(3):363–73.PubMedCentralPubMedCrossRefGoogle Scholar
  79. 79.
    Ziemke T, Lowe R. On the role of emotion in embodied cognitive architectures: from organisms to robots. Cogn Comput. 2009;1:104–17.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Marcin Skowron
    • 1
    Email author
  • Stefan Rank
    • 2
  • Aleksandra Świderska
    • 3
  • Dennis Küster
    • 3
  • Arvid Kappas
    • 3
  1. 1.Austrian Research Institute for Artificial IntelligenceViennaAustria
  2. 2.Westphal College of Media Arts and DesignDrexel UniversityPhiladelphiaUSA
  3. 3.Jacobs University BremenBremenGermany

Personalised recommendations