Emotion and Attitude Modeling for Non-player Characters

  • Brian RavenetEmail author
  • Florian Pecune
  • Mathieu Chollet
  • Catherine Pelachaud
Part of the Socio-Affective Computing book series (SAC, volume 4)


Within this chapter, we are presenting how game developers could take inspiration from the research in Embodied Conversational Agent to develop non-player characters capable of expressing believable emotional and social reactions. Inspired by the social theories about human emotional and social reactions, the researchers working with Embodied Conversational Agents developed different computational models to reproduce these human mechanisms within virtual characters. We are listing some of these works, comparing the different approaches and theories considered.


Facial Expression Basic Emotion Virtual Character Virtual Agent Virtual Human 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adam C, Herzig A, Longin D (2009) A logical formalization of the occ theory of emotions. Synthese 168(2):201–248CrossRefGoogle Scholar
  2. 2.
    Ahn J, Gobron S, Thalmann D, Boulic R (2013) Asymmetric facial expressions: revealing richer emotions for embodied conversational agents. Comput Animat Virtual Worlds 24(6):539–551CrossRefGoogle Scholar
  3. 3.
    Argyle M (1988) Bodily communication. University paperbacks, MethuenGoogle Scholar
  4. 4.
    Aylett R, Vala M, Sequeira P, Paiva A (2007) Fearnot!–an emergent narrative approach to virtual dramas for anti-bullying education. In: Virtual storytelling. Using virtual reality technologies for storytelling. Springer, Berlin, pp 202–205Google Scholar
  5. 5.
    Ballin D, Gillies M, Crabtree I (2004) A framework for interpersonal attitude and non-verbal communication in improvisational visual media production. In: Proceedings of the 1st European conference on visual media production, LondonGoogle Scholar
  6. 6.
    Bartneck C (2002) Integrating the occ model of emotions in embodied characters. In: Workshop on virtual conversational characters. CiteseerGoogle Scholar
  7. 7.
    Becker-Asano C (2014) Wasabi for affect simulation in human-computer interaction. In: Proceedings of international workshop on emotion representations and modelling for HCI systemsGoogle Scholar
  8. 8.
    Bee N, Franke S, Andrea E (2009) Relations between facial display, eye gaze and head tilt: dominance perception variations of virtual agents. In: 3rd international conference on affective computing and intelligent interaction and workshops, ACII 2009, pp 1–7Google Scholar
  9. 9.
    Bee N, Pollock C, Andrea E, Walker M (2010) Bossy or wimpy: expressing social dominance by combining gaze and linguistic behaviors. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents. Volume 6356 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 265–271Google Scholar
  10. 10.
    Bickmore T, Cassell J (2005) Social dialongue with embodied conversational agents. In: van Kuppevelt J, Dybkjær L, Bernsen NO (eds) Advances in natural multimodal dialogue systems. Springer, Dordrecht, pp 23–54CrossRefGoogle Scholar
  11. 11.
    Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput-Hum Interact 12(2):293–327CrossRefGoogle Scholar
  12. 12.
    Brave S, Nass C, Hutchinson K (2005) Computers that care: investigating the effects of orientation of emotion exhibited by an embodied computer agent. Int J Hum-Comput Stud 62(2):161–178CrossRefGoogle Scholar
  13. 13.
    Burgoon JK, Buller DB, Hale JL, de Turck MA (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10(3):351–378Google Scholar
  14. 14.
    Cafaro A, Vilhjálmsson H, Bickmore T, Heylen D, Jóhannsdóttir K, Valgar\(\partial\) sson G (2012) First impressions: users’ judgments of virtual agents’ personality and interpersonal attitude in first encounters. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents. Volume 7502 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 67–80Google Scholar
  15. 15.
    Callejas Z, Ravenet B, Ochs M, Pelachaud C (2014) A computational model of social attitudes for a virtual recruiter. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland County, pp 93–100Google Scholar
  16. 16.
    Campos H, Campos J, Cabral J, Martinho C, Nielsen JH, Paiva A (2013) My dream theatre. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland County, pp 1357–1358Google Scholar
  17. 17.
    Cassell J, Sullivan J, Prevost S, Churchill E (2000) Embodied Conversational Agents. MIT, CambridgeGoogle Scholar
  18. 18.
    Castelfranchi C, Miceli M, Cesta A (1992) Dependence relations among autonomous agents. Decentralized AI 3:215–227Google Scholar
  19. 19.
    Chindamo M, Allwood J, Ahlsén E (2012) Some suggestions for the study of stance in communication. In: Privacy, security, risk and trust (PASSAT), 2012 international confernece on social computing (SocialCom), pp 617–622. IEEE, Los AlamitosGoogle Scholar
  20. 20.
    Chollet M, Ochs M, Pelachaud C (2014) From non-verbal signals sequence mining to bayesian networks for interpersonal attitudes expression. In: Bickmore T, Marsella S, Sidner C (eds) Intelligent virtual agents. Volume 8637 of lecture notes in computer science. Springer, Cham, pp 120–133Google Scholar
  21. 21.
    Damasio A (2008) Descartes’ error: emotion, reason and the human brain. Random HouseGoogle Scholar
  22. 22.
    Darwin C (1872) The expression of the emotions in man and animals. Oxford University PressCrossRefGoogle Scholar
  23. 23.
    Dias J, Mascarenhas S, Paiva A (2014) Fatima modular: towards an agent architecture with a generic appraisal framework. In: Emotion modeling. Springer, Cham, pp 44–56Google Scholar
  24. 24.
    Ding Y, Prepin K, Huang J, Pelachaud C, Artières T (2014) Laughter animation synthesis. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, AAMAS ’14, Richland. International Foundation for Autonomous Agents and Multiagent Systems, pp 773–780Google Scholar
  25. 25.
    Ekman P, Friesen WV (1971) Constants across cultures in the face and emotion. J Personal Soc Psychol 17(2):124CrossRefGoogle Scholar
  26. 26.
    Ekman P, Friesen WV (1977) Facial action coding systemGoogle Scholar
  27. 27.
    Ennis C, Hoyet L, Egges A, McDonnell R (2013) Emotion capture: emotionally expressive characters for games. In: Proceedings of motion on games. ACM, pp 53–60Google Scholar
  28. 28.
    Enz S, Zoll C, Vannini N, Hall L, Paiva A, Aylett R, Lim MY (2012) Orient: the intercultural empathy. In: Cases on cultural implications and considerations in online learning, p 282Google Scholar
  29. 29.
    Fontaine JR, Scherer KR, Roesch EB, Ellsworth PC (2007) The World of emotions is not two-dimensional. Psychol Sci 18(12):1050–1057CrossRefPubMedGoogle Scholar
  30. 30.
    Fourati N, Pelachaud C (2014) Emilya: emotional body expression in daily actions database. In: LRECGoogle Scholar
  31. 31.
    Franklin S, Madl T, D’mello S, Snaider J (2014) Lida: a systems-level architecture for cognition, emotion, and learning. IEEE Trans Auton Ment Dev 6(1):19–41CrossRefGoogle Scholar
  32. 32.
    Fukayama A, Ohno T, Mukawa N, Sawaki M, Hagita N (2002) Messages embedded in gaze of interface agents – impression management with agent’s gaze. In: Proceedings of the 2002 SIGCHI conference on human factors in computing systems. ACM, New York, pp 41–48Google Scholar
  33. 33.
    Gebhard P (2005) Alma: a layered model of affect. In: Proceedings of the fourth international joint conference on autonomous agents and multiagent systems. ACM, New York, pp 29–36CrossRefGoogle Scholar
  34. 34.
    Gehm TL, Scherer KR (1988) Factors determining the dimensions of subjective emotional space.Google Scholar
  35. 35.
    Gratch J, Marsella S (2005) Evaluating a computational model of emotion. Auton Agents Multi-Agent Syst 11(1):23–43CrossRefGoogle Scholar
  36. 36.
    Hall ET (1969) The hidden dimension, vol 1990. Anchor Books, New YorkGoogle Scholar
  37. 37.
    Hexmoor H (2002) A model of absolute autonomy and power: toward group effects. Connect Sci 14(4):323–333CrossRefGoogle Scholar
  38. 38.
    Kasap Z, Ben Moussa M, Chaudhuri P, Magnenat-Thalmann N (2009) Making them remember – emotional virtual characters with memory. IEEE Comput Graph Appl 29(2):20–29Google Scholar
  39. 39.
    Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters, vol 7. CUP Archive, Cambridge/New YorkGoogle Scholar
  40. 40.
    Kleinginna PR Jr, Kleinginna AM (1981) A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv Emot 5(4):345–379CrossRefGoogle Scholar
  41. 41.
    Krämer K, Bente G, Kuzmanovic B, Barisic I, Pfeiffer UJ, Georgescu AL, Vogeley K (2014) Neural correlates of emotion perception depending on culture and gaze direction. Cult Brain 2(1):27–51CrossRefGoogle Scholar
  42. 42.
    Laird J (2012) The soar cognitive architecture. MIT, Cambridge/LondonGoogle Scholar
  43. 43.
    Lance B, Marsella SC (2007). Emotionally expressive head and body movement during gaze shifts. In: Intelligent virtual agents. Springer, Berlin/New York, pp 72–85CrossRefGoogle Scholar
  44. 44.
    Lee J, Marsella S (2011) Modeling side participants and bystanders: the importance of being a laugh track. In: Vilhjálmsson H, Kopp S, Marsella S, Thórisson K (eds) Intelligent virtual agents. Volume 6895 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 240–247Google Scholar
  45. 45.
    Luck M, Aylett R (2000) Applying artificial intelligence to virtual reality: intelligent virtual environments. Appl Artif Intell 14(1):3–32CrossRefGoogle Scholar
  46. 46.
    Marinier RP, Laird JE, Lewis RL (2009) A computational unification of cognitive behavior and emotion. Cogn Syst Res 10(1):48–69CrossRefGoogle Scholar
  47. 47.
    Marsella SC, Gratch J (2009) Ema: a process model of appraisal dynamics. Cogn Syst Res 10(1):70–90CrossRefGoogle Scholar
  48. 48.
    Marsella S, Gratch J, Petta P (2010) Computational models of emotion. In: Scherer KR, Banziger T, Roesch E (eds) A blueprint for affective computing–a sourcebook and manual. Oxford University Press, Oxford, pp 21–46Google Scholar
  49. 49.
    Marsella S, Xu Y, Lhommet M, Feng A, Scherer S, Shapiro A (2013) Virtual character performance from speech. In: Proceedings of the 12th ACM SIGGRAPH/Eurographics symposium on computer animation. ACM, New York, pp 25–35CrossRefGoogle Scholar
  50. 50.
    Mehrabian A (1969) Significance of posture and position in the communication of attitude and status relationships. Psychol Bull 71(5):359CrossRefPubMedGoogle Scholar
  51. 51.
    Mehrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol 14(4):261–292CrossRefGoogle Scholar
  52. 52.
    Meyer J-JC (2006) Reasoning about emotional agents. Int J Intell Syst 21(6):601–619CrossRefGoogle Scholar
  53. 53.
    Niewiadomski R, Hyniewska S, Pelachaud C (2009) Modeling emotional expressions as sequences of behaviors. In: Intelligent virtual agents. Springer, Berlin/New York, pp 316–322CrossRefGoogle Scholar
  54. 54.
    Niewiadomski R, Mancini M, Baur T, Varni G, Griffin H, Aung M (2013) MMLI: multimodal multiperson corpus of laughter in interaction. In: Salah A, Hung H, Aran O, Gunes H (eds) Human behavior understanding. Volume 8212 of lecture notes in computer science. Springer, Cham, pp 184–195Google Scholar
  55. 55.
    Ochs M, Sabouret N, Corruble V (2009) Simulation of the dynamics of nonplayer characters’ emotions and social relations in games. IEEE Trans Comput Intell AI Games 1(4):281–297CrossRefGoogle Scholar
  56. 56.
    Ochs M, Niewiadomski R, Pelachaud C (2010) How a virtual agent should smile? In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents. Volume 6356 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 427–440Google Scholar
  57. 57.
    Ochs M, Sadek D, Pelachaud C (2012) A formal model of emotions for an empathic rational dialog agent. Auton Agents Multi-Agent Syst 24(3):410–440CrossRefGoogle Scholar
  58. 58.
    Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University PressCrossRefGoogle Scholar
  59. 59.
    Osgood CE, Tannenbaum PH (1955) The principle of congruity in the prediction of attitude change. Psychol Rev 62(1):42CrossRefPubMedGoogle Scholar
  60. 60.
    Pelachaud C (2009) Modelling multimodal expression of emotion in a virtual agent. Philos Trans R Soc B: Biol Sci 364(1535):3539–3548CrossRefGoogle Scholar
  61. 61.
    Plutchik R (2001) The nature of emotions human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am Sci 89(4):344–350CrossRefGoogle Scholar
  62. 62.
    Popescu A, Broekens J, van Someren M (2014) Gamygdala: an emotion engine for games. IEEE Trans Affect Comput 5(1):32–44CrossRefGoogle Scholar
  63. 63.
    Prada R, Paiva A (2009) Teaming up humans with autonomous synthetic characters. Artif Intell 173(1):80–103CrossRefGoogle Scholar
  64. 64.
    Prendinger H, Descamps S, Ishizuka M (2002) Scripting affective communication with life-like characters in web-based interaction systems. Appl Artif Intell 16(7–8):519–553CrossRefGoogle Scholar
  65. 65.
    Reisenzein R (1994) Pleasure-arousal theory and the intensity of emotions. J Personal Soc Psychol 67(3):525CrossRefGoogle Scholar
  66. 66.
    Rist T, Schmitt M, Pelachaud C, Bilvi M (2003) Towards a simulation of conversations with expressive embodied speakers and listeners. In: 16th international conference on computer animation and social agents. IEEE, Los Alamitos, pp 5–10Google Scholar
  67. 67.
    Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39(6):1161CrossRefGoogle Scholar
  68. 68.
    Scherer K (2000) Emotion. In: Hewstone M, Stroebe W (eds) Introduction to social psychology: a European perspective.Google Scholar
  69. 69.
    Scherer KR (2001) Appraisal considered as a process of multilevel sequential checking. Apprais Processes Emot: Theory Methods Res 92:120Google Scholar
  70. 70.
    Scherer K (2005) What are emotions? And how can they be measured? Soc Sci InfGoogle Scholar
  71. 71.
    Scherer KR, Ellgring H (2007) Multimodal expression of emotion: affect programs or componential appraisal patterns? Emotion 7(1):158CrossRefPubMedGoogle Scholar
  72. 72.
    Schutz WC (1958) FIRO: a three-dimensional theory of interpersonal behavior. Rinehart, New YorkGoogle Scholar
  73. 73.
    Sundström P (2005) Exploring the affective loop. PhD thesis, Stockholm UniversityGoogle Scholar
  74. 74.
    Tsapatsoulis N, Raouzaiou A, Kollias S, Cowie R, Douglas-Cowie E (2002) Emotion recognition and synthesis based on MPEG-4 FAPs. In: Pandzic IS, Forchheimer R (eds) MPEG-4 facial animation. Wiley, Chichester/Hoboken, pp 141–167CrossRefGoogle Scholar
  75. 75.
    Wallbott HG (1998) Bodily expression of emotion. Eur J Soc Psychol 28(6):879–896CrossRefGoogle Scholar
  76. 76.
    Wiggins J (1996) The five-factor model of personality: theoretical perspectives. The Guilford Press, New YorkGoogle Scholar
  77. 77.
    Yannakakis G, Karpouzis K, Paiva A, Hudlicka E (2011) Emotion in games. In: Affective computing and intelligent interaction, Memphis, pp 497–497Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Brian Ravenet
    • 1
    Email author
  • Florian Pecune
    • 1
  • Mathieu Chollet
    • 1
  • Catherine Pelachaud
    • 1
  1. 1.LTCI, CNRS, Télécom ParisTechUniversité Paris-SaclayParisFrance

Personalised recommendations