ERiSA: Building Emotionally Realistic Social Game-Agents Companions

  • Andry Chowanda
  • Peter Blanchfield
  • Martin Flintham
  • Michel Valstar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8637)


We propose an integrated framework for social and emotional game-agents to enhance their believability and quality of interaction, in particular by allowing an agent to forge social relations and make appropriate use of social signals. The framework is modular including sensing, interpretation, behaviour generation, and game components. We propose a generic formulation of action selection rules based on observed social and emotional signals, the agent’s personality, and the social relation between agent and player. The rules are formulated such that its variables can easily be obtained from real data. We illustrate and evaluate our framework using a simple social game called The Smile Game.


Social Relationship Framework Game-Agents Interactions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Almaev, T.R., Valstar, M.F.: Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In: Affective Computing and Intelligent Interaction (ACII 2013), pp. 356–361 (September 2013)Google Scholar
  2. 2.
    Bates, J.: The role of emotion in believable agents. Commun. ACM 37(7), 122–125 (1994)CrossRefGoogle Scholar
  3. 3.
    Bickmore, T., Vardoulakis, L., Jack, B., Paasche-Orlow, M.: Automated promotion of technology acceptance by clinicians using relational agents. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS, vol. 8108, pp. 68–78. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  4. 4.
    Bogdanovych, A., Ijaz, K., Simoff, S.: The city of uruk: Teaching ancient history in a virtual world. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 28–35. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Campbell, J., Core, M., Artstein, R., et al.: Developing inots to support interpersonal skills practice. In: Aerospace Conference, pp. 1–14 (March 2011)Google Scholar
  6. 6.
    Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS): Manual. A Human Face, Salt Lake City, USA (2002)Google Scholar
  7. 7.
    Hartholt, A., Traum, D., Marsella, S.C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.-P., Gratch, J.: All together now: Introducing the virtual human toolkit. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS, vol. 8108, pp. 368–381. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Hilgard, E.R.: Introduction to psychology. Harcourt, Brace (1953)Google Scholar
  9. 9.
    Kopp, S., Krenn, B., Marsella, S.C., Marshall, A.N., Pelachaud, C., Pirker, H., Thórisson, K.R., Vilhjálmsson, H.H.: Towards a common framework for multimodal generation: The behavior markup language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 205–217. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    McKeown, G., Valstar, M.F., Cowie, R., Pantic, M., Schröder, M.: The semaine database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent. T. Affective Computing 3(1), 5–17 (2012)Google Scholar
  11. 11.
    Ochs, M., Sabouret, N., Corruble, V.: Simulation of the dynamics of nonplayer characters’ emotions and social relations in games. IEEE Transactions on Computational Intelligence and AI in Games 1(4), 281–297 (2009)CrossRefGoogle Scholar
  12. 12.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  13. 13.
    Saucier, G., Goldberg, L.R.: The language of personality: Lexical perspectives on the five-factor model. In: Wiggins, J.S. (ed.) The Five-Factor Model of Personality: Theoretical Perspectives, pp. 21–50. Guilford, New York (1996)Google Scholar
  14. 14.
    Schröder, M.: The SEMAINE API: Towards a standards-based framework for building emotion-oriented systems. In: Advances in Human-Computer Interaction, Article ID 319406 (2010)Google Scholar
  15. 15.
    Sollenberger, D.J., Singh, M.P.: Koko: An architecture for affect-aware games. Autonomous Agents and Multi-Agent Systems 24(2), 255–286 (2012)CrossRefGoogle Scholar
  16. 16.
    Turing, A.M.: Computing machinery and intelligence. Mind 59(236), 433–460 (1950)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Urbain, J., Niewiadomski, R., Mancini, M., Griffin, H., Çakmak, H., Ach, L., Volpe, G.: Multimodal analysis of laughter for an interactive system. In: Mancas, M., d’ Alessandro, N., Siebert, X., Gosselin, B., Valderrama, C., Dutoit, T. (eds.) INTETAIN 2013. LNICST, vol. 124, pp. 183–192. Springer, Heidelberg (2013)Google Scholar
  18. 18.
    Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: Survey of an emerging domain. Image Vision Comput. 27(12), 1743–1759 (2009)CrossRefGoogle Scholar
  19. 19.
    Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (ssi) framework: multimodal signal processing and recognition in real-time. In: ACM Multimedia, pp. 831–834. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Andry Chowanda
    • 1
    • 2
  • Peter Blanchfield
    • 1
  • Martin Flintham
    • 1
  • Michel Valstar
    • 1
  1. 1.School of Computer ScienceThe University of NottinghamNottinghamUK-GB
  2. 2.School of Computer ScienceBina Nusantara UniversityJakartaUK

Personalised recommendations