Evaluating a Computational Model of Emotion

  • Jonathan Gratch
  • Stacy Marsella


Spurred by a range of potential applications, there has been a growing body of research in computational models of human emotion. To advance the development of these models, it is critical that we evaluate them against the phenomena they purport to model. In this paper, we present one method to evaluate an emotion model that compares the behavior of the model against human behavior using a standard clinical instrument for assessing human emotion and coping. We use this method to evaluate the Emotion and Adaptation (EMA) model of emotion Gratch and Marsella. The evaluation highlights strengths of the approach and identifies where the model needs further development.


emotion modeling lifelike characters 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. Gratch and S. Marsella, “Tears and fears: Modeling emotions and emotional behaviors in synthetic agents”, presented at Fifth International Conference on Autonomous Agents, Montreal, Canada, 2001.Google Scholar
  2. 2.
    S. Marsella and J. Gratch, “The Modeling coping behaviors in virtual humans: Don’t worry, be happy”, presented at Second International Joint Conference on Autonomous Agents and Multi-agent Systems, Melbourne, Australia, 2003.Google Scholar
  3. 3.
    Gratch, J., Marsella, S. 2004“A domain independent framework for modeling emotion”J. Cognitive Sys. Res5269306CrossRefGoogle Scholar
  4. 4.
    Conati, C. 2002“Probabilistic assessment of user’s emotions in educational games”J. Appl. Artif. Intelligence, special issue on “Merging Cognition and Affect in HCI”16555575Google Scholar
  5. 5.
    C. Conati and H. MacLaren, “Evaluating a probabilistic model of student affect,” presented at Seventh International Conference on Intelligent Tutoring Systems, Maceio, Brazil, 2004.Google Scholar
  6. 6.
    S. Marsella, W. L. Johnson, and C. LaBore, “Interactive pedagogical drama,” presented at Fourth International Conference on Autonomous Agents, Montreal, Canada, 2000.Google Scholar
  7. 7.
    Rothbaum, B. O., Hodges, L. F., Alarcon, R., Ready, D., Shahar, F., Graap, K., Pair, J., Hebert, P., Wills, B., Baltzell, D. 1999“Virtual environment exposure therapy for PTSD vietnam veterans: A case study”J. Trauma Stress12263272CrossRefPubMedGoogle Scholar
  8. 8.
    Lester, J. C., Stone, B. A., Stelling, G. D. 1999“Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments”User Model User-Adapted Instruction.9144CrossRefGoogle Scholar
  9. 9.
    Ryokai, K., Vaucelle, C., Cassell, J. 2003“Virtual Peers as Partners in Storytelling and Literacy Learning”Journal of Computer Assisted Learning19195208CrossRefGoogle Scholar
  10. 10.
    Shaw, E., Johnson, W. L., Ganeshan, R. 1999“Pedagogical agents on the web”, presented at Proceedings of the Third International Conference on Autonomous AgentsSeattleWAGoogle Scholar
  11. 11.
    André, E., Rist, T., Mulken, S. V., Klesen, M. 2000“The automated design of believable dialogues for animated presentation teams”Cassell, J.Sullivan, J.Prevost, S.Churchill, E. eds. Embodied Conversational AgentsMIT PressCambridge, MA220255Google Scholar
  12. 12.
    Cassell, J., Bickmore, T., Campbell, L., Vilhjálmsson, H., Yan, H. 2000“Human conversation as a system framework: Designing embodied conversational agents”Cassell, J.Sullivan, J.Prevost, S.Churchill, E. eds. Embodied Conversational AgentsMIT PressBoston2963Google Scholar
  13. 13.
    B. G. Silverman, M. Johns, K. O’Brien, R. Weaver, and J. Cornwell, “Constructing Virtual Asymmetric Opponents from Data and Models in the Literature: Case of Crowd Rioting,” 11th Conference on Computer-Generated Forces and Behavior Representation, Orlanda FL, pp. 97–105, 2002.Google Scholar
  14. 14.
    Gratch, J., Marsella, S. 2003“Fight the way you train: The role and limits of emotions in training for combat”Brown J. World Affairs106376Google Scholar
  15. 15.
    S. Marsella, D. Pynadath, and S. Read, “PsychSim: Agent-based modeling of social interactions and influence,” presented at International Conference on Cognitive Modeling, 2004.Google Scholar
  16. 16.
    Minsky, M. 1986The Society of MindSimon and SchusterNew YorkGoogle Scholar
  17. 17.
    Simon, H. A. 1967“Motivational and emotional controls of cognition”Psychological Rev742939Google Scholar
  18. 18.
    Damasio, A. R. 1994Descartes’ Error: Emotion, Reason, and the Human BrainAvon BooksNew YorkGoogle Scholar
  19. 19.
    Oatley, K., Johnson-Laird, P. N. 1987“Towards a Cognitive Theory of Emotions”Cognition and Emotion12950Google Scholar
  20. 20.
    Sloman, A., Croucher, M. 1981“Why robots will have emotions”, presented at International Joint Conference on Artificial IntelligenceCanadaVancouverGoogle Scholar
  21. 21.
    Lisetti, C., Gmytrasiewicz, P. 2002“Can a rational agent afford to be affectless? A formal approach”Appl. Artif. Intelligence16577609CrossRefGoogle Scholar
  22. 22.
    Frank, R. 1988Passions with Reason: The Strategic Role of the EmotionsW. W. NortonNew York, NYGoogle Scholar
  23. 23.
    Mele, A. R. 2001Self-Deception UnmaskedPrinceton University PressPrinceton, NJGoogle Scholar
  24. 24.
    W. Mao and J. Gratch, “Social judgment in multiagent interactions”, presented at Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004.Google Scholar
  25. 25.
    J. Gratch, J. Rickel, E. André, J. Cassell, E. Petajan, and N. Badler, “Creating interactive virtual humans: Some assembly required”, in IEEE Intelligent Syst., vol. July/August, pp. 54–61, 2002.Google Scholar
  26. 26.
    J. Rickel, S. Marsella, J. Gratch, R. Hill, D. Traum, and W. Swartout, “Toward a new generation of virtual humans for interactive experiences”, in IEEE Intelligent Syst., vol. July/August, pp. 32–38 2002.Google Scholar
  27. 27.
    S. Marsella, W. L. Johnson, and C. LaBore, “Interactive pedagogical drama for health interventions”, presented at Conference on Artificial Intelligence in Education, Sydney, Australia, 2003.Google Scholar
  28. 28.
    M. Cavazza, F. Charles, and S. J. Mead, “Agents’ interaction in virtual storytelling”, presented at Third International Workshop on Intelligent Virtual Agents, 2001.Google Scholar
  29. 29.
    Lisetti, C. L., Schiano, D. 2000“Facial expression recognition: Where human-computer interaction, artificial intelligence, and cognitive science intersect”Pragma Cognition8185235Google Scholar
  30. 30.
    Picard, R. W. 1997Affective ComputingMIT PressCambridge, MAGoogle Scholar
  31. 31.
    S. van Mulken, E. André, and J. Muller, “The persona effect: how substantial is it,” presented at Human Computer Interaction Conference, Berlin, 1998.Google Scholar
  32. 32.
    W. Swartout, R. Hill, J. Gratch, W. L. Johnson, C. Kyriakakis, C. LaBore, R. Lindheim, S. Marsella, D. Miraglia, B. Moore, J. Morie, J. Rickel, M. Thiebaux, L. Tuch, R. Whitney, and J. Douglas, “Toward the holodeck: Integrating graphics, sound, character and story,” presented at Fifth International Conference on Autonomous Agents, Montreal, Canada, 2001.Google Scholar
  33. 33.
    Lazarus, R. 1991Emotion and AdaptationOxford University PressNYGoogle Scholar
  34. 34.
    J. Velásquez, “When robots weep: Emotional memories and decision-making.,” presented at 15th National Conference on Artificial Intelligence, Madison, WI, 1998.Google Scholar
  35. 35.
    Smith, C. A., Lazarus, R. 1990“Emotion and Adaptation”Pervin, L. A. eds. Handbook of Personality: Theory and researchGuilford PressNY609637Google Scholar
  36. 36.
    Peacock, E., Wong, P. 1990“The stress appraisal measure (SAM): A multidimensional approach to cognitive appraisal”Stress Medic6227236Google Scholar
  37. 37.
    Blythe, J. 1999“Decision Theoretic Planning”in AI Magazine203754Google Scholar
  38. 38.
    Grosz, B., Kraus, S. 1996“Collaborative plans for complex group action”Artif. Intelligence86269357CrossRefGoogle Scholar
  39. 39.
    Shaver, K. G. 1985The Attribution of Blame: Causality, Responsibility, and BlameworthinessSpringer-VerlagNYGoogle Scholar
  40. 40.
    Ortony, A., Clore, G., Collins, A. 1988The Cognitive Structure of EmotionsCambridge University PressCambridgeGoogle Scholar
  41. 41.
    Gratch, J., Marsella, S. 2005“Lessons from Emotion Psychology for the Design of Lifelike Characters”Applied Artif Intelligence19215233Google Scholar
  42. 42.
    C. Pelachaud, V. Carofiglio, B. D. Carolis, F. d. Rosis, and I. Poggi, “First international joint conference on autonomous agents and multiagent systems”, presented at Embodied Contextual Agent in Information Delivering Application, Bologna, Italy, 2002.Google Scholar
  43. 43.
    Klesen, M. 2005“Using theatrical concepts for role-plays with educational agents”Appl. Artif. Intelligence Special Issue “Educational Agents - Beyond Virtual Tutors”19413431Google Scholar
  44. 44.
    Nakanishi, H., Shimizu, S., Isbister, K. 2005“Sensitizing social agents for virtual training”Appl. Artif. Intelligence Special Issue “Educational Agents - Beyond Virtual Tutors”19393412Google Scholar
  45. 45.
    A. Cowell and K. M. Stanney, “Embodiement and interaction guidelines for designing credible, trustworthy embodied conversational agents”, presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.Google Scholar
  46. 46.
    Lester, J. C., Towns, S. G., Callaway, C. B., Voerman, J. L., FitzGerald, P. J. 2000“Deictic and emotive communication in animated pedagogical agents”Cassell, J.Prevost, S.Sullivan, J.Churchill, E. eds. Embodied Conversational AgentsMIT PressCambridge123154Google Scholar
  47. 47.
    G. Biswas, D. Schwartz, K. Leelawong, N. Vye, and TAG-V, “Learning by teaching. A new agent paradigm for educational software,” Appl. Artif. Intelligence special Issue “Educ. Agents - Beyond Virtual Tutors”, vol. 19, 2005.Google Scholar
  48. 48.
    C. Elliott, “The affective reasoner: A process model of emotions in a multi-agent system,” Ph.D Dissertation, Northwestern University Institute for the Learning Sciences, Northwestern, IL, vol. 32, 1992.Google Scholar
  49. 49.
    ElNasr, M. S., Yen, J., Ioerger, T. 2000“FLAME: Fuzzy Logic Adaptive Model of Emotions”Autonomous Agents and Multi-Agent Systems3219257CrossRefGoogle Scholar
  50. 50.
    D. Moffat and N. Frijda, “Where there’s a Will there’s an agent,” presented at Workshop on Agent Theories, Architectures and Languages, 1995.Google Scholar
  51. 51.
    Armony, J. L., Servan-Schreiber, D., Cohen, J. D., LeDoux, J. E. 1997“Computational modeling of emotion: Explorations through the anatomy and physiology of fear conditioning”Trends in Cognitive Sci12834CrossRefGoogle Scholar
  52. 52.
    D. Cañamero, “Modeling motivations and emotions as a basis for intelligent behavior”, presented at International Conference on Autonomous Agents, Marina del Rey, CA, 1997.Google Scholar
  53. 53.
    Arkin, R. C. 2005“Moving up the food chain: Motivation and Emotion in behavior-based robots”Fellous, J.Arbib, M. eds. Who Needs Emotions: The Brain Meets the RobotOxford University PressNew YorkGoogle Scholar
  54. 54.
    Busemeyer, J. R., Townsend, J. T., Stout, J. C. 2002“Motivational underpinnings of utility in decision making: Decision field theory analysis of deprivation and satiation”Moore, S. eds. Emotional CognitionJohn BenjaminsAmsterdamGoogle Scholar
  55. 55.
    P. Gmytrasiewicz and C. Lisetti, “Using decision theory to formalize emotions for multi-agent systems,” presented at Second ICMAS-2000 Workshop on Game Theoretic and Decision Theoretic Agents, Boston, 2000.Google Scholar
  56. 56.
    H. Prendinger, S. Mayer, J. Mori, and M. Ishizuka, “Persona effect revisited,” presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.Google Scholar
  57. 57.
    N. C. Kramer, B. Tietz, and G. Bente, “Effects of embodied interface agents and their gestural activity,” presented at Intelligent Virtual Agents, Kloster Irsee, Germany, 2003.Google Scholar
  58. 58.
    M. Scheutz and A. Sloman, “Affect and agent control: experiments with simple affective states,” presented at IAT, 2001.Google Scholar
  59. 59.
    Perrez, M., Reicherts, M. 1992Stress, Coping, and HealthHogrefe and Huber PublishersSeattle, WAGoogle Scholar

Copyright information

© Springer Science+Business Media Inc. 2005

Authors and Affiliations

  1. 1.Institute for Creative TechnologyUniversity of Southern CaliforniaMarina del ReyUSA
  2. 2.Information Sciences InstituteUniversity of Southern CaliforniaMarina del ReyUSA

Personalised recommendations