Autonomous Agents and Multi-Agent Systems

, Volume 28, Issue 1, pp 126–153 | Cite as

An emotion understanding framework for intelligent agents based on episodic and semantic memories

  • Mohammad Kazemifard
  • Nasser Ghasem-Aghaee
  • Bryan L. Koenig
  • Tuncer I. Ören


Emotional intelligence is the ability to process information about one’s own emotions and the emotions of others. It involves perceiving emotions, understanding emotions, managing emotions and using emotions in thought processes and in other activities. Emotion understanding is the cognitive activity of using emotions to infer why an agent is in an emotional state and which actions are associated with the emotional state. For humans, knowledge about emotions includes, in part, emotional experiences (episodic memory) and abstract knowledge about emotions (semantic memory). In accordance with the need for more sophisticated agents, the current research aims to increase the emotional intelligence of software agents by introducing and evaluating an emotion understanding framework for intelligent agents. The framework organizes the knowledge about emotions using episodic memory and semantic memory. Its episodic memory learns by storing specific details of emotional events experienced firsthand or observed. Its semantic memory is a lookup table of emotion-related facts combined with semantic graphs that learn through abstraction of additional relationships among emotions and actions from episodic memory. The framework is simulated in a multi-agent system in which agents attempt to elicit target emotions in other agents. They learn what events elicit emotions in other agents through interaction and observation. To evaluate the importance of different memory components, we run simulations with components “lesioned”. We show that our framework outperformed Q-learning, a standard method for machine learning.


Artificial intelligence Emotion understanding ability Episodic memory Semantic memory Intelligent agents Multi-agent simulation 


  1. 1.
    Ghasem-Aghaee, N., & Ören, T. I. (2003). Towards fuzzy agents with dynamic personality for human behavior simulation. 2003 Summer Computer Simulation Conference (pp. 3–10), Montreal, PQ, Canada.Google Scholar
  2. 2.
    Ören, T. I. (2000). Understanding: A taxonomy and performance factors. In D. Thiel (Ed.), FOODSIM’2000 (pp. 3–10). Nantes, France. San Diego, CA: SCS, Invited Opening Paper.Google Scholar
  3. 3.
    Ören, T. I., Ghasem-Aghaee, N., & Yilmaz, L. (2007). An ontology-based dictionary of understanding as a basis for software agents with understanding abilities. Spring Simulation Multiconference (SpringSim’07) (pp. 19–27), Norfolk, VA.Google Scholar
  4. 4.
    Kazemifard, M., Ghasem-Aghaee, N., & Ören, T. I. (2009). Agents with ability to understand emotions. Summer Computer Simulation Conference, Istanbul, Turkey (Vol. 41, pp. 254–260, Vol. Simulation Series). San Diego, CA: SCS.Google Scholar
  5. 5.
    Wooldridge, M. (2001). An introduction to multiagent systems. New York: Wiley.Google Scholar
  6. 6.
    Kazemifard, M., Zaeri, A., Ghasem-Aghaee, N., NematBakhsh, M. A., & Mardukhi, F. (2011). Fuzzy Emotional COCOMO II Software Cost Estimation (FECSCE) using multi-agent systems. Applied Soft Computing, 11(2), 2260–2270.CrossRefGoogle Scholar
  7. 7.
    Ammar, M. B., Neji, M., Alimi, A. M., & Gouardères, G. (2009). The affective tutoring system. Expert Systems with Applications. doi:10.1016/j.eswa.2009.09.031.
  8. 8.
    Toptsis, A. A. (2010). Toward an emotion machine-based hybrid adaptive tutoring system. 3rd International Conference on Information Sciences and Interaction Sciences (ICIS) (pp. 133–137), Chengdu, China.Google Scholar
  9. 9.
    Woolf, B. P., Arroyo, I., Cooper, D., Burleson, W., & Muldner, K. (2010). Affective tutors: Automatic detection of and response to student emotion. In R. Nkambou, J. Bourdeau, & R. Mizoguchi (Eds.), Advances in intelligent tutoring systems. Studies in computational intelligence (pp. 207–227). Berlin: Springer.Google Scholar
  10. 10.
    Fatahi, S., Kazemifard, M., & Ghasem-Aghaee, N. (2009). Design and implementation of an E-learning model by considering learner’s personality and emotions. In S. I. Ao & L. Gelman (Eds.), Advances in electrical engineering and computational science (Vol. 39, pp. 423–434). Berlin: Springer.CrossRefGoogle Scholar
  11. 11.
    Shaikh, M. A. M., Prendinger, H., & Ishizuka, M. (2009). A linguistic interpretation of the OCC emotion model for affect sensing from text. In J. Tao & T. Tan (Eds.), Affective information processing (pp. 45– 73). Berlin: Springer.Google Scholar
  12. 12.
    Shen, Z., Miao, C., Zhang, L., Yu, H., & Chavez, M. J. (2010). An emotion aware agent platform for interactive storytelling and gaming. ACM Future Play Conference, Vancouver, Canada.Google Scholar
  13. 13.
    Saulnier, P., Sharlin, E., & Greenberg, S. (2009). Using bio-electrical signals to influence the social behaviours of domesticated robots. 4th ACM/IEEE international conference on Human robot interaction, La Jolla, California, USA (pp. 263–264). New York: ACM.Google Scholar
  14. 14.
    Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence. In P. Salovey & D. Sluyter (Eds.), Emotional development and emotional intelligence: Educational implications (pp. 3–31). New York: Basic Books.Google Scholar
  15. 15.
    Mayer, J. D., Salovey, P., & Caruso, D. R. (2008). Emotional intelligence: New ability or eclectic traits? American Psychologist, 63(6), 513–517.CrossRefGoogle Scholar
  16. 16.
    Dias, J., & Paiva, A. (2009). Agents with emotional intelligence for storytelling. Paper presented at the 8th International Conference on Autonomouse Agents and MultiAgent Systems, Doctoral Mentoring Program Budapest, Hungary.Google Scholar
  17. 17.
    Dias, J. o., & Paiva, A. (2011). Agents with emotional intelligence for storytelling. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction. Lecture Notes in Computer Science (Vol. 6974, pp. 77–86). Berlin: Springer.Google Scholar
  18. 18.
    Kazemifard, M., Ghasem-Aghaee, N., & Ören, T. I. (2012). Emotive and cognitive simulations by agents: Roles of three levels of information processing. Cognitive Systems Research, 13(1), 24–38.CrossRefGoogle Scholar
  19. 19.
    Tulving, E. (1983). Elements of episodic memory. New York: Oxford University Press.Google Scholar
  20. 20.
    Brom, C., Lukavský, J., & Kadlec, R. (2010). Episodic memory for human-like agents and human-like agents for episodic memory. International Journal of Machine Consciousness (IJMC), 2(2), 227–244.CrossRefGoogle Scholar
  21. 21.
    Magnenat-Thalmann, N., & Kasap, Z. Virtual humans in serious games. International Conference on CyberWorlds, Bradford, West Yorkshire, UK, 2009 (pp. 71–79), IEEE Computer Society.Google Scholar
  22. 22.
    Ho, W. C., & Dautenhahn, K. (2008). Towards a narrative mind: The creation of coherent life stories for believable virtual agents. In H. Prendinger, J. Lester, & M. Ishizuka (Eds.), Intelligent virtual agents. Lecture Notes in Computer Science (Vol. 5208, pp. 59–72). Berlin: Springer.Google Scholar
  23. 23.
    Kasap, Z., & Magnenat-Thalmann, N. (2010). Towards episodic memory-based long-term affective interaction with a human-like robot. 19th IEEE International Symposium on Robot and Human Interactive Communication, Principe di Piemonte-Viareggio, Italy (pp. 479–484).Google Scholar
  24. 24.
    Brom, C., Pešková, K., & Lukavský, J. (2007). What does your actor remember? Towards characters with a full episodic memory. In M. Cavazza, & S. Donikian (Eds.), Virtual storytelling. Using virtual reality technologies for storytelling. Lecture Notes in Computer Science (Vol. 4871, pp. 89–101). Berlin: Springer.Google Scholar
  25. 25.
    Faghihi, U., Fournier-Viger, P., Nkambou, R., & Poirier, P. (2009). A generic episodic learning model implemented in a cognitive agent by means of temporal pattern mining. In B. C. Chien, T. P. Hong & S. M. Chen (Eds.), Next-generation applied intelligence (Vol. 5579, pp. 545–555). Berlin: Springer.Google Scholar
  26. 26.
    Deutsch, T., Gruber, A., Lang, R., & Velik, R. (2008). Episodic memory for autonomous agents. Human System Interactions (HSI 2008), Krakau (pp. 621–626).Google Scholar
  27. 27.
    Nuxoll, A. (2007). Enhancing intelligent agents with episodic memory. Computer Science and Engineering: The University of Michigan.Google Scholar
  28. 28.
    Nuxoll, A., & Laird, J. E. (2007). Extending cognitive architecture with episodic memory. 22nd National Conference on Artificial Intelligence, Vancouver, British Columbia, Canada (Vol. 2, pp. 1560–1565). AAAI Press.Google Scholar
  29. 29.
    Nuxoll, A., & Laird, J. E. (2004). A cognitive model of episodic memory integrated with a general cognitive architecture. International Conference on Cognitive Modeling, ICCM 2004, Pittsburgh, Pennsylvania, USA (pp. 220–225).Google Scholar
  30. 30.
    Gorski, N. A., & Laird, J. E. (2011). Learning to use episodic memory. Cognitive Systems Research, 12(2), 144–153. doi:10.1016/j.cogsys.2010.08.001.CrossRefGoogle Scholar
  31. 31.
    Sieber, G., & Krenn, B. (2010). Episodic memory for companion dialogue. 2010 Workshop on Companionable Dialogue Systems, ACL 2010, Uppsala, Sweden (pp. 1–6).Google Scholar
  32. 32.
    Wang, W., Subagdja, B., & Tan, A.-H. (2010). A self-organizing approach to episodic memory modeling. WCCI 2010 IEEE World Congress on Computational Intelligence, CCIB, Barcelona, Spain (pp. 447–454).Google Scholar
  33. 33.
    Castro, E. C. D., & Gudwin, R. R. (2010). An episodic memory implementation for a virtual creature. In L. Magnani, W. Carnielli, & C. Pizzi (Eds.), Model-based reasoning in science and technology. Studies in Computational Intelligence (pp. 393–406). Berlin: Springer.Google Scholar
  34. 34.
    Morrison, C. M., & Conway, M. A. (2010). First words and first memories. Cognition, 116(1), 23–32.CrossRefGoogle Scholar
  35. 35.
    Sato, N., & Yamaguchi, Y. (2010). Simulation of human episodic memory by using a computational model of the hippocampus. Advances in Artificial Intelligence. doi:10.1155/2010/392868
  36. 36.
    Talarico, J. M., LaBar, K. S., & Rubin, D. C. (2004). Emotional intensity predicts autobiographical memory experience. Memory and Cognition, 32(7), 1118–1132.CrossRefGoogle Scholar
  37. 37.
    Marinier, R. P., Laird, J. E., & Lewis, R. L. (2009). A computational unification of cognitive behavior and emotion. Cognitive Systems Research, 10(1), 48–69.CrossRefGoogle Scholar
  38. 38.
    Schacter, D. L. (1996). Searching for memory: The brain, the mind, and the past. New York: Basic Books.Google Scholar
  39. 39.
    Wang, Y., & Laird, J. E. (2006). Integrating semantic memory into a cognitive architecture. Ann Arbor, MI: University of Michigan Center for Cognitive Architecture.Google Scholar
  40. 40.
    Szymański, J., Sarnatowicz, T., & Duch, W. (2005). Semantic memory for avatars in cyberspace. International Conference on Cyberworlds (CW’05), Singapore (pp. 165–171).Google Scholar
  41. 41.
    Veale, R., & Liston, R. (2008). Semantic memory in a computer: A synthetic model proposal. Paper presented at the 23rd annual spring conference of the Pennsylvania Computer and Information Science Educators (PACISE), Kutztown, PA.Google Scholar
  42. 42.
    Van Kleef, G. A., De Dreu, C. K. W., & Manstead, A. S. R. (2010). An interpersonal approach to emotion in social decision making: The emotions as social information model. Advances in Experimental Social Psychology, 42, 45–96.Google Scholar
  43. 43.
    Harelia, S., & Hessb, U. (2010). What emotional reactions can tell us about the nature of others: An appraisal perspective on person perception. Cognition & Emotion, 24(1), 128–140.CrossRefGoogle Scholar
  44. 44.
    de Melo, C. M., Gratch, J., & Carnevale, P. (2011). Reverse appraisal: Inferring from emotion displays who is the cooperator and the competitor in a social dilemma. Paper presented at the 33rd Annual Meeting of the Cognitive Science Society (CogSci).Google Scholar
  45. 45.
    Ekman, P. (2004). Emotion revealed: Recognizing faces and feelings to improve communication and emotional life. New York: Henry Holt and Company.Google Scholar
  46. 46.
    Tao, J., & Tan, T. (2005). Affective computing: A review. In J. Tao, T. Tan, & R. W. Picard (Eds.), Afective computing and intelligent interaction (Vol. 3784, pp. 981–995). Berlin: Springer.CrossRefGoogle Scholar
  47. 47.
    Sloman, A., Chrisley, R., & Scheutz, M. (2005). The architectural basis of affective states and processes. In J. Fellous & M. Arbib (Eds.), Who needs emotions: The brain meets the machine. New York: Oxford University Press.Google Scholar
  48. 48.
    Gratch, J., & Marsella, S. (2004). A domain-independent framework for modeling emotion. Cognitive Systems Research, 5(4), 269–306.CrossRefGoogle Scholar
  49. 49.
    Dias, J. (2005). Fearnot!: Creating emotional autonomous synthetic characters for empathic interactions. Master’s thesis, Universidade Técnica de Lisboa, Instituto Superior Técnica, Lisboa.Google Scholar
  50. 50.
    Dias, J., & Paiva, A. (2005). Feeling and reasoning: A computational model for emotional characters. In C. Bento, A. Cardoso & G. Dias (Eds.), Progress in artificial intelligence (pp. 127–140). Berlin: Springer.Google Scholar
  51. 51.
    Biermann, A. W., & Ramm, D. (2002). Great ideas in computer science with Java. Cambridge: MIT Press.Google Scholar
  52. 52.
    Solms, M., & Turnbull, O. (2003). The brain and the inner world: An introduction to the neuroscience of subjective experience. New York: Other Press.Google Scholar
  53. 53.
    Anderson, J. R. (2000). Cognitive psychology and its implications (5th ed.). New York: Worth Publishers.Google Scholar
  54. 54.
    Allen, P. A., Kauta, K. P., & Lord, R. R. (2008). Emotion and episodic memory. In E. Dere, A. Easton, L. Nadel, & J. P. Huston (Eds.), Handbook of behavioral neuroscience: Episodic memory research (Vol. 18, pp. 115–132). Oxford: Elsevier.CrossRefGoogle Scholar
  55. 55.
    LaBar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), 54–64.CrossRefGoogle Scholar
  56. 56.
    Hatfield, E., Cacioppo, J. T., & Rapson, L. (1993). Emotional contagion. Current Directions in Psychological Science, 2(3), 96–100.CrossRefGoogle Scholar
  57. 57.
    Ortony, A. (1975). How episodic is semantic memory?. In Theoretical issues in natural language processing: An interdisciplinary workshop in computational linguistics, psychology, linguistics, and artificial intelligence, Cambridge, MA.Google Scholar
  58. 58.
    Tulving, E. (1993). What is episodic memory. Current Directions in Psychological Science, 2(3), 67–70.CrossRefGoogle Scholar
  59. 59.
    Baddeley, A. D. (1976). The psychology of memory. New York: Basic Books.Google Scholar
  60. 60.
    Kompus, K., Olsson, C.-J., Larsson, A., & Nyberg, L. (2009). Dynamic switching between semantic and episodic memory systems. Neuropsychologia, 47(11), 2252–2260.CrossRefGoogle Scholar
  61. 61.
    Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  62. 62.
    Kazemifard, M., Ghasem-Aghaee, N., & Ören, T. I. (2011). Design and implementation of GEmA: A generic emotional agent. Expert systems with applications, 38(3), 2640–2652.CrossRefGoogle Scholar
  63. 63.
    Ortony, A., Norman, D., & Revelle, W. (2005). Affect and proto-affect in effective functioning. In J. Fellous & M. Arbib (Eds.), Who needs emotions: The brain meets the machine (pp. 173–202). New York: Oxford University Press.CrossRefGoogle Scholar
  64. 64.
    Becker-Asano, C., & Wachsmuth, I. (2008). Affect simulation with primary and secondary emotions. The 8th international conference on Intelligent Virtual, Tokyo, Springer.Google Scholar
  65. 65.
    Darwin, C. (1872/1998). The expression of the emotions in man and animals (3rd ed.). Oxford: Oxford University Press.Google Scholar
  66. 66.
    Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. New York: Harper & Row.Google Scholar
  67. 67.
    Shortliffe, E. H., & Buchanan, B. G. (1975). A model of inexact reasoning in medicine. Mathematical Biosciences, 23(3–4), 351–379. doi:10.1016/0025-5564(75)90047-4.CrossRefMathSciNetGoogle Scholar
  68. 68.
    Durkin, J. (1994). Expert systems: Design and developement. Englewood Cliffs: Prentice Hall.Google Scholar
  69. 69.
    Ball, G., & Hall, D. (1967). A clustering technique for summarizing multivariate data. Behavioral Science, 12(2), 153–155.CrossRefGoogle Scholar
  70. 70.
    Xu, R., & Wunsch, D. C, I. I. (2005). Survey of clustering algorithms. IEEE Transactions on Neural Networks, 16(5), 645–678.CrossRefGoogle Scholar
  71. 71.
    Rijsbergen, C. J. V. (1979). Information retrieval (2nd ed.). London: Butterworths.Google Scholar
  72. 72.
    Barrett, L. F., Mesquita, B., Ochsner, K. N., & Gross, J. J. (2007). The experience of emotion. Annual Review Psychology, 58, 373–403.CrossRefGoogle Scholar
  73. 73.
    Fatahi, S., Ghasem-Aghaee, N., & Kazemifard, M. (2008). Design an expert system for Virtual Classmate Agent (VCA). International Conference of Computational Intelligence and Intelligent Systems (ICCIIS’08), London, UK (pp. 102–106).Google Scholar
  74. 74.
    Ören, T. I., & Yilmaz, L. (2011). Semantic agents with understanding abilities and factors affecting misunderstanding. In A. Elci, M. T. Traore, & M. A. Orgun (Eds.), Semantic agent systems: Foundations and applications (pp. 295–313). Berlin: Springer.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2012

Authors and Affiliations

  • Mohammad Kazemifard
    • 1
  • Nasser Ghasem-Aghaee
    • 2
  • Bryan L. Koenig
    • 3
    • 4
  • Tuncer I. Ören
    • 5
  1. 1.Department of Computer EngineeringRazi UniversityKermanshahIran
  2. 2.Department of Computer EngineeringSheikh Bahaei University and University of IsfahanIsfahanIran
  3. 3.Computational Social Cognition, Institute of High Performance ComputingAgency for Science, Technology, and ResearchSingaporeSingapore
  4. 4.Department of PsychologyNational University of SingaporeSingaporeSingapore
  5. 5.School of Electrical Engineering and Computer ScienceUniversity of OttawaOttawaCanada

Personalised recommendations