Integration of Semantic and Episodic Memories for Task Intelligence

  • Ue-Hwan Kim
  • Jong-Hwan KimEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1015)


For successful completion of a task, an intelligent agent needs a carefully designed memory system which could enhance the autonomy of the whole system. Semantic and episodic memories play a key role in majority of memory systems by providing conceptual and sequential information. We propose a novel architecture that integrates both semantic and episodic memories for task intelligence. The adaptive resonance theory (ART) based episodic memory used in our work allows unsupervised learning of new experiences. The semantic memory developed in our architecture incorporates situational information as context. Considering the context enables the agent to generalize the situations where the learned-task can be performed. Thus, the integrated memory architecture progresses the autonomy and the performance of the agent improves even in new environments. To verify the performance of the proposed memory architecture, experiments are conducted under various conditions and the experiment results are analyzed.



This work was partly supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2018-0-00677, Development of Robot Hand Manipulation Intelligence to Learn Methods and Procedures for Handling Various Objects with Tactile Robot Hands) and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2017R1A2A1A17069837).


  1. 1.
    Baldonado, M., Chang, C.-C.K., Gravano, L., Paepcke, A.: The Stanford digital library metadata architecture. Int. J. Digit. Libr. 1, 108–121 (1997)CrossRefGoogle Scholar
  2. 2.
    Bruce, K.B., Cardelli, L., Pierce, B.C.: Comparing object encodings. In: Abadi, M., Ito, T. (eds.) TACS 1997. LNCS, vol. 1281, pp. 415–438. Springer, Heidelberg (1997). Scholar
  3. 3.
    van Leeuwen, J. (ed.): Computer Science Today: Recent Trends and Developments. LNCS, vol. 1000. Springer, Heidelberg (1995). Scholar
  4. 4.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Heidelberg (1996). Scholar
  5. 5.
    Craig, J.J.: Introduction to Robotics: Mechanics and Control, 3rd edn. Prentice Hall, Upper Saddle River (2003)Google Scholar
  6. 6.
    Tsai, L.-W.: Robot Analysis: The Mechanics of Serial and Parallel Manipulators. Wiley, New York (1999)Google Scholar
  7. 7.
    Channon, P.H., Hopkins, S.H., Pham, D.T.: Derivation of optimal walking motions for a bipedal walking robot. Robotica 10, 165–172 (1992)CrossRefGoogle Scholar
  8. 8.
    Elfes, A.: Using occupancy grids for mobile robot perception and navigation. Computer 22, 46–57 (1989)CrossRefGoogle Scholar
  9. 9.
    Correa, J., Soto, A.: Active visual perception for mobile robot localization. J. Intell. Robot. Syst. 58, 339–354 (2009)CrossRefGoogle Scholar
  10. 10.
    Friedman, L.: Robot perception learning. ACM SIGART Bull. (1977)Google Scholar
  11. 11.
    Brooks, R.A.: Symbolic error analysis and robot planning. Int. J. Robot. Res. 1, 29–78 (1982)CrossRefGoogle Scholar
  12. 12.
    Qureshi, A.H., Ayaz, Y., Osaka: Reference tools. Auton. Rob. 1–15 (2015)Google Scholar
  13. 13.
    McDermott, D., Davis, E.: Planning routes through uncertain territory. Artif. Intell. 22, 107–156 (1984)CrossRefGoogle Scholar
  14. 14.
    Schaal, S., Atkeson, C.G.: Robot juggling: implementation of memory-based learning. IEEE Control Syst. 14, 57–71 (1994)CrossRefGoogle Scholar
  15. 15.
    Menegatti, E., Maeda, T., Ishiguro, H.: Image-based memory for robot navigation using properties of omnidirectional images. Rob. Auton. Syst. 47, 251–267 (2004)CrossRefGoogle Scholar
  16. 16.
    Park, G.-M., Kim, J.-H.: Deep adaptive resonance theory for learning biologically inspired episodic memory. In: 2016 International Joint Conference on Neural Networks (IJCNN), Vancouber (2016)Google Scholar
  17. 17.
    Beetz, M., et al.: Cognition-enabled autonomous robot control for the realization of home chore task intelligence. Proc. IEEE 100(8), 2454–2471 (2012)CrossRefGoogle Scholar
  18. 18.
    Kim, J.-H., Choi, S.-H., Park, I.-W., Zaheer, S.A.: Intelligence technology for robots that think [application notes]. IEEE Comput. Intell. Mag. 8, 70–84 (2013)CrossRefGoogle Scholar
  19. 19.
    Yoo, Y.-H., Kim, J.-H.: Procedural memory learning from demonstration for task performance. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Kowloon, pp. 2435–2440 (2015)Google Scholar
  20. 20.
    Park, J.-Y., Yoo, Y.-H., Kim, D.-H., Kim, J.-H.: Integrated adaptive resonance theory neural model for episodic memory with task memory for task performance of robots. In: 2016 International Joint Conference on Neural Networks (IJCNN), Vancouber (2016)Google Scholar
  21. 21.
    Ghallab, M.: PDDL - The planning domain definition language V. 2. Technical report, report CVC TR-98–003/DCS TR-1165, Yale Center for Computational Vision and Control (1998)Google Scholar
  22. 22.
    Fox, M., Long, D.: PDDL2.1: an extension to PDDL for expressing temporal planning domains. J. Al Res. 20, 61–124 (2003)zbMATHGoogle Scholar
  23. 23.
    Nieoiescu, M., Mataric, M.: Natural methods for robot task learning: instructive demonstrations, generalization and practice. In: Proceedings of International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 241–248 (2003)Google Scholar
  24. 24.
    Galindo, C., Fernandez-Madrigal, J., Gonzalez, J.: Improving efficiency in mobile robot task planning through world abstraction. IEEE Trans. Rob. 20, 677–690 (2004)CrossRefGoogle Scholar
  25. 25.
    Harnad, S.: The symbol grounding problem. Physica D: Nonlinear Phenomena 42, 335–346 (1990)CrossRefGoogle Scholar
  26. 26.
    Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernández-Madrigal, J., Gonzalez, J.: Multi-hierarchical semantic maps for mobile robotics. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2278–2283 (2005)Google Scholar
  27. 27.
    Galindo, C., Fernandez-Madrigal, J., Gonzalez, J., Saffiotti, A.: Using semantic information for improving efficiency of robot task planning. In: ICRA Workshop: Semantic Information in Robotics (2007)Google Scholar
  28. 28.
    Beeson, P., MacMahon, M., Modayil, J., Murarka, A., Kuipers, B., Stankiewicz, B.: Integrating multiple representations of spatial knowledge for mapping, navigation, and communication. In: Interaction Challenges for Intelligent Assistants, pp. 1–9 (2007)Google Scholar
  29. 29.
    Galindo, C., Fernández-Madrigal, J., González, J., Saffiotti, A.: Robot task planning using semantic maps. Rob. Auton. Syst. (2008)Google Scholar
  30. 30.
    Tulving, E.: Episodic and semantic memory 1. In: Organization of Memory, vol. 381, no. 4. Academic, London (1972)Google Scholar
  31. 31.
    Carpenter, G., Grossberg, S., Rosen, D.: Fuzzy ART: an adaptive resonance algorithm for rapid, stable classification of analog patterns. In: Proceedings of 1991 International Joint Conference on Neural Networks (1991)Google Scholar
  32. 32.
    Wang, W., Subagdja, B., Tan, A.-H., Starzyk, J.A.: Neural modeling of episodic memory: encoding, retrieval, and forgetting. IEEE Trans. Neural Netw. Learn. Syst. 23, 1574–1586 (2012)CrossRefGoogle Scholar
  33. 33.
    Chemero, A.: An outline of a theory of affordances. Ecol. Psychol. 15, 18–195 (2003)Google Scholar
  34. 34.
    Sahin, E., Cakmak, M., Dogar, M., Ugur, E., Ucoluk, G.: To afford or not to afford: a new formalization of affordances towards affordance-based robot control. Adapt. Behav. 15, 447–472 (2007)CrossRefGoogle Scholar
  35. 35.
    Jeong, I.-B., Ko, W.-R., Park, G.-M., Kim, D.-H., Yoo, Y.-H., Kim, J.-H.: Task intelligence of robots: neural model-based mechanism of thought and online motion planning. IEEE Trans. Emerg. Top. Comput. Intell. 1, 41–50 (2017)CrossRefGoogle Scholar
  36. 36.
    Kim, D.-H., Park, G.-M., Yoo, Y.-H., Ryu, S.-J., Jeong, I.-B., Kim, J.-H.: Realization of task intelligence for service robots in an unstructured environment. Annu. Rev. Control 44, 9–18 (2017)CrossRefGoogle Scholar
  37. 37.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)CrossRefGoogle Scholar
  38. 38.
    Park, G.-M., Yoo, Y.-H., Kim, D.-H., Kim, J.-H.: Deep ART neural model for biologically inspired episodic memory and its application to task performance of robots. IEEE Trans. Cybern. 48, 1786–1799 (2017)CrossRefGoogle Scholar
  39. 39.
    Han, J.-H., Lee, S.-J., Kim, J.-H.: Behavior hierarchy-based affordance map for recognition of human intention and its application to human-robot interaction. IEEE Trans.-Hum. Mach. Syst. 48, 1786–1799 (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.KAISTDaejeonRepublic of Korea

Personalised recommendations