A Hybrid Architecture Combining Reactive Plan Execution and Reactive Learning

  • Samin Karim
  • Liz Sonenberg
  • Ah-Hwee Tan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4099)


Developing software agents has been complicated by the problem of how knowledge should be represented and used. Many researchers have identified that agents need not require the use of complex representations, but in many cases suffice to use “the world” as their representation. However, the problem of introspection, both by the agents themselves and by (human) domain experts, requires a knowledge representation with a higher level of abstraction that is more ‘understandable’. Learning and adaptation in agents has traditionally required knowledge to be represented at an arbitrary, low-level of abstraction. We seek to create an agent that has the capability of learning as well as utilising knowledge represented at a higher level of abstraction.

We firstly explore a reactive learner (Falcon) and reactive plan execution engine based on BDI (JACK) through experiments and analysis. We then describe an architecture we have developed that combines the BDI framework to the low-level reinforcement learner and present promising results from experiments using our minefield navigation domain.


Reinforcement Learner Multiagent System Domain Expert Navigation Task Hybrid Architecture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ackley, D., Littman, M.: Generalizaion and scaling in reinforcement learning. In: Advances in Neural Information Processing Systems 2, pp. 550–557 (1990)Google Scholar
  2. 2.
    Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., Soda, G.: Bidirectional Dynamics for Protein Secondary Structure Prediction. In: Sun, R., Giles, C.L. (eds.) IJCAI-WS 1999. LNCS (LNAI), vol. 1828, Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. 3.
    Bratman, M.E., Israel, D., Pollack, M.: Plans and resource-bounded practical reasoning. Computational Intelligence 4(4), 349–355 (1988)CrossRefGoogle Scholar
  4. 4.
    Brooks, R.: Cambrian Intelligence: The Early History of the New AI. MIT Press, Cambridge (1999) (A Bradford Book)zbMATHGoogle Scholar
  5. 5.
    Carpenter, G.A., Grossberg, S.: A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing 37, 54–115 (1987)CrossRefGoogle Scholar
  6. 6.
    Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 3, 698–713 (1992)CrossRefGoogle Scholar
  7. 7.
    Gordan, D., Subramanian, D.: A cognitive model of learning to navigate. In: Nineteenth Annual Conference of the Cognitive Science Society (1997)Google Scholar
  8. 8.
    Heintz, F., Doherty, P.: DyKnow: A Framework for Processing Dynamic Knowledge and Object Structures in Autonomous Systems. In: Intl. Workshop on Monitoring, Security and Rescue Techniques in MAS (2004)Google Scholar
  9. 9.
    Heinze, C., Goss, S., Pearce, A.: Plan Recognition in Military Simulation: Incorporating Machine Learning with Intelligent Agents. In: Proceedings, IJCAI Workshop on Team Behaviour and Plan Recognition, Stockholm, Sweden, pp. 53–63 (1999)Google Scholar
  10. 10.
    Kaelbling, L., Littman, M., Moore, A.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
  11. 11.
    Karim, S., Heinze, C.: Experiences with the Design and Implementation of an Agent-based Autonomous UAV Controller. In: Proccedings of the Fourth International Conference on Autonomous Agents and Multiagent Systems (AAMAS), University of Utrecht, The Netherlands (2005)Google Scholar
  12. 12.
    Newell, A.: Unified Theories of Cognition. Harvard University Press, Cambridge (1990)Google Scholar
  13. 13.
    Rao, A.S., Georgeff, M.P.: BDI-agents: from theory to practice. In: Proceedings, First International Conference on Multiagent Systems, San Francisco (1995)Google Scholar
  14. 14.
    Sun, R.: Duality of the mind: A bottom-up approach toward cognition. Lawrence Erlbaum Associates, Inc., Mahwah (2002)Google Scholar
  15. 15.
    Sun, R., Sessions, C.: Learning plans without a priori knowledge. Adaptive Behavior 8(3/4), 225–254 (2000)CrossRefGoogle Scholar
  16. 16.
    Tan, A.H.: Adaptive Resonance Associative Map. Neural Networks 8(3), 437–446 (1995)CrossRefGoogle Scholar
  17. 17.
    Tan, A.H.: FALCON: A fusion architecture for learning, cognition, and navigation. In: Proceedings, IJCNN, Budapest, pp. 3297–3302 (2004)Google Scholar
  18. 18.
    Veloso, M., Carbonell, J., Pérez, A., Borrajo, D., Fink, E., Blythe, J.: Integrating Planning and Learning: The PRODIGY Architecture. Journal of Experimental and Theoretical Artificial Intelligence 7(1), 81–120 (1995)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Samin Karim
    • 1
  • Liz Sonenberg
    • 1
  • Ah-Hwee Tan
    • 2
  1. 1.Department of Information SystemsUniversity of MelbourneMelbourneAustralia
  2. 2.School of Computer EngineeringNanyang Technological UniversitySingapore

Personalised recommendations