A Hybrid Architecture Combining Reactive Plan Execution and Reactive Learning
Developing software agents has been complicated by the problem of how knowledge should be represented and used. Many researchers have identified that agents need not require the use of complex representations, but in many cases suffice to use “the world” as their representation. However, the problem of introspection, both by the agents themselves and by (human) domain experts, requires a knowledge representation with a higher level of abstraction that is more ‘understandable’. Learning and adaptation in agents has traditionally required knowledge to be represented at an arbitrary, low-level of abstraction. We seek to create an agent that has the capability of learning as well as utilising knowledge represented at a higher level of abstraction.
We firstly explore a reactive learner (Falcon) and reactive plan execution engine based on BDI (JACK) through experiments and analysis. We then describe an architecture we have developed that combines the BDI framework to the low-level reinforcement learner and present promising results from experiments using our minefield navigation domain.
KeywordsReinforcement Learner Multiagent System Domain Expert Navigation Task Hybrid Architecture
Unable to display preview. Download preview PDF.
- 1.Ackley, D., Littman, M.: Generalizaion and scaling in reinforcement learning. In: Advances in Neural Information Processing Systems 2, pp. 550–557 (1990)Google Scholar
- 7.Gordan, D., Subramanian, D.: A cognitive model of learning to navigate. In: Nineteenth Annual Conference of the Cognitive Science Society (1997)Google Scholar
- 8.Heintz, F., Doherty, P.: DyKnow: A Framework for Processing Dynamic Knowledge and Object Structures in Autonomous Systems. In: Intl. Workshop on Monitoring, Security and Rescue Techniques in MAS (2004)Google Scholar
- 9.Heinze, C., Goss, S., Pearce, A.: Plan Recognition in Military Simulation: Incorporating Machine Learning with Intelligent Agents. In: Proceedings, IJCAI Workshop on Team Behaviour and Plan Recognition, Stockholm, Sweden, pp. 53–63 (1999)Google Scholar
- 10.Kaelbling, L., Littman, M., Moore, A.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
- 11.Karim, S., Heinze, C.: Experiences with the Design and Implementation of an Agent-based Autonomous UAV Controller. In: Proccedings of the Fourth International Conference on Autonomous Agents and Multiagent Systems (AAMAS), University of Utrecht, The Netherlands (2005)Google Scholar
- 12.Newell, A.: Unified Theories of Cognition. Harvard University Press, Cambridge (1990)Google Scholar
- 13.Rao, A.S., Georgeff, M.P.: BDI-agents: from theory to practice. In: Proceedings, First International Conference on Multiagent Systems, San Francisco (1995)Google Scholar
- 14.Sun, R.: Duality of the mind: A bottom-up approach toward cognition. Lawrence Erlbaum Associates, Inc., Mahwah (2002)Google Scholar
- 17.Tan, A.H.: FALCON: A fusion architecture for learning, cognition, and navigation. In: Proceedings, IJCNN, Budapest, pp. 3297–3302 (2004)Google Scholar