Chapter

Multiagent System Technologies

Volume 4687 of the series Lecture Notes in Computer Science pp 37-48

Subgoal Identification for Reinforcement Learning and Planning in Multiagent Problem Solving

  • Chung-Cheng ChiuAffiliated withDepartment of Computer Science, National Tsing Hua University, 101, Section 2 Kuang Fu Road, Hsinchu, Taiwan
  • , Von-Wun SooAffiliated withDepartment of Computer Science, National Tsing Hua University, 101, Section 2 Kuang Fu Road, Hsinchu, TaiwanDepartment of Computer Science and Information Engineering, National Kaohsiung University, 700, Kaohsiung University Rd, Nan Tzu Dist., 811. Kaohsiung, Taiwan

* Final gross prices may vary according to local VAT.

Get Access

Abstract

We provide a new probability flow analysis algorithm to automatically identify subgoals in a problem space. Our flow analysis, inspired by preflow-push algorithms, measures the topological structure of the problem space to identify states that connect different subset of state space as the subgoals within linear-time complexity. Then we apply a hybrid approach known as subgoal-based SMDP (semi-Markov Decision Process) that is composed of reinforcement learning and planning based on the identified subgoals to solve the problem in a multiagent environment. The effectiveness of this new method used in a multiagent system is demonstrated and evaluated using a capture-the-flag scenario. We showed also that the cooperative coordination emerged between two agents in the scenario through distributed policy learning.