Subgoal Identification for Reinforcement Learning and Planning in Multiagent Problem Solving

  • Chung-Cheng Chiu
  • Von-Wun Soo
Conference paper

DOI: 10.1007/978-3-540-74949-3_4

Volume 4687 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Chiu CC., Soo VW. (2007) Subgoal Identification for Reinforcement Learning and Planning in Multiagent Problem Solving. In: Petta P., Müller J.P., Klusch M., Georgeff M. (eds) Multiagent System Technologies. MATES 2007. Lecture Notes in Computer Science, vol 4687. Springer, Berlin, Heidelberg

Abstract

We provide a new probability flow analysis algorithm to automatically identify subgoals in a problem space. Our flow analysis, inspired by preflow-push algorithms, measures the topological structure of the problem space to identify states that connect different subset of state space as the subgoals within linear-time complexity. Then we apply a hybrid approach known as subgoal-based SMDP (semi-Markov Decision Process) that is composed of reinforcement learning and planning based on the identified subgoals to solve the problem in a multiagent environment. The effectiveness of this new method used in a multiagent system is demonstrated and evaluated using a capture-the-flag scenario. We showed also that the cooperative coordination emerged between two agents in the scenario through distributed policy learning.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Chung-Cheng Chiu
    • 1
  • Von-Wun Soo
    • 1
    • 2
  1. 1.Department of Computer Science, National Tsing Hua University, 101, Section 2 Kuang Fu Road, Hsinchu, TaiwanR.O.C.
  2. 2.Department of Computer Science and Information Engineering, National Kaohsiung University, 700, Kaohsiung University Rd, Nan Tzu Dist., 811. Kaohsiung, TaiwanR.O.C.