Autonomous Agents and MultiAgent Systems
, Volume 26, Issue 1, pp 86119
First online:
Cooperative reinforcement learning in topologybased multiagent systems
 Dan XiaoAffiliated withSchool of Computer Engineering, Nanyang Technological University Email author
 , AhHwee TanAffiliated withSchool of Computer Engineering, Nanyang Technological University
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
Topologybased multiagent systems (TMAS), wherein agents interact with one another according to their spatial relationship in a network, are well suited for problems with topological constraints. In a TMAS system, however, each agent may have a different state space, which can be rather large. Consequently, traditional approaches to multiagent cooperative learning may not be able to scale up with the complexity of the network topology. In this paper, we propose a cooperative learning strategy, under which autonomous agents are assembled in a binary tree formation (BTF). By constraining the interaction between agents, we effectively unify the state space of individual agents and enable policy sharing across agents. Our complexity analysis indicates that multiagent systems with the BTF have a much smaller state space and a higher level of flexibility, compared with the general form of nary (n > 2) tree formation. We have applied the proposed cooperative learning strategy to a class of reinforcement learning agents known as temporal differencefusion architecture for learning and cognition (TDFALCON). Comparative experiments based on a generic network routing problem, which is a typical TMAS domain, show that the TDFALCON BTF teams outperform alternative methods, including TDFALCON teams in single agent and nary tree formation, a Qlearning method based on the table lookup mechanism, as well as a classical linear programming algorithm. Our study further shows that TDFALCON BTF can adapt and function well under various scales of network complexity and traffic volume in TMAS domains.
Keywords
Topologybased multiagent systems Cooperative learning Reinforcement learning Binary tree formation Policy sharing Title
 Cooperative reinforcement learning in topologybased multiagent systems
 Journal

Autonomous Agents and MultiAgent Systems
Volume 26, Issue 1 , pp 86119
 Cover Date
 201301
 DOI
 10.1007/s1045801191834
 Print ISSN
 13872532
 Online ISSN
 15737454
 Publisher
 Springer US
 Additional Links
 Topics
 Keywords

 Topologybased multiagent systems
 Cooperative learning
 Reinforcement learning
 Binary tree formation
 Policy sharing
 Industry Sectors
 Authors

 Dan Xiao ^{(1)}
 AhHwee Tan ^{(1)}
 Author Affiliations

 1. School of Computer Engineering, Nanyang Technological University, Nanyang Avenue, Singapore, 639798, Singapore