Abstract
We examine whether a team of agents can learn geometric and strategic group formations by using deep reinforcement learning in adversarial multi-agent systems. This is a significant point underlying the control and coordination of multiple autonomous and intelligent agents. While there are many possible approaches to solve this problem, we are interested in fully end-to-end learning method where agents do not have any prior knowledge of the environment and its dynamics. In this paper, we propose a scalable and distributed double DQN framework to train adversarial multi-agent systems. We show that a large number of agents can learn to cooperatively move, attack and defend themselves in various geometric formations and battle tactics like encirclement, guerrilla warfare, frontal attack, flanking maneuver, and so on. We finally show that agents create an emergent and collective flocking behaviors by using local views from the environment only.
This work is partly supported by JSPS KAKENHI Grant Number 17KT0044.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balch, T., Arkin, R.C.: Behavior-based formation control for multirobot teams. IEEE Trans. Robot. Autom. 14(6), 926–939 (1998)
Barfoot, T.D., Clark, C.M.: Motion planning for formations of mobile robots. Robot. Autonom. Syst. 46(2), 65–78 (2004)
Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: an overview. In: Srinivasan, D., Jain L.C. (eds) Innovations in Multi-Agent Systems and Applications - 1, Studies in Computational Intelligence, vol 310, Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14435-6_7
Desai, J.P., Ostrowski, J., Kumar, V.: Controlling formations of multiple mobile robots. In: Proceedings 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146), vol. 4, pp. 2864–2869. May 1998
Diallo, E.A.O., Sugiyama, A., Sugawara, T.: Learning to coordinate with deep reinforcement learning in doubles pong game. In: 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 14–19. Dec 2017
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)
Nathan, A., Barbosa, V.C.: V-like formations in flocks of artificial birds. CoRR abs/cs/0611032 (2006). http://arxiv.org/abs/cs/0611032
Rana, O.F., Stout, K.: What is scalability in multi-agent systems? In: Proceedings of the Fourth International Conference on Autonomous Agents, pp. 56–63. AGENTS 2000, ACM, New York (2000)
Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. In: ACM SIGGRAPH Computer Graphics, vol. 21, pp. 25–34. ACM (1987)
Sukhbaatar, S., Szlam, A., Fergus, R.: Learning multiagent communication with backpropagation. In: NIPS (2016)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. AAAI 16, 2094–2100 (2016)
Wooldridge, M.: An Introduction to Multiagent Systems. John Wiley & Sons, Hoboken (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Diallo, E.A.O., Sugawara, T. (2018). Learning Strategic Group Formation for Coordinated Behavior in Adversarial Multi-Agent with Double DQN. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds) PRIMA 2018: Principles and Practice of Multi-Agent Systems. PRIMA 2018. Lecture Notes in Computer Science(), vol 11224. Springer, Cham. https://doi.org/10.1007/978-3-030-03098-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-03098-8_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03097-1
Online ISBN: 978-3-030-03098-8
eBook Packages: Computer ScienceComputer Science (R0)