Abstract
Multiagent systems (MAS) have been generally applied in numerous applications, including computer networks, robotics, and smart grids due to their flexibility, reliability for complex problem-solving. Communication is an important factor for the multiagent world to stay organized and productive. Previously, most existing studies try to pre-define the communication protocols or adopt additional decision modules for instructing the communication schedule, which induces significant communication cost overhead and cannot generalized to a large collection of agents directly. In this paper, we propose a lightweight communication framework—Pheromone Collaborative Deep Q-Network (PCDQN), which combines deep Q-network with the pheromone-driven stigmergy mechanism. In partially observable environments, this framework exploits the stigmergy as circuitous communication connections among independent reinforcement learning agents. Experiments dependent on the minefield navigation task have shown that PCDQN displays superiority in accomplishing higher learning productivity of multiple agents when contrasted with Deep Q-network (DQN).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dorigo, M., Bonabeau, E., Theraulaz, G.: Ant algorithms and stigmergy. Futur. Gener. Comput. Syst. 16(8), 851–871 (2000)
Dorigo, M., Gambardella, L.M.: Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans. Evol. Comput. 1(1), 53–66 (1997)
Dorri, A., Kanhere, S.S., Jurdak, R.: Multi-agent systems: a survey. IEEE Access 6, 28573–28593 (2018)
Heylighen, F.: Stigmergy as a universal coordination mechanism I: definition and components. Cogn. Syst. Res. 38, 4–13 (2016)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Musil, J., Musil, A., Biffl, S.: Introduction and challenges of environment architectures for collective intelligence systems. In: Weyns, D., Michel, F. (eds.) E4MAS 2014. LNCS (LNAI), vol. 9068, pp. 76–94. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23850-0_6
Naeem, B., Javed, S., Kasi, M.K., Sani, K.A.: Hybrid fuzzy logic engine for ping-pong effect reduction in cognitive radio network. Wireless Pers. Commun. 116(1), 177–205 (2021)
Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. arXiv preprint arXiv:1511.05952 (2015)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introductio, 2nd edn. The MIT Press (2018). http://incompleteideas.net/book/the-book-2nd.html
Tan, A.H., Lu, N., Xiao, D.: Integrating temporal difference methods and self-organizing neural networks for reinforcement learning with delayed evaluative feedback. IEEE Trans. Neural Networks 19(2), 230–244 (2008)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Vinyals, O., et al.: Alphastar: mastering the real-time strategy game starcraft II. DeepMind Blog 2 (2019)
Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., Freitas, N.: Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1995–2003. PMLR (2016)
Weiß, G.: Adaptation and learning in multi-agent systems: some remarks and a bibliography. In: Weiß, G., Sen, S. (eds.) IJCAI 1995. LNCS, vol. 1042, pp. 1–21. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-60923-7_16
Acknowledgement
This work was supported in part by the National Natural Science Foundation of China under Grant 61906032, the NSFC-Liaoning Province United Foundation under Grant U1908214, the Fundamental Research Funds for the Central Universities under grant DUT21TD107, the LiaoNing Revitalization Talents Program, No. XLYC2008017, the National Key Research and Development Program of China under Grant 2018YFC0910500, the National Natural Science Foundation of China under Grant 61976034, and the Liaoning Key Research and Development Program under Grant 2019JH2/10100030.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, K., Hou, Y., Yu, H., Zhu, W., Feng, L., Zhang, Q. (2021). Pheromone Based Independent Reinforcement Learning for Multiagent Navigation. In: Zhang, H., Yang, Z., Zhang, Z., Wu, Z., Hao, T. (eds) Neural Computing for Advanced Applications. NCAA 2021. Communications in Computer and Information Science, vol 1449. Springer, Singapore. https://doi.org/10.1007/978-981-16-5188-5_4
Download citation
DOI: https://doi.org/10.1007/978-981-16-5188-5_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5187-8
Online ISBN: 978-981-16-5188-5
eBook Packages: Computer ScienceComputer Science (R0)