Abstract
This paper is based on the multi-agent deep deterministic policy gradient (MADDPG) deep reinforcement learning algorithm, combined with the Leader-Follow method to complete the multi-agent circular formation control problem, considering the distance constraints and angle constraints between the agents. Overcome the problem that it is difficult to accurately model objects in previous control methods, and do not need to care about network topology, system order and other preconditions. In addition, for the formation movement problem, we predefine a virtual leader that moves according to a random curve trajectory, and adopt a two-stage training method. Based on the circular formation behavior strategy, each agent continues to train to follow the virtual leader. Simulation experiments verify the effectiveness of the algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, C., Yan, P., Pan, W., Guo, J.: Learning-based multi-robot formation control with obstacle avoidance. IEEE Trans. Intell. Transport. Syst. (2021)
Luo, J., Liu, J., Liang, Q.C.: UAV swarm trajectory planning based on a novel particle swarm optimization. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 509–520. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_51
Oh, K.K., Park, M.C., Ahn, H.S.: A survey of multi-agent formation control. Automatica 53, 424–440 (2015)
Huang, H., Bai, Yu., Zhang, D., Fu, W., Bao, R.: Mission planning and synchronous coupling formation control algorithm design for multiple UAVs. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 3486–3496. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_344
Chen, Q., Huang, H., Luo, S., Huang, G., Wang, M.: Time-varying formation control for multi-agent systems with asynchronously switched directed topologies. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 2538–2547. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_250
Canese, L., et al.: Multi-agent reinforcement learning: a review of challenges and applications. Appl. Sci. 11(11), 4948 (2021)
Liu, I.J., Jain, U., Yeh, R.A., Schwing, A.: Cooperative exploration for multi-agent deep reinforcement learning. In: International Conference on Machine Learning, pp. 6826–6836. PMLR (2021)
Chen, Y.F., Everett, M., Liu, M., How, J.P.: Socially aware motion planning with deep reinforcement learning. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1343–1350. IEEE (2017)
Yu, J., Wu, Z., Yang, X., Yang, Y., Zhang, P.: Underwater target tracking control of an untethered robotic fish with a camera stabilizer. IEEE Trans. Syst. Man Cybern. Syst. 51(10), 6523–6534 (2020)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)
Zhang, T., Li, Y., Li, S., Ye, Q., Wang, C., Xie, G.: Decentralized circle formation control for fish-like robots in the real-world via reinforcement learning. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 8814–8820. IEEE (2021)
Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Machine Learning Proceedings 1994, pp. 157–163. Elsevier (1994)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M.: Deterministic policy gradient algorithms. In: International Conference on Machine Learning, pp. 387–395. PMLR (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Beijing HIWING Sci. and Tech. Info Inst
About this paper
Cite this paper
Pan, C., Nian, X., Dai, X., Wang, H., Xiong, H. (2023). Formation Control of Multi-agent Based on Deep Reinforcement Learning. In: Fu, W., Gu, M., Niu, Y. (eds) Proceedings of 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022). ICAUS 2022. Lecture Notes in Electrical Engineering, vol 1010. Springer, Singapore. https://doi.org/10.1007/978-981-99-0479-2_104
Download citation
DOI: https://doi.org/10.1007/978-981-99-0479-2_104
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0478-5
Online ISBN: 978-981-99-0479-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)