Abstract
The rise of the artificial intelligence (AI) brings golden opportunity to accelerate the development of the intelligent transportation system (ITS). The platoon control of connected autonomous vehicle (CAV) as the key technology exhibits superior for improving traffic system. However, there still exist some challenges in multi-objective platoon control and multi-agent interaction. Therefore, this paper proposed a connected autonomous vehicle latoon control approach with multi-agent deep reinforcement learning (MADRL). Finally, the results in stochastic mixed traffic flow based on SUMO (simulation of urban mobility) platform demonstrate that the proposed method is feasible, effective and advanced.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, D., Wang, Y., Shen, Y.: Electric vehicle charging and discharging coordination on distribution network using multi-objective particle swarm optimization and fuzzy decision making. Energies 9(3), 186 (2016)
Delgarm, N., Sajadi, B., Kowsary, F., et al.: Multi-objective optimization of the building energy performance: a simulation-based approach by means of particle swarm optimization (PSO). Appl. Energy 170, 293–303 (2016)
Zhang, Y., Guo, L., Gao, B., Qu, T., Chen, H.: Deterministic promotion reinforcement learning applied to longitudinal velocity control for automated vehicles. IEEE Trans. Veh. Technol. 69(1), 338–348 (2020). https://doi.org/10.1109/TVT.2019.2955959
Xu, G., et al.: Hierarchical speed control for autonomous electric vehicle through deep reinforcement learning and robust control. IET Control Theory Appl. 1–13 (2021). https://doi.org/10.1049/cth2.12211
Jardine, P.T.: A reinforcement learning approach to predictive control design: autonomous vehicle applications. Queen’s University (Canada) (2018)
Hang, P., Lv, C., Huang, C., Cai, J., Hu, Z., Xing, Y.: An integrated framework of decision making and motion planning for autonomous vehicles considering social behaviors. Electr. Eng. Syst. Sci. 1–11 (2020)
Xu, J., Shu, H., Shao, Y.: Modeling of driver behavior on trajectory-speed decision making in minor traffic roadways with complex features. IEEE Trans. Intell. Transp. Syst. 20(1), 41–53 (2019). https://doi.org/10.1109/TITS.2018.2800086
Liu, T., Wang, B., Cao, D., Tang, X., Yang, Y.: Integrated longitudinal speed decision-making and energy efficiency control for connected electrified vehicles. Electr. Eng. Syst. Sci. 1–11 (2020)
He, X., Fei, C., Liu, Y., Yang, K., Ji, X.: Multi-objective longitudinal decision-making for autonomous electric vehicle: a entropy-constrained reinforcement learning approach. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, pp. 1–6 (2020). https://doi.org/10.1109/ITSC45102.2020.9294736
Kreidieh, A.R., Wu, C., Bayen, A.M.: Dissipating stop-and-go waves in closed and open networks via deep reinforcement learning. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 1475–1480. IEEE, November 2018
Vinitsky, E., et al.: Benchmarks for reinforcement learning in mixed-autonomy traffic. In: Conference on Robot Learning, pp. 399–409. PMLR, October 2018
Achiam, J., Held, D., Tamar, A., Abbeel, P.: Constrained policy optimization. In: International Conference on Machine Learning, pp. 22–31. PMLR, July 2017
Bhatnagar, S., Sutton, R.S., Ghavamzadeh, M., Lee, M.: Natural actor-critic algorithms. Automatica 45(11), 2471–2482 (2009)
Cao, X.R.: A basic formula for online policy gradient algorithms. IEEE Trans. Autom. Control 50(5), 696–699 (2005)
Schulman, J., Wolski, F., Dhariwal, P., et al.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Wu, C., Kreidieh, A.R., Parvate, K., Vinitsky, E., Bayen, A.M.: Flow: a modular learning framework for mixed autonomy traffic. IEEE Trans. Robot. (2021)
Krajzewicz, D., Erdmann, J., Behrisch, M., Bieker, L.: Recent development and applications of sumo-simulation of urban mobility. Int. J. Adv. Syst. Meas. 5(3&4) (2012)
Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P.: Benchmarking deep reinforcement learning for continuous control. CoRR, vol. abs/1604.06778 (2016). http://arxiv.org/abs/1604.06778
Liang, E., et al.: Ray RLlib: a composable and scalable reinforcement learning library. arXiv preprint arXiv:1712.09381 (2017)
Brockman, G., et al.: OpenAI Gym. arXiv preprint arXiv:1606.01540 (2016)
Treiber, M., Kesting, A.: Trajectory and floating-car data. In: Treiber, M., Kesting, A. (eds.) Traffic Flow Dynamics, pp. 7–12. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-32460-4_2.
Xu, G., et al.: Hierarchical speed control for autonomous electric vehicle through deep reinforcement learning and robust control. IET Control Theory Appl. (2021)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Xu, G., Chen, B., Li, G., He, X. (2022). Connected Autonomous Vehicle Platoon Control Through Multi-agent Deep Reinforcement Learning. In: Xiang, W., Han, F., Phan, T.K. (eds) Broadband Communications, Networks, and Systems. BROADNETS 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 413. Springer, Cham. https://doi.org/10.1007/978-3-030-93479-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-93479-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93478-1
Online ISBN: 978-3-030-93479-8
eBook Packages: Computer ScienceComputer Science (R0)