Abstract
Deep reinforcement learning (DRL) methodology with traffic control systems plays a vital role in adaptive traffic signal controls. However, previous studies have frequently disregarded the significance of vehicles near intersections, which typically involve higher decision-making requirements and safety considerations. To overcome this challenge, this paper presents a novel DRL-based method for traffic signal control, which incorporates an attention mechanism into the Dueling Double Deep Q Network (D3QN) framework. This approach emphasizes the priority of vehicles near intersections by assigning them higher weights and more attention. Moreover, the state design incorporates signal light statuses to facilitate a more comprehensive understanding of the current traffic environment. Furthermore, the model’s performance is enhanced through the utilization of Double DQN and Dueling DQN techniques. The experimental findings demonstrate the superior efficacy of the proposed method in critical metrics such as vehicle waiting time, queue length, and the number of halted vehicles when compared to D3QN, traditional DQN, and fixed timing strategies.
This work was supported in part by JiangXi Education Department under Grant No. GJJ191688.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Papageorgiou, M., Diakaki, C., Dinopoulou, V., Kotsialos, A., Wang, Y.: Review of road traffic control strategies. Proc. IEEE 91(12), 2043–2067 (2003). https://doi.org/10.1109/JPROC.2003.819610
Sutton, R.S., Barto, A.G., et al.: Introduction to Reinforcement Learning, vol. 135. MIT Press Cambridge (1998)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., Freitas, N.: Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1995–2003. PMLR (2016)
El-Tantawy, S., Abdulhai, B.: An agent-based learning towards decentralized and coordinated traffic signal control. In: 13th International IEEE Conference on Intelligent Transportation Systems, pp. 665–670. IEEE (2010)
Jin, J., Ma, X.: A group-based traffic signal control with adaptive learning ability. Eng. Appl. Artif. Intell. 65, 282–293 (2017)
Li, L., Lv, Y., Wang, F.Y.: Traffic signal timing via deep reinforcement learning. IEEE/CAA J. Automatica Sinica 3(3), 247–254 (2016)
Liang, X., Du, X., Wang, G., Han, Z.: A deep q learning network for traffic lights’ cycle control in vehicular networks. IEEE Trans. Veh. Technol. 68(2), 1243–1253 (2019)
Genders, W., Razavi, S.: Using a deep reinforcement learning agent for traffic signal control (2016)
Zhang, L., et al.: DynamicLight: dynamically tuning traffic signal duration with DRL (2022)
Gao, J., Shen, Y., Liu, J., Ito, M., Shiratori, N.: Adaptive traffic signal control: deep reinforcement learning algorithm with experience replay and target network. arXiv preprint arXiv:1705.02755 (2017)
Mao, F., Li, Z., Li, L.: A comparison of deep reinforcement learning models for isolated traffic signal control. IEEE Intell. Transp. Syst. Mag. 15(1), 160–180 (2023). https://doi.org/10.1109/MITS.2022.3144797
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks (2019)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Hallinan, A.J. Jr.: A review of the Weibull distribution. J. Qual. Technol. 25(2), 85–93 (1993)
Webster, F.V.: Traffic signal settings. Road Research Technical Paper 39 (1958)
Mnih, V., et al.: Playing Atari with deep reinforcement learning (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ni, W., Wang, P., Li, Z., Li, C. (2024). Traffic Signal Control Optimization Based on Deep Reinforcement Learning with Attention Mechanisms. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14449. Springer, Singapore. https://doi.org/10.1007/978-981-99-8067-3_11
Download citation
DOI: https://doi.org/10.1007/978-981-99-8067-3_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8066-6
Online ISBN: 978-981-99-8067-3
eBook Packages: Computer ScienceComputer Science (R0)