Abstract
This paper presents an adaptive traffic control system based on deep convolution neural network (DCNN) technique for a multimodal traffic environment. The deep learning controller technique is based on feature mapping to estimate the optimal state-action value function. The controller executes acyclic phase assignment based on minimum green time duration for traffic signal operation. Moreover, the research implements Discrete Lane Cells (DLC) approach for state representation. A number of traffic performance measures were examined and selected traffic conditions were tested. The findings indicate that the DCNN system has superior performance in over-saturated traffic condition. The DCNN agent has achieved significant 85% to 95% lower waiting time, l7% to 38% shorter travel time, and it has mitigated the highest median flow rate at 295 veh/s in comparison to other traffic systems. For under-saturated test scenarios, a fair comparable performance is measured for the proposed controller. Overall, the DCNN system provides stable performance across all the tested signal junction scenarios in comparison to other controller systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdoos M, Mozayani N, Bazzan AL (2014) Hierarchical control of traffic signals using Q-learning with tile coding. Applied Intelligence 40(2):201–213, DOI: https://doi.org/10.1007/s10489-013-0455-3
Abdulhai B, Pringle R, Karakoulas GJ (2003) Reinforcement learning for true adaptive traffic signal control. Journal of Transportation Engineering 129(3):278–285, DOI: https://doi.org/10.1061/(asce)0733-947x(2003)129:3(278)
Aslani M, Megsgari MS, Wiering M (2017) Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events. Transportation Research Part C: Emerging Technologies 85:732–752, DOI: https://doi.org/10.1016/j.trc.2017.09.020
Casas N (2017) Deep deterministic policy gradient for urban traffic light control, arXiv preprint arXiv:1703.09035, Mar. 2017, DOI: https://doi.org/10.48550/arXiv.1703.09035
Chin YK, Bolong N, Kiring A, Yang SS, Teo KTK (2011) Q-learning based traffic optimization in management of signal timing plan. International Journal of Simulation, Systems, Science & Technology 12(3):29–35, DOI: https://doi.org/10.1109/CICSyN.2011.64
Chu KF, Lam AYS, Li VOK (2021) Traffic signal control using end-to-end off-policy deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems 1–12, DOI: https://doi.org/10.1109/TITS.2021.3067057
Chu T, Wang J, Codecà L, Li Z (2019) Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on Intelligent Transportation Systems 21(3):1086–1095, DOI: https://doi.org/10.1109/TITS.2019.2901791
Day CM, Bullock DM, Li H, Lavrenz S, Smith WB, Sturdevant JR (2014) Performance measures for traffic signal systems: An outcome-oriented approach. West Lafayette, Purdue University, USA, DOI: https://doi.org/10.5703/1288284316063
Gao J, Shen Y, Liu J, Ito M, Shiratori N (2017) Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network. arXiv preprint arXiv:1705.02755, DOI: https://doi.org/10.48550/arXiv.1705.02755
Genders W, Razavi S (2016) Using a deep reinforcement learning agent for traffic signal control, arXiv preprint arXiv:1611.01142, DOI: https://doi.org/10.48550/arXiv.1611.01142
Han M (2018) Reinforcement learning approaches in dynamic environments. PhD Thesis, Télécom ParisTech, Paris, France
Haydari A, Yilmaz Y (2020) Deep reinforcement learning for intelligent transportation systems: A survey. IEEE Transactions on Intelligent Transportation Systems 201–216, DOI: https://doi.org/10.1109/TITS.2020.3008612
Joo H, Ahmed SH, Lim Y (2020) Traffic signal control for smart cities using reinforcement learning. Computer Communications 154:324–330, DOI: https://doi.org/10.1016/j.comcom.2020.03.005
Khamis MA, Gomaa W (2014) Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. Engineering Applications of Artificial Intelligence 29:134–151, DOI: https://doi.org/10.1016/j.engappai.2014.01.007
Krishna KH, Kumar KV, Rao CH (2018) Signal design using Webster’s method (4 legged intersection). Indian J. Sci. Res. 17(2):113119
Li L, Lv Y, Wang FY (2016) Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica 3(3):247–254, DOI: https://doi.org/10.1109/JAS.2016.7508798
Liang X, Du X, Wang G, Han Z (2019) A deep reinforcement learning network for traffic light cycle control. IEEE Transactions on Vehicular Technology 68(2):1243–1253, DOI: https://doi.org/10.1109/TVT.2018.2890726
Liu W, Qin G, He Y, Jiang F (2017) Distributed cooperative reinforcement learning-based traffic signal control that integrates v2x networks Dynamic Clustering. IEEE Transactions on Vehicular Technology 66(10):8667–8681, DOI: https://doi.org/10.1109/tvt.2017.2702388
Ma J, Fontaine MD, Zhou F, Hu J, Hale DK, Clements MO (2016) Estimation of crash modification factors for an adaptive traffic-signal control system. Journal of Transportation Engineering 142(12): 04016061, DOI: https://doi.org/10.1061/(asce)te.1943-5436.0000890
Nagabandi A, Kahn G, Fearing RS, Levine S (2018) Neural network dynamics for model-based deep reinforcement learning with modelfree fine-tuning. 2018 IEEE International Conference on Robotics and Automation (ICRA), May 21, 7559–7566
Park B, Won J, Perfater MA (2006) Microscopic simulation model calibration and validation handbook (No. FHWA/VTRC 07-CR6). Virginia Transportation Research Council, Charlottesville, VA, USA
Pham TT, Brys T, Taylor ME (2013) Learning coordinated traffic light control. In Proceedings of the Adaptive and Learning Agents workshop (at AAMAS-13) 10:1196–1201
Prashanth LA, Bhatnagar S (2011) Reinforcement learning with function approximation for traffic signal control. IEEE Transactions on Intelligent Transportation Systems 12(2):412–421, DOI: https://doi.org/10.1109/TITS.2010.2091408
Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT Press
Thorpe TL (1997) Vehicle traffic light control using SARSA. Master’s Project Rep., Computer Science Dept., Colorado State University, Fort Collins, Colorado, USA
Urbanik T, Tanaka A, Lozner B, Lindstrom E, Lee K, Quayle S, Beaird S, Tsoi S, Ryus P, Gettman D, Sunkari S (2014) Signal timing manual. vol 1, 2nd ed. Washington, Transportation Research Board, Washington, DC, USA
Wan CH, Hwang MC (2018) Value-based deep reinforcement learning for adaptive isolated intersection signal control. IET Intelligent Transport Systems 12(9):1005–1010, DOI: https://doi.org/10.1049/iet-its.2018.5170
Wang S, Xie X, Huang K, Zeng J, Cai Z (2019) Deep reinforcement learning-based traffic signal control using high-resolution event-based data. Entropy 21(8):744, DOI: https://doi.org/10.3390/e21080744
Wang Y, Yang X, Liang H, Liu Y (2018) A review of the self-adaptive traffic signal control system based on future traffic environment. Journal of Advanced Transportation 2018:1–12, DOI: https://doi.org/10.1155/2018/1096123
Wunderlich RJ (2007) A Longest-queue-first signal scheduled algorithm with quality of service provisioning for an isolated intersection. MSc Thesis, University of Tennessee, Tennessee, United States
Yen CC, Ghosal D, Zhang M, Chuah CN (2020) A deep on-policy learning agent for traffic signal control of multiple intersections. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) 1–6, DOI: https://doi.org/10.1109/ITSC45102.2020.9294471
Yin B, Dridi M, El Moudni A (2014) Approximate dynamic programming for traffic signal control at isolated intersection. Advances in Intelligent Systems and Computing 285:369–381, DOI: https://doi.org/10.1007/978-3-319-06740-731
Younes MB, Boukerche A (2015) Intelligent traffic light controlling algorithms using vehicular networks. IEEE Transactions on Vehicular Technology 65(8):5887–5899, DOI: https://doi.org/10.1109/TVT.2015.2472367
Zhao D, Dai Y, Zhang Z (2012) Computational intelligence in urban traffic signal control: A survey. Ieee Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42(4):485–494, DOI: https://doi.org/10.1109/TSMCC.2011.2161577
Acknowledgments
The authors acknowledge the assistance of JA Project Consultant Sdn. Bhd., Kuala Lumpur, Malaysia in obtaining the valuable traffic data required to conduct this research.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ahmed, M.A.A., Khoo, H.L. & Ng, OE. Application of Convolution Neural Network for Adaptive Traffic Controller System. KSCE J Civ Eng 26, 4062–4072 (2022). https://doi.org/10.1007/s12205-022-1936-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12205-022-1936-x