Skip to main content

Advertisement

SpringerLink
Log in

Adaptive traffic light control using deep reinforcement learning technique

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Smart city growth needs information and communication technology to increase urban sustainability but faces critical traffic congestion and vehicle classification issues. It is crucial to dynamically change the traffic light on the road network to reduce the delay of vehicles and avoid congestion in the smart city. Modifying the traffic light should be adaptive, considering the number of vehicles on the road and the options available to route the vehicles toward their destination. Our scheme is the first proposed model based on deep learning to solve the problem of traffic congestion in the urban environment. This model classifies the vehicle’s type on the road and assigns different vehicle weights. We assign 0.0 for no vehicles, and 1.0, 2.0, 3.0 for light-weight, moderate-weight, and heavy-weight vehicles respectively. The proposed work has trained using experience replay and target network based on a deep double-Q learning mechanism. Our resultant model applies in a real-time traffic network that uses Dedicated-Short-Range-Communication (DSRC) protocol for wireless communication. The simulation of this work uses SUMO (Simulation in Urban MObility) with the data generated on SUMO using a random function. The results show that the traffic light of a certain traffic intersection becomes adaptive, aligning with the goals mentioned above. The proposed model efficiently reduces the average waiting time up to 91.7% at the intersection points of the road which is shown in the graph in the result section.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Taken from multi-agent [37]

Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data is generated through a random function on SUMO.

Code availability

Custom code using SUMO.

References

  1. Bellman R, Kalaba R (1957) Dynamic programming and statistical communication theory. Proc Natl Acad Sci USA 43(8):749

    Article  MathSciNet  Google Scholar 

  2. Casas N (2017) Deep deterministic policy gradient for urban traffic light control,” arXiv preprint arXiv:1703.09035

  3. Christopher J (1992) Watkins and peter dayan. Q-Learn Mach Learn 8(3):279–292

    Google Scholar 

  4. Coşkun M, Baggag A, Chawla S (2018) Deep reinforcement learning for traffic light optimization. In: 2018 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 564–571

  5. Farazi NP, Ahamed T, Barua L, Zou B (2020) Deep reinforcement learning and transportation research: A comprehensive review. https://doi.org/10.48550/arXiv.2010.06187

  6. Gao J, Shen Y, Liu J, Ito M, Shiratori N (2017) Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network. arXiv preprint arXiv:1705.02755

  7. Garg D, Chli M, Vogiatzis G (2018) Deep reinforcement learning for autonomous traffic light control. In: 2018 3rd IEEE international conference on intelligent transportation engineering (ICITE), IEEE, Singapore, pp 214–218. https://doi.org/10.1109/ICITE.2018.8492537

  8. Genders W, Razavi S (2016) Using a deep reinforcement learning agent for traffic signal control. https://doi.org/10.48550/arXiv.1611.01142

  9. Gong Y, Abdel-Aty M, Cai Q, Rahman MS (2019) Decentralized network level adaptive signal control by multi-agent deep reinforcement learning. Transp Res Interdiscip Perspect 1:100020

    Google Scholar 

  10. Hu X, Zhao C, Wang G (2020) A traffic light dynamic control algorithm with deep reinforcement learning based on GNN Prediction. https://doi.org/10.48550/arXiv.2009.14627

  11. Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1–2):99–134

    Article  MathSciNet  Google Scholar 

  12. Kumar N, Rahman SS, Dhakad N (2020) Fuzzy inference enabled deep reinforcement learning-based traffic light control for intelligent transportation system. IEEE Trans IntellTransp Syst

  13. Li C, Ma X, Xia L, Zhao Q, Yang J (2020) Fairness control of traffic light via deep reinforcement learning. In: 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE). IEEE, Hong Kong, China, pp 652–658. https://doi.org/10.1109/CASE48305.2020.9216899

  14. Li L, Lv Y, Wang F-Y (2016) Traffic signal timing via deep reinforcement learning. IEEE/CAA J Automat Sin 3(3):247–254

    Article  MathSciNet  Google Scholar 

  15. Liang X (2019) Applied deep learning in intelligent transportation systems and embedding exploration, Ph.D. thesis, New Jersey Institute of Technology

  16. Liang X, Du X, Wang G, Han Z (2018) Deep reinforcement learning for traffic light control in vehicular networks. arXiv preprint arXiv:1803.11115

  17. Liang X, Yan T, Lee J, Wang G (2018) A distributed intersection management protocol for safety, efficiency, and driver’s comfort. IEEE Internet Things J 5(3):1924–1935

    Article  Google Scholar 

  18. Liang X, Du X, Wang G, Han Z (2019) A deep reinforcement learning network for traffic light cycle control. IEEE Trans Veh Technol 68(2):1243–1253

    Article  Google Scholar 

  19. Ma Z, Cui T, Deng W, Jiang F, Zhang L (2021) Adaptive optimization of traffic signal timing via deep reinforcement learning. J Adv Transp 2021

  20. Ma D, Zhou B, Song X, Dai H (2021) A deep reinforcement learning approach to traffic signal control with temporal traffic pattern mining. IEEE Trans Intell Transp Syst

  21. Maadi S, Stein S, Hong J, Murray-Smith R (2022) Real-time adaptive traffic signal control in a connected and automated vehicle environment: optimisation of signal planning with reinforcement learning under vehicle speed guidance. Sensors 22(19):7501

    Article  Google Scholar 

  22. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  23. Mousavi SS, Schukat M, Howley E (2017) Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning. IET Intel Transport Syst 11(7):417–423

  24. Navarro-Espinoza A, López-Bonilla OR, García-Guerrero EE, Tlelo-Cuautle E, López-Mancilla D, Hernández-Mejía C, Inzunza-González E (2022) Traffic flow prediction for smart traffic lights using machine learning algorithms. Technologies 10(1):5

  25. Pang H, Gao W (2019) Deep Deterministic policy gradient for traffic signal control of single intersection. In: 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China, pp 5861–5866. https://doi.org/10.1109/CCDC.2019.8832406

  26. Prosper HB (2017) Deep learning and Bayesian methods. EPJ Web Conf 137:9. https://doi.org/10.1051/epjconf/201713711007

  27. Raeisi M, Mahboob AS (2021) Intelligent control of urban intersection traffic light based on reinforcement learning algorithm. In: 2021 26th International Computer Conference, Computer Society of Iran (CSICC). IEEE, 1–5

  28. Rasheed F, Yau K-LA, Low Y-C (2020) Deep reinforcement learning for traffic signal control under disturbances: A case study on Sunway city, Malaysia. Futur Gener Comput Syst 109:431–445

    Article  Google Scholar 

  29. Sahu SP, Dewangan DK, Agrawal A, Priyanka TS (2021) Traffic light cycle control using deep reinforcement technique. In: 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS). IEEE, Coimbatore, India, pp 697–702. https://doi.org/10.1109/ICAIS50930.2021.9395880

  30. Saleem M, Abbas S, Ghazal TM, Khan MA, Sahawneh N, Ahmad M (2022) Smart cities: Fusion-based intelligent traffic congestion control system for vehicular networks using machine learning techniques. Egypt Inf J 23(3):417–426

    Google Scholar 

  31. Schneider C (2020) Intelligent signalized intersection management for mixed traffic using Deep Q-Learning. Not applicable

  32. Sutton RS, Barto AG et al (1998) Introduction to reinforcement learning, volume 135. MIT press Cambridge

  33. Tan KL, Poddar S, Sarkar S, Sharma A (2019) Deep reinforcement learning for adaptive traffic signal control. In: Dynamic Systems and Control Conference, volume 59162. American Society of Mechanical Engineers, V003T18A006

  34. Tong W, Hussain A, Bo WX, Maharjan S (2019) Artificial Intelligence for Vehicle-to-Everything: A Survey. IEEE Access 7:10823–10843. https://doi.org/10.1109/ACCESS.2019.2891073

    Article  Google Scholar 

  35. van der Pol E (2016) Deep reinforcement learning for coordination in traffic light control. Master’s thesis, University of Amsterdam

  36. Wei H, Zheng G, Yao H, Li Z (2018) Intellilight: A reinforcement learning approach for intelligent traffic light control. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2496–2505

  37. Wu T, Zhou P, Liu K, Yuan Y, Wang X, Huang H, Wu DO (2020) Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks. IEEE Trans Veh Technol 69(8):8243–8256

    Article  Google Scholar 

  38. Yu B, Guo J, Zhao Q, Li J, Rao W (2020) Smarter and safer traffic signal controlling via deep reinforcement learning. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 3345–3348

  39. Yuan X (2021) Faster Finding of Optimal Path in Robotics Playground Using Q-Learning with “Exploitation-Exploration Trade-Off”. J Phys Conf Ser 1748(2):022008

  40. Zeinaly Z, Sojoodi M, Bolouki S (2023) A resilient intelligent traffic signal control scheme for accident scenario at intersections via deep reinforcement learning. Sustainability 15(2):1329

    Article  Google Scholar 

  41. Zhang R, Ishikawa A, Wang W, Striner B, Tonguz OK (2020) Using reinforcement learning with partial vehicle detection for intelligent traffic signal control. IEEE Trans Intell Transp Syst 22(1):404–415

    Article  Google Scholar 

Download references

Acknowledgements

I am highly thankful to my co-author Mr. Nistala Venkata Kameshwer Sharma, for his important contribution. After that, I thank my supervisor, Dr. Vijay K Chaurasiya, for guiding me in this research work. At last, I am also thankful to Dr. Shishupal Kumar for his direction from time to time.

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology, Allahabad, India

    Ritesh Kumar, Nistala Venkata Kameshwer Sharma & Vijay K. Chaurasiya

Authors
  1. Ritesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nistala Venkata Kameshwer Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vijay K. Chaurasiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ritesh Kumar.

Ethics declarations

Conflicts of interest/Competing interests

No conflict.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, R., Sharma, N.V.K. & Chaurasiya, V.K. Adaptive traffic light control using deep reinforcement learning technique. Multimed Tools Appl 83, 13851–13872 (2024). https://doi.org/10.1007/s11042-023-16112-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16112-3

Keywords

Search

Navigation