Skip to main content
SpringerLink
Log in

Mitigating congestion in multi-agent traffic signal control: an efficient self-attention proximal policy optimization approach

  • Original Research
  • Published:
International Journal of Information Technology Aims and scope Submit manuscript

Abstract

Traffic congestion is a persistent problem that effects cities worldwide, necessitating innovative solutions. This paper presents a novel traffic light control system using multi-agent proximal policy optimization with self-attention. Our approach outperforms traditional methods by 30% in reducing waiting times in high-traffic demand scenarios. By utilizing transfer learning and encoding mechanisms for dynamic input size adaptation, our approach enables scalability to larger networks without the need for costly training. This study underscores the potential of our approach as a dependable solution for addressing large-scale traffic congestion challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The dataset used in this work was generated using SUMO and can be found in this link: https://github.com/cherouss/SAMAPPO/data

Code availability

The code of this work can be found in this repository: https://github.com/cherouss/SAMAPPO.

References

  1. Albino V, Berardi U, Dangelico RM (2015) Smart cities: definitions, dimensions, performance, and initiatives. J Urban Technol 22:3–21

    Article  Google Scholar 

  2. Schrank D, Albert L, Eisele B, Lomax T (2021) Urban Mobility Report. Texas A&M Transportation Institute, College Station

  3. Christidis P, Rivas NI (2012) Measuring road congestion. Institute for Prospective Technological Studies, European Commission Joint Research Centre

  4. Higgins CD, Sweet MN, Kanaroglou PS (2018) All minutes are not equal: travel time and the effects of congestion on commute satisfaction in Canadian cities. Transportation 45:1249–1268. https://doi.org/10.1007/s11116-017-9766-2

    Article  Google Scholar 

  5. Fattah MdA, Morshed SR, Kafy A-A (2022) Insights into the socio-economic impacts of traffic congestion in the port and industrial areas of Chittagong city. Bangladesh Transport Eng 9:100122. https://doi.org/10.1016/j.treng.2022.100122

    Article  Google Scholar 

  6. ChengAaron Z, Pang M-S, Pavlou PA (2020) Mitigating traffic congestion: the role of intelligent transportation systems. Inf Syst Res 31:653–674. https://doi.org/10.1287/isre.2019.0894

    Article  Google Scholar 

  7. Agarwal V, Sharma S (2023) DQN Algorithm for network resource management in vehicular communication network. Int J Inf Technol. https://doi.org/10.1007/s41870-023-01399-0

    Article  Google Scholar 

  8. Yadav R, Dahiya PK, Mishra R (2020) Comparative analysis of automotive radar sensor for collision detection and warning system. Int J Inf Technol 12:289–294. https://doi.org/10.1007/s41870-018-0167-3

    Article  Google Scholar 

  9. Mohapatra H, Rath AK, Panda N (2022) IoT infrastructure for the accident avoidance: an approach of smart transportation. Int J Inf Technol 14:761–768. https://doi.org/10.1007/s41870-022-00872-6

    Article  Google Scholar 

  10. Sharma R, Singh U (2021) Fuzzy based energy efficient clustering for designing WSN-based smart parking systems. Int J Inf Technol 13:2381–2387. https://doi.org/10.1007/s41870-021-00789-6

    Article  Google Scholar 

  11. Wang Y, Yang X, Liang H, Liu Y (2018) A review of the self-adaptive traffic signal control system based on future traffic environment. J Adv Transp 2018:e1096123. https://doi.org/10.1155/2018/1096123

    Article  Google Scholar 

  12. Wiering MA, Veenen J van, Vreeken J, Koopman A (2004) Intelligent traffic light control. In: Utrecht University: Information and Computing Science

  13. Arel I, Liu C, Urbanik T, Kohls AG (2010) Reinforcement learning-based multi-agent system for network traffic signal control. IET Intell Transp Syst 4:128. https://doi.org/10.1049/iet-its.2009.0070

    Article  Google Scholar 

  14. Abdoos M, Mozayani N, Bazzan ALC (2011) Traffic light control in non-stationary environments based on multi agent Q-learning. In: 2011 14th International IEEE conference on ıntelligent transportation systems (ITSC). IEEE, Washington, pp 1580–1585

  15. Mousavi SS, Schukat M, Howley E (2017) Traffic light control using deep policy-gradient and value-function-based reinforcement learning. IET Intel Transport Syst 11:417–423. https://doi.org/10.1049/iet-its.2017.0153

    Article  Google Scholar 

  16. Calvo JA, Dusparic I (2018) Heterogeneous Multi-Agent Deep Reinforcement Learning for Traffic Lights Control. In: Proceedings for the 26th Irish Conference on Artificial Intelligence and Cognitive Science (AICS), vol 2259, pp 2–13

  17. Chu T, Wang J, Codeca L, Li Z (2020) Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Trans Intell Transport Syst 21:1086–1095. https://doi.org/10.1109/TITS.2019.2901791

    Article  Google Scholar 

  18. Chen C, Wei H, Xu N et al (2020) Toward a thousand lights: decentralized deep reinforcement learning for large-scale traffic signal control. Proc AAAI Conf Artif Intell 34:3414–3421. https://doi.org/10.1609/aaai.v34i04.5744

    Article  Google Scholar 

  19. Yang J, Zhang J, Wang H (2021) Urban traffic control in software defined internet of things via a multi-agent deep reinforcement learning approach. IEEE Trans Intell Transport Syst 22:3742–3754. https://doi.org/10.1109/TITS.2020.3023788

    Article  Google Scholar 

  20. Wang Y, Xu T, Niu X et al (2022) STMARL: a spatio-temporal multi-agent reinforcement learning approach for cooperative traffic light control. IEEE Trans Mobile Comput 21:2228–2242. https://doi.org/10.1109/TMC.2020.3033782

    Article  Google Scholar 

  21. Li Z, Yu H, Zhang G et al (2021) Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning. Transport Res Part C: Emerg Technol 125:103059. https://doi.org/10.1016/j.trc.2021.103059

    Article  Google Scholar 

  22. Shijie W, Shangbo W (2023) A novel multi-agent deep RL approach for traffic signal control. In: 2023 IEEE ınternational conference on pervasive computing and communications workshops and other affiliated events (PerCom Workshops), pp 15–20

  23. Wang T, Cao J, Hussain A (2021) Adaptive traffic signal control for large-scale scenario with cooperative group-based multi-agent reinforcement learning. Transport Res Part C: Emerg Technol 125:103046. https://doi.org/10.1016/j.trc.2021.103046

    Article  Google Scholar 

  24. Mo Z, Li W, Fu Y et al (2022) CVLight: decentralized learning for adaptive traffic signal control with connected vehicles. Transport Res Part C: Emerg Technol 141:103728. https://doi.org/10.1016/j.trc.2022.103728

    Article  Google Scholar 

  25. Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction, Second. The MIT Press

  26. Mnih V, Badia AP, Mirza M et al (2016) Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33rd ınternational conference on ınternational conference on machine learning—volume 48. JMLR.org, New York, pp 1928–1937

  27. Schulman J, Levine S, Abbeel P et al (2015) Trust region policy optimization. In: Proceedings of the 32nd ınternational conference on machine learning. PMLR, pp 1889–1897

  28. Schulman J, Wolski F, Dhariwal P, et al (2017) Proximal Policy Optimization Algorithms. CoRR abs/1707.06347:

  29. Dhrisya K, Remya G, Mohan A (2020) Fine-grained entity type classification using GRU with self-attention. Int j inf tecnol 12:869–878. https://doi.org/10.1007/s41870-020-00499-5

    Article  Google Scholar 

  30. Shaw P, Uszkoreit J, Vaswani A (2018) Self-attention with relative position representations. In: Proceedings of the 2018 Conference of the North American chapter of the association for computational linguistics: human language technologies, volume 2 (Short Papers). Association for Computational Linguistics, New Orleans, Louisiana, pp 464–468

  31. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Advances in neural ınformation processing systems. Curran Associates, New York

  32. Alegre LN (2019) SUMO-RL, https://github.com/LucasAlegre/sumo-rl

  33. Wei H, Chen C, Zheng G et al (2019) PressLight: learning max pressure control to coordinate traffic signals in arterial network. In: Proceedings of the 25th ACM SIGKDD ınternational conference on knowledge discovery & data mining. ACM, Anchorage AK USA, pp 1290–1298

Download references

Funding

The authors declare that no funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Laboratory of Signals and Systems Analysis, Faculty of Technology, University of M′sila, PO Box 166 Ichebilia, 28000, M′sila, Algeria

    Oussama Chergui

  2. Faculty of Mathematics and Computer Science, University of M′sila, PO Box 166 Ichebilia, 28000, M′sila, Algeria

    Oussama Chergui

  3. Laboratory of Informatics and its Applications of M′sila (LIAM), Faculty of Mathematics and Computer Science, University of M′sila, PO Box 166 Ichebilia, 28000, M′sila, Algeria

    Lamri Sayad

Authors
  1. Oussama Chergui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lamri Sayad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oussama Chergui.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chergui, O., Sayad, L. Mitigating congestion in multi-agent traffic signal control: an efficient self-attention proximal policy optimization approach. Int. j. inf. tecnol. 16, 2273–2282 (2024). https://doi.org/10.1007/s41870-023-01545-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41870-023-01545-8

Keywords

Search

Navigation