Abstract
In this paper we present an application of Deep Reinforcement Learning to lane-free traffic, where vehicles do not adhere to the notion of lanes, but are rather able to be located at any lateral position within the road boundaries. This constitutes an entirely different problem domain for autonomous driving compared to lane-based traffic, as vehicles consider the actual two dimensional space available, and their decision making needs to adapt to this concept. We also consider that each vehicle wishes to maintain a (different) desired speed, therefore creating many situations where vehicles need to perform overtaking, and react appropriately to the behaviour of others. As such, in this work, we design a Reinforcement Learning agent for the problem at hand, considering different components of reward functions tied to the environment at various levels of information. Finally, we examine the effectiveness of our approach using the Deep Deterministic Policy Gradient algorithm.
Keywords
- Deep reinforcement learning
- Lane-free traffic
- Autonomous driving
The research leading to these results has received funding from the European Research Council under the European Union’s Horizon 2020 Research and Innovation programme/ ERC Grant Agreement n. [833915], project TrafficFluid.
This is a preview of subscription content, access via your institution.
Buying options
Notes
- 1.
Videos showcasing a trained agent with ‘All-Components Reward Function’ can be found at: https://bit.ly/3O0LjJW.
References
Aradi, S.: Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans. Intell. Transp. Syst. 23(2), 740–759 (2022)
Badia, A.P., et al.: Agent57: outperforming the Atari human benchmark. In: Proceedings of the 37th International Conference on Machine Learning, PMLR, vol. 119, pp. 507–517 (2020)
Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75538-8_7
Di, X., Shi, R.: A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to AI-guided driving policy learning. Transp. Res. Part C: Emerg. Technol. 125(4), 103008 (2021)
Gu, S., Lillicrap, T., Sutskever, I., Levine, S.: Continuous deep Q-learning with model-based acceleration. In: Proceedings of The 33rd International Conference on Machine Learning, PMLR, vol. 48, pp. 2829–2838 (2016)
Karafyllis, I., Theodosis, D., Papageorgiou, M.: Two-dimensional cruise control of autonomous vehicles on lane-free roads. In: 60th IEEE conference on Decision and Control (CDC2021), pp. 2683–2689 (2021)
Kendall, A., et al.: Learning to drive in a day. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8248–8254 (2019)
Kiran, B.R., et al.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. PP(99), 1–18 (2021)
Li, C., Czarnecki, K.: Urban driving with multi-objective deep reinforcement learning. In: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, pp. 359–367. AAMAS 2019 (2019)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. In: Proceedings of the 4th International Conference on Learning Representations, ICLR (2016)
Mnih, V., et al.: Playing atari with deep reinforcement learning (2013)
Papageorgiou, M., Mountakis, K.S., Karafyllis, I., Papamichail, I., Wang, Y.: Lane-free artificial-fluid concept for vehicular traffic. Proc. IEEE 109(2), 114–121 (2021)
Plappert, M.: keras-RL. https://github.com/keras-rl/keras-rl (2016)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Troullinos, D., Chalkiadakis, G., Papamichail, I., Papageorgiou, M.: Collaborative multiagent decision making for lane-free autonomous driving. In: Proceedings of AAMAS-2020, pp. 1335–1343 (2021)
Wu, C., Kreidieh, A.R., Parvate, K., Vinitsky, E., Bayen, A.M.: Flow: a modular learning framework for mixed autonomy traffic. IEEE Trans. Robot. 38, 1–17 (2021)
Yanumula, V.K., Typaldos, P., Troullinos, D., Malekzadeh, M., Papamichail, I., Papageorgiou, M.: Optimal path planning for connected and automated vehicles in lane-free traffic. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 3545–3552 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Karalakou, A., Troullinos, D., Chalkiadakis, G., Papageorgiou, M. (2022). Deep RL Reward Function Design for Lane-Free Autonomous Driving. In: Dignum, F., Mathieu, P., Corchado, J.M., De La Prieta, F. (eds) Advances in Practical Applications of Agents, Multi-Agent Systems, and Complex Systems Simulation. The PAAMS Collection. PAAMS 2022. Lecture Notes in Computer Science(), vol 13616. Springer, Cham. https://doi.org/10.1007/978-3-031-18192-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-18192-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18191-7
Online ISBN: 978-3-031-18192-4
eBook Packages: Computer ScienceComputer Science (R0)