Abstract
Autonomous driving is one of the most challenging problems of the last decades. The development in recent years is mainly due to the continuous expansion of Artificial Intelligence. Nowadays, most self-driving systems use Deep Learning techniques. In recent years, however, thanks to the successful learning demonstrations of Atari games and AlphaGo by Google DeepMind, new frameworks based on Deep Reinforcement Learning are being developed. The objective is to combine the advantages of image processing and feature extraction of convolutional networks, and the learning process through the interaction of one or multiple agents with their environment. This work aims to deepen and explore these new methodologies applied to autonomous driving cars. In particular, we developed a framework for controlling a car in a simulated environment. The agent learns to drive within a neighborhood with constant speed, variable light conditions, and avoiding collisions with external objects. The proposed techniques are based on Double Deep Q-learning and Dueling Double Deep Q-learning. We implemented two variants of the algorithms: one trained from random weights and one exploiting the concepts of Transfer Learning. After a simulation study, the Dueling Double Deep Q-learning with Transfer Learning has showed promising performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bansal, M., Krizhevsky, A., Ogale, A.: ChauffeurNet: learning to drive by imitating the best and synthesizing the worst. arXiv:1812.03079, pp. 1–20 (2018)
Kiran, B.-R., et al.: Deep reinforcement learning for autonomous driving: a survey. arXiv:2002.00444, pp. 1–18 (2020)
Lillicrap, T., et al.: Continuous control with deep reinforcement learning. arXiv:1509.02971, pp. 1–14 (2015)
Lowd, D., Meek, C.: Adversarial learning. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, KDD, pp. 641–647. ACM, Beijing China (2005)
Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv:1312.5602, pp. 1–9 (2013)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.-A., Veness, J., Bellemare, M.-G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Ngai, D.-C.-K., Yung, N.-H.-C.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. 12(2), 509–522 (2011)
Sallab, S.-E.-L., Abdou, M., Perot, E., Yogamani, S.: Deep reinforcement learning framework for autonomous driving. Electron. Imaging 2017(19), 70–76 (2017)
Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: high-fidelity visual and physical Simulation for autonomous vehicles. arXiv:1705.05065, pp. 1–14 (2017)
Spryn, M., Sharma, A., Parkar, D., Shrimal, M.: Distributed deep reinforcement learning on the cloud for autonomous driving. In: 2018 IEEE/ACM Proceedings of the 1\(^{st}\) International Workshop on Software Engineering for AI in Autonomous Systems, SEFAIAS, pp. 16–22. IEEE, Gothenburg, Sweden (2018)
SullyChen Dataset: driving dataset. https://github.com/SullyChen/driving-datasets. Accessed 19 Jan 2021
Sutton, R.-S-., Barto, A.-G.: Reinforcement Learning: An Introduction. The MIT Press, Massachusetts (2018)
Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. https://www.sae.org/standards/. Accessed 19 Jan 2021
Tesla Autopilot: Future of Driving. https://www.tesla.com/autopilot. Accessed 19 Jan 2021
Torcs, the open racing car simulator. http://torcs.sourceforge.net. Accessed 19 Jan 2021
van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. arXiv:1509.06461, pp. 1–13 (2015)
Wang, P., Chan, C.-Y., de La Fortelle, A.: A reinforcement learning based approach for automated lane change maneuvers. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 1379–1384. IEEE, Changshu, China (2018). https://doi.org/10.1109/IVS.2018.8500556
Wang, S., Jia, D., Weng, X.: Deep reinforcement learning for autonomous driving. arXiv:1811.11329, pp. 1–9 (2018)
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., Freitas, N.: Dueling network architectures for deep reinforcement learning. In: Proceedings of the 33\(^{rd}\) International Conference on Machine Learning, PMLR, pp. 1–9, New York, USA, JMLR (2016)
Acknowledgements
We greatly acknowledge the DEMS Data Science Lab for supporting this work by providing computational resources.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Riboni, A., Candelieri, A., Borrotti, M. (2022). Deep Autonomous Agents Comparison for Self-driving Cars. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2021. Lecture Notes in Computer Science(), vol 13163. Springer, Cham. https://doi.org/10.1007/978-3-030-95467-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-95467-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95466-6
Online ISBN: 978-3-030-95467-3
eBook Packages: Computer ScienceComputer Science (R0)