Abstract
Online deep reinforcement learning training poses challenges due to its length and instability, despite the development of learning algorithms targeted to overcome these issues. Offline learning has emerged as a potential solution, but it reintroduces the issue of dataset production, which is resource-consuming and challenging even in simulation environments. This paper investigates efficient dataset creation for offline learning in the context of automated driving. Our proposed solution is a pipeline based on the CARLA simulator, which offers a wide variety in terms of car models, weather conditions, and environments. The pipeline aims to produce high-quality datasets for pre-training, training, and fine-tuning models, targeting improved training speed and reduced divergence. By leveraging CARLA’s level of realism, we address the resource-intensive nature of dataset production, providing researchers and car manufacturers with a valuable tool for advancing the development of robust automated driving systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, Y.: Deep Reinforcement Learning: An Overview. http://arxiv.org/abs/1701.07274 (2018). https://doi.org/10.48550/arXiv.1701.07274
Ghosh, D., Bellemare, M.G.: Representations for stable off-policy reinforcement learning. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, pp. 3556–3565. PMLR (2020)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms. http://arxiv.org/abs/1707.06347 (2017)
Dankwa, S., Zheng, W.: Twin-delayed DDPG: a deep reinforcement learning technique to model a continuous movement of an intelligent robot agent. In: Proceedings of the 3rd International Conference on Vision, Image and Signal Processing, pp. 1–5. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3387168.3387199
Lazzaroni, L., Bellotti, F., Capello, A., Cossu, M., De Gloria, A., Berta, R.: Deep reinforcement learning for automated car parking. In: Berta, R., De Gloria, A. (eds.) Applications in Electronics Pervading Industry, Environment and Society, pp. 125–130. Springer Nature Switzerland, Cham (2023). https://doi.org/10.1007/978-3-031-30333-3_16
Torabi, F., Warnell, G., Stone, P.: Behavioral Cloning from Observation. http://arxiv.org/abs/1805.01954 (2018). https://doi.org/10.48550/arXiv.1805.01954
Levine, S., Kumar, A., Tucker, G., Fu, J.: Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. http://arxiv.org/abs/2005.01643 (2020). https://doi.org/10.48550/arXiv.2005.01643
Agarwal, R., Schuurmans, D., Norouzi, M.: An optimistic perspective on offline reinforcement learning. In: Proceedings of the 37th International Conference on Machine Learning, pp. 104–114. PMLR (2020)
Fang, X., Zhang, Q., Gao, Y., Zhao, D.: Offline reinforcement learning for autonomous driving with real world driving data. In: 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 3417–3422 (2022). https://doi.org/10.1109/ITSC55140.2022.9922100
Kidambi, R., Rajeswaran, A., Netrapalli, P., Joachims, T.: MOReL: model-based offline reinforcement learning. In: Advances in Neural Information Processing Systems, pp. 21810–21823. Curran Associates, Inc. (2020)
Fu, J., Kumar, A., Nachum, O., Tucker, G., Levine, S.: D4RL: Datasets for Deep Data-Driven Reinforcement Learning. http://arxiv.org/abs/2004.07219 (2021)
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: An Open Urban Driving Simulator. http://arxiv.org/abs/1711.03938 (2017). https://doi.org/10.48550/arXiv.1711.03938
Cossu, M., Berta, R., Capello, A., De Gloria, A., Lazzaroni, L., Bellotti, F.: Developing a toolchain for synthetic driving scenario datasets. In: Berta, R., De Gloria, A. (eds.) Applications in Electronics Pervading Industry, Environment and Society. pp. 222–228. Springer Nature Switzerland, Cham (2023). https://doi.org/10.1007/978-3-031-30333-3_29
Motta, J., et al.: Developing a synthetic dataset for driving scenarios. In: Saponara, S., De Gloria, A. (eds.) Applications in Electronics Pervading Industry, Environment and Society, pp. 310–316. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-030-95498-7_43
Krajzewicz, D., Hertkorn, G., Feld, C., Wagner, P.: SUMO (Simulation of Urban mobility); An open-source traffic simulation (2002)
Xu, R., Guo, Y., Han, X., Xia, X., Xiang, H., Ma, J.: OpenCDA: an open cooperative driving automation framework integrated with co-simulation. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 1155–1162 (2021). https://doi.org/10.1109/ITSC48978.2021.9564825
Leurent, E.: An Environment for Autonomous Driving Decision-Making (2018). https://github.com/eleurent/highway-env
Pighetti, A., et al.: High-level decision-making non-player vehicles. In: Kiili, K., Antti, K., De Rosa, F., Dindar, M., Kickmeier-Rust, M., Bellotti, F. (eds.) Games and Learning Alliance, pp. 223–233. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-22124-8_22
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lazzaroni, L., Pighetti, A., Bellotti, F., Berta, R. (2024). Building a Pipeline for Efficient Production of Synthetic Datasets for Improving RL in Automated Driving. In: Ciofi, C., Limiti, E. (eds) Proceedings of SIE 2023. SIE 2023. Lecture Notes in Electrical Engineering, vol 1113. Springer, Cham. https://doi.org/10.1007/978-3-031-48711-8_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-48711-8_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48710-1
Online ISBN: 978-3-031-48711-8
eBook Packages: EngineeringEngineering (R0)