Skip to main content

Learning to Drive Fast on a DuckieTown Highway

  • Conference paper
  • First Online:
Intelligent Autonomous Systems 16 (IAS 2021)

Abstract

We train a small Nvidia AI JetRacer to follow the road on a small DuckieTown highway. In the real-world, roads do not always have the same appearance, so the system should not be trained on lane markings alone but on the complete view of the front camera. To make this possible, the system is trained in simulation using a recent reinforcement learning approach in an end-to-end fashion. This driving experience is then transferred to the circumstances encountered on a real track. Transfer learning is surprisingly successful, although this method is very sensitive to the details of the vehicle dynamics. We trained multiple models at different speeds and evaluated their performance both in simulation and in the real world. Increasing the velocity proves difficult, as the learned policy breaks down at higher speeds. The result is a small Nvidia AI JetRacer, which is able to drive around a DuckieTown highway, based on simulated experiences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Supported by the Meaningful Control of Autonomous Systems initiative from TNO, CWI and UvA.

  2. 2.

    https://www.waveshare.com/jetracer-ai-kit.htm.

  3. 3.

    https://www.duckietown.org/about/hardware.

  4. 4.

    https://gym.openai.com/.

  5. 5.

    Driving on the Zigzag map used for training: https://youtu.be/7cju-CylTHQ.

  6. 6.

    Driving on the unseen Udem1 map: https://youtu.be/7xWd9aIqzmg.

  7. 7.

    Driving in the real-world Duckietown Highway: https://youtu.be/hi1qfQrnXq4.

References

  1. Almási, P., Moni, R., Gyires-Tóth, B.: Robust reinforcement learning-based autonomous driving agent for simulation and real world. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2020). https://doi.org/10.1109/IJCNN48605.2020.9207497

  2. Anderson, M.: The road ahead for self-driving cars: the AV industry has had to reset expectations, as it shifts its focus to level 4 autonomy - [News]. IEEE Spectr. 57(5), 8–9 (2020). https://doi.org/10.1109/MSPEC.2020.9078402

    Article  Google Scholar 

  3. Bojarski, M., et al.: End to end learning for self-driving cars. preprint arXiv:1604.07316 (2016)

  4. Broggi, A., Zelinsky, A., Özgüner, Ü., Laugier, C.: Intelligent Vehicles. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1627–1656. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32552-1_62

    Chapter  Google Scholar 

  5. Brooks, R.: Robotic cars won’t understand us, and we won’t cut them much slack. IEEE Spectr. 54(8), 34–51 (2017). https://doi.org/10.1109/MSPEC.2017.8000288

    Article  Google Scholar 

  6. Chevalier-Boisvert, M., Golemo, F., Cao, Y., Mehta, B., Paull, L.: Duckietown environments for openai gym. https://github.com/duckietown/gym-duckietown (2018)

  7. Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A survey of deep learning techniques for autonomous driving. J. Field Robot. 37(3), 362–386 (2020). https://doi.org/10.1002/rob.21918

    Article  Google Scholar 

  8. Holen, M., Saha, R., Goodwin, M., Omlin, C.W., Sandsmark, K.E.: Road detection for reinforcement learning based autonomous car. In: Proceedings of the 2020 The 3rd International Conference on Information Science and System, ACM, March 2020. https://doi.org/10.1145/3388176.3388199

  9. Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning. ACM Comput. Surv. 50(2), 1–35 (2017). https://doi.org/10.1145/3054912

  10. Jaritz, M., de Charette, R., Toromanoff, M., Perot, E., Nashashibi, F.: End-to-end race driving with deep reinforcement learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2018. https://doi.org/10.1109/icra.2018.8460934

  11. Kiran, B.R., et al.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. 1–18 (2021). https://doi.org/10.1109/tits.2021.3054625

  12. LeCun, Y., Muller, U., Ben, J., Cosatto, E., Flepp, B.: Off-road obstacle avoidance through end-to-end learning. In: Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference, pp. 739–746. Advances in Neural Information Processing Systems (2005)

    Google Scholar 

  13. Ma, X.: Car racing with pytorch. https://github.com/xtma/pytorch_car_caring (2019)

  14. Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: Balcan, M.F., Weinberger, K.Q. (eds.) Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 48, pp. 1928–1937. PMLR, New York, 20–22 June 2016. http://proceedings.mlr.press/v48/mniha16.html

  15. Osinski, B., et al.: Simulation-based reinforcement learning for real-world autonomous driving. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2020. https://doi.org/10.1109/icra40945.2020.9196730

  16. Pan, X., You, Y., Wang, Z., Lu, C.: Virtual to real reinforcement learning for autonomous driving. In: Proceedings of the British Machine Vision Conference 2017. British Machine Vision Association (2017). https://doi.org/10.5244/c.31.11

  17. Paull, L., et al.: Duckietown: an open, inexpensive and flexible platform for autonomy education and research. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2017. https://doi.org/10.1109/icra.2017.7989179

  18. Pomerleau, D.A.: Alvinn: an autonomous land vehicle in a neural network. In: Proceedings of the 1st International Conference on Neural Information Processing Systems, NIPS 1988, pp. 305–313. MIT Press, Cambridge, MA, USA (1988)

    Google Scholar 

  19. Sazanovich, M., Chaika, K., Krinkin, K., Shpilman, A.: Imitation learning approach for AI driving Olympics trained on real-world and simulation data simultaneously (2020). arXiv:2007.03514

  20. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv:1707.06347

  21. Shalev-Shwartz, S., Shammah, S., Shashua, A.: On a formal model of safe and scalable self-driving cars. CoRR abs/1708.06374 (2017). http://arxiv.org/abs/1708.06374

  22. Sharma, S., Lakshminarayanan, A.S., Ravindran, B.: Learning to repeat: fine grained action repetition for deep reinforcement learning (2017). arXiv:1702.06054

  23. Zilly, J., et al.: The AI driving Olympics at NeurIPS 2018. In: Escalera, S., Herbrich, R. (eds.) The NeurIPS ’18 Competition. TSSCML, pp. 37–68. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-29135-8_3

    Chapter  Google Scholar 

Download references

Acknowledgments

Many thanks to Douwe van der Wal and Thomas van Orden for the interesting discussions and helpful suggestions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas P. A. Wiggers .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wiggers, T.P.A., Visser, A. (2022). Learning to Drive Fast on a DuckieTown Highway. In: Ang Jr, M.H., Asama, H., Lin, W., Foong, S. (eds) Intelligent Autonomous Systems 16. IAS 2021. Lecture Notes in Networks and Systems, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-95892-3_14

Download citation

Publish with us

Policies and ethics