Skip to main content
SpringerLink
Log in

End-to-end learning for off-road terrain navigation using the Chrono open-source simulation platform

  • Published:
Multibody System Dynamics Aims and scope Submit manuscript

Abstract

This contribution (\(i\)) describes an open-source, physics-based simulation infrastructure that can be used to learn and test control policies in off-road navigation; and (\(ii\)) demonstrates the use of the simulation platform in an end-to-end learning exercise that relies on simulated sensor data fusion (camera, GPS and IMU). For (\(i\)), the 0.5 million lines of open-source code support vehicle dynamics (wheeled/tracked vehicles, rovers), deformable & non-deformable terrains, and virtual sensing. The library has a Python API for interfacing with existing Machine Learning frameworks. For \((ii)\), we use a Gator off-road vehicle to demonstrate how a policy learned on non-deformable terrain performs when used in hilly conditions while navigating around a course of randomly placed obstacles on deformable terrain. The hilly terrain covers an 80×80 m patch and the soil can be controlled by the user to assume various behavior, e.g. non-deformable, deformable hard (silt-like), deformable soft (snow-like), etc. To the best of our knowledge, there is no other open-source, physics-based engine that can be used to simulate off-road mobility of autonomous agents operating on deformable terrains. The results reported herein can be reproduced with models and data available in a public repository (UW-Madison Simulation Based Engineering Laboratory, Supporting models, scripts, data, https://go.wisc.edu/arflqq, 2021). Animations associated with the tests run are available online (UW-Madison Simulation Based Engineering Laboratory, Supporting simulations, https://go.wisc.edu/256xb9, 2021).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. By “equivalent” we mean that the number-of-obstacles/surface ratio is the same; for terrain height difference, “equivalent” means that the maximum slope is the same.

References

  1. UW-Madison Simulation Based Engineering Laboratory: Supporting models, scripts, data. https://go.wisc.edu/arflqq (2021)

  2. UW-Madison Simulation Based Engineering Laboratory: Supporting simulations. https://go.wisc.edu/256xb9 (2021)

  3. Jakobi, N., Husbands, P., Harvey, I.: Noise and the reality gap: the use of simulation in evolutionary robotics. In: European Conference on Artificial Life, pp. 704–720. Springer, Berlin (1995)

    Google Scholar 

  4. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30. IEEE, Piscataway, NJ (2017)

    Chapter  Google Scholar 

  5. Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., Fox, D.: Closing the sim-to-real loop: adapting simulation randomization with real world experience. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8973–8979. IEEE, Piscataway, NJ (2019)

    Chapter  Google Scholar 

  6. Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., Zaremb, W.: Learning dexterous in-hand manipulation. Int. J. Robot. Res. 39(1), 3–20 (2020). https://doi.org/10.1177/0278364919887447

    Article  Google Scholar 

  7. Negrut, D., Serban, R., Elmquist, A., Taves, J., Young, A., Tasora, A., Benatti, S.: Enabling Artificial Intelligence studies in off-road mobility through physics-based simulation of multi-agent scenarios. In: NDIA Ground Vehicle Systems Engineering and Technology Symposium (2020)

    Google Scholar 

  8. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., et al.: End to end learning for self-driving cars (2016). arXiv:1604.07316

  9. Amini, A., Rosman, G., Karaman, S., Rus, D.: Variational End-to-End Navigation and Localization (2018). http://arxiv.org/abs/1811.1011

    Google Scholar 

  10. Koenig, N.P., Howard, A.: Design and use paradigms for Gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 4, pp. 2149–2154. IEEE, Design and use paradigms for Gazebo, an open-source multi-robot simulator (2004)

    Google Scholar 

  11. Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033. IEEE, Piscataway, NJ (2012)

    Chapter  Google Scholar 

  12. Matas, J., James, S., Davison, A.J.: Sim-to-Real Reinforcement Learning for Deformable Object Manipulation (2018). https://arxiv.org/abs/1806.07851

    Google Scholar 

  13. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning, pp. 1–16 (2017)

    Google Scholar 

  14. Shah, S., Dey, D., Lovett, C., Kapoor, A.: Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and Service Robotics, pp. 621–635. Springer, Berlin (2018)

    Chapter  Google Scholar 

  15. Rong, G., Shin, B.H., Tabatabaee, H., Lu, Q., Lemke, S., Možeiko, M., Boise, E., Uhm, G., Gerow, M., Mehta, S., Agafonov, E., Kim, T.H., Sterner, E., Ushiroda, K., Reyes, M., Zelenkovsky, D., Kim, S.: LGSVL simulator: a high fidelity simulator for autonomous driving. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6. IEEE, Piscataway, NJ (2020)

    Google Scholar 

  16. Espié, E., Guionneau, C., Wymann, B., Dimitrakakis, C.: TORCS – the Open Racing Car Simulator (2020). https://sourceforge.net/projects/torcs/

    Google Scholar 

  17. Epic Games: Unreal engine. https://www.unrealengine.com (2020). Accessed: 2021-11-23

  18. Unity3D: Main website. https://unity3d.com/ (2016). Accessed: 2021-11-23

  19. NVIDIA: PhysX simulation engine (2019). Available online at http://developer.nvidia.com/object/physx.html

  20. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing atari with deep reinforcement learning (2013). CoRR, arXiv:1312.5602

  21. Zhu, Y., Wang, Z., Merel, J., Rusu, A., Erez, T., Cabi, S., Tunyasuvunakool, S., Kramár, J., Hadsell, R., de Freitas, N., Heess, N.: Reinforcement and imitation learning for diverse visuomotor skills. In: Robotics: Science and Systems (2018)

    Google Scholar 

  22. Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies (2015). CoRR, arXiv:1504.00702

  23. You, Y., Pan, X., Wang, Z., Lu, C.: Virtual to real reinforcement learning for autonomous driving (2017). CoRR, arXiv:1704.03952

  24. Amini, A., Gilitschenski, I., Phillips, J., Moseyko, J., Banerjee, R., Karaman, S., Rus, D.: Learning robust control policies for end-to-end autonomous driving from data-driven simulation. IEEE Robot. Autom. Lett. 5(2), 1143–1150 (2020)

    Article  Google Scholar 

  25. Bohez, S., Verbelen, T., Coninck, E.D., Vankeirsbilck, B., Simoens, P., Dhoedt, B.: Sensor Fusion for Robot Control Through Deep Reinforcement Learning (2017). http://arxiv.org/abs/1703.04550

    Book  Google Scholar 

  26. Patel, N., Choromańska, A., Krishnamurthy, P., Khorrami, F.: Sensor modality fusion with CNNs for UGV autonomous driving in indoor environments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1531–1536 (2017)

    Google Scholar 

  27. Pan, Y., Cheng, C., Saigol, K., Lee, K., Yan, X., Theodorou, E.A., Boots, B.: Agile Autonomous Driving Using End-to-End Deep Imitation Learning (2017). http://arxiv.org/abs/1709.07174

    Google Scholar 

  28. Project Chrono Development Team: Chrono: an open source framework for the physics-based simulation of dynamic systems. https://github.com/projectchrono/chrono. Accessed: 2022-01-10

  29. Tasora, A., Serban, R., Mazhar, H., Pazouki, A., Melanz, D., Fleischmann, J., Taylor, M., Sugiyama, H., Negrut, D.: Chrono: an open source multi-physics dynamics engine. In: Kozubek, T. (ed.) High Performance Computing in Science and Engineering. Lecture Notes in Computer Science, pp. 19–49. Springer, Cham (2016)

    Chapter  Google Scholar 

  30. Serban, R., Taylor, M., Negrut, D., Tasora, A.: Chrono::Vehicle template-based ground vehicle modeling and simulation. Int. J. Veh. Perform. 5(1), 18–39 (2019)

    Article  Google Scholar 

  31. Tasora, A., Mangoni, D., Negrut, D., Serban, R., Jayakumar, P.: Deformable soil with adaptive level of detail for tracked and wheeled vehicles. Int. J. Veh. Perform. 5(1), 60–76 (2019)

    Article  Google Scholar 

  32. Hu, W., Rakhsha, M., Yang, L., Kamrin, K., Negrut, D.: Modeling granular material dynamics and its two-way coupling with moving solid bodies using a continuum representation and the SPH method. Comput. Methods Appl. Mech. Eng. 385, 114022 (2021). https://doi.org/10.1016/j.cma.2021.114022

    Article  MathSciNet  MATH  Google Scholar 

  33. Kelly, C., Olsen, N., Negrut, D.: Billion degree of freedom granular dynamics simulation on commodity hardware via heterogeneous data-type representation. Multibody Syst. Dyn. 50, 355–379 (2020)

    Article  MathSciNet  Google Scholar 

  34. Elmquist, A., Serban, R., Negrut, D.: A sensor simulation framework for training and testing robots and autonomous vehicles. ASME J. Auton. Veh. Syst. 1(2), 021001 (2021)

    Google Scholar 

  35. Goodin, C., Doude, M., Hudson, C., Carruth, D.: Enabling off-road autonomous navigation-simulation of lidar in dense vegetation. Electronics 7(9), 154 (2018)

    Article  Google Scholar 

  36. Tang, Z., von Gioi, R.G., Monasse, P., Morel, J-M.: A precision analysis of camera distortion models. IEEE Trans. Image Process. 26(6), 2694–2704 (2017)

    Article  MathSciNet  Google Scholar 

  37. Working group, EMVA 1288: Standard for characterization of image sensors and cameras. Release 3.0. Issued by European Machine Vision Association (November 2010)

  38. Beazley, D.M.: SWIG: an easy to use tool for integrating scripting languages with C and C++. In: Proc. 4th Conf. on USENIX Tcl/Tk Workshop, USA, vol. 4, p. 15 (1996)

    Google Scholar 

  39. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). CoRR, arXiv:1707.06347

  40. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in PyTorch. In: NIPS 2017 Workshop Autodiff (2017)

    Google Scholar 

  41. Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, pp. 41–48. Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1553374.1553380.

    Chapter  Google Scholar 

  42. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017). CoRR, arXiv:1412.6980 [cs.LG]

  43. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym (2016). CoRR, arXiv:1606.01540

  44. Dhariwal, P., Hesse, C., Klimov, O., Nichol, A., Plappert, M., Radford, A., Schulman, J., Sidor, S., Wu, Y., Zhokhov, P.: OpenAI baselines. https://github.com/openai/baselines

  45. Project Chrono: Chrono documentation and API reference. http://api.projectchrono.org/. Accessed: 2021-11-24

  46. Bekker, M.G.: Introduction to Terrain-Vehicle Systems. University of Michigan Press, Ann Arbor (1969)

    Google Scholar 

  47. Wong, J.Y.: Theory of Ground Vehicles, 4th edn. Wiley, New York (2008)

    Google Scholar 

  48. Janosi, Z., Hanamoto, B.: The analytical determination of drawbar pull as a function of slip for tracked vehicles in deformable soils. In: Proc of the 1st Int Conf Mech Soil–Vehicle Systems, Turin, Italy (1961)

    Google Scholar 

  49. Yarpiz: Path planning using PSO in MATLAB. https://www.mathworks.com/matlabcentral/fileexchange/53146-path-planning-using-pso-in-matlab. Accessed: 2020-06-17

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Wisconsin-Madison, Madison, 53706, WI, USA

    Simone Benatti, Aaron Young, Asher Elmquist, Jay Taves, Radu Serban & Dan Negrut

  2. Dipartimento di Ingegneria ed Architettura, Università di Parma, Parma, I-43100, Italy

    Alessandro Tasora

Authors
  1. Simone Benatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asher Elmquist
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jay Taves
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessandro Tasora
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Radu Serban
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dan Negrut
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radu Serban.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Benatti, S., Young, A., Elmquist, A. et al. End-to-end learning for off-road terrain navigation using the Chrono open-source simulation platform. Multibody Syst Dyn 54, 399–414 (2022). https://doi.org/10.1007/s11044-022-09816-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11044-022-09816-1

Keywords

Search

Navigation