Hardware in the loop framework proposal for a semi-autonomous car architecture in a closed route environment

Abstract

The development of intelligent vehicles has been increasing at great speed in recent years, which has allowed to improve their capabilities in autonomous driving systems. Many of these features are related to advanced driving assistance systems and autonomous driving systems. This capability improvement, has been achieved because of recent developments of automation oriented software and hardware. Such improvements, allowed the vehicle to achieve a more precise perception of it’s working environment. For this improvement it is important the integration and simulation of the systems in different configurations, such as hardware-in-the-loop, software-in-the-loop and model-in-the-loop. In this paper, we present a framework proposal that allows the design and testing of computer vision and control systems for the partial automation of a vehicle, with the use of hardware and software systems in the loop. This proposal is focused on the rapid experimental development of these systems for the implementation by autonomous vehicle designers and engineers. Our proposal allows faster data capture, either in a real or simulated environment, to improve and optimize data training with machine learning algorithms; this proposal integrates several open source systems and hardware with the necessary capacity for real-time implementation.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. 1.

    Ahamed, M.F.S., Tewolde, G.S., Kwon, J.: Software-in-the-loop modeling and simulation framework for autonomous vehicles. In: 2018 IEEE International Conference on Electro/Information Technology (EIT), pp. 0305–0310 (2018)

  2. 2.

    ANYVERSE: Machine Learning Synthetic Database for ADAS and Autonomous Robotics. https://anyverse.ai/ (2019). Accessed 27 June 2019

  3. 3.

    APOLLO: Open Platform. http://apollo.auto/ (2019). Accessed 27 June 2019

  4. 4.

    Automotive, I.: ADAS and Automated Driving. https://ipg-automotive.com/areas-of-application/adas-automated-driving/ (2019). Accessed 27 June 2019

  5. 5.

    Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, D., L., Monfort, M., Muller, U., Zhang, J., Xin, Z., Zhao, J., Zieba, K.: End to End Learning for Self-driving Cars, pp. 1–9 (2016). arxiv:1604.07316

  6. 6.

    Bonsai.ai: Unlocking the Power of Deep Reinforcement Learning for Industrial Systems. https://www.bons.ai/ (2019). Accessed 27 June 2019

  7. 7.

    Bounini, F., Gingras, D., Lapointe, V., Pollart, H.: Autonomous vehicle and real time road lanes detection and tracking. In: 2015 IEEE Vehicle Power and Propulsion Conference (VPPC), pp. 1–6 (2015). https://doi.org/10.1109/VPPC.2015.7352903

  8. 8.

    Brown, M.: Waymo vs. tesla: who will win the self-driving car race? Inverse p. 1. https://www.inverse.com/article/50456-waymo-vs-tesla-who-will-win-the-self-driving-car-race (2018). Accessed 27 June 2019

  9. 9.

    CARLA: Open-Source Simulator for Autonomous Driving Research. http://carla.org/ (2019). Accessed 27 June 2019

  10. 10.

    Castaño, F., Beruvides, G., Haber, R.E., Artuñedo, A.: Obstacle recognition based on machine learning for on-chip LiDAR sensors in a cyber-physical system. Sensors (2017). https://doi.org/10.3390/s17092109

    Article  Google Scholar 

  11. 11.

    Chakraborty, S., Al Faruque, M.A., Chang, W., Goswami, D., Wolf, M., Zhu, Q.: Automotive cyber-physical systems: a tutorial introduction. IEEE Des. Test 33(4), 92–108 (2016). https://doi.org/10.1109/MDAT.2016.2573598

    Article  Google Scholar 

  12. 12.

    Chen, J., Yuan, B., Tomizuka, M.: Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety (2019) arXiv:1903.00640

  13. 13.

    Codevilla, F., Miiller, M., López, A., Koltun, V., Dosovitskiy, A.: End-to-end driving via conditional imitation learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–9 (2018)

  14. 14.

    Curiel-Ramirez, L., Ramirez-Mendoza, R., Carrera, G., Izquierdo-Reyes, J., Bustamante-Bello, M.: Towards of a modular framework for semi-autonomous driving assistance systems. Int. J. Interact. Des. Manuf.: IJIDeM (2018). https://doi.org/10.1007/s12008-018-0465-9

    Article  Google Scholar 

  15. 15.

    Damelin, S., Miller, W.: The Mathematics of Signal Processing. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  16. 16.

    DeBord, M.: A Waymo engineer told us why a virtual-world simulation is crucial to the future of self-driving cars. Business Insider, p. 1. https://www.businessinsider.com/waymo-engineer-explains-why-testing-self-driving-cars-virtually-is-critical-2018-8 (2018). Accessed 27 June 2019

  17. 17.

    Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning, pp. 1–16 (2017)

  18. 18.

    dSpace: Adas and Autonomous Driving. https://www.dspace.com/en/inc/home/applicationfields/our_solutions_for/driver_assistance_systems.cfm (2019). Accessed 27 June 2019

  19. 19.

    Eraqi, H.M., Moustafa, M.N., Honer, J.: End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies (2017). CoRR arxiv:1710.03804

  20. 20.

    Escobar, L., Carvajal, N., Naranjo, J., Ibarra, A., Villacís, C., Zambrano, M., Galárraga, F.: Design and implementation of complex systems using mechatronics and cyber-physical systems approaches. In: 2017 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 147–154 (2017). https://doi.org/10.1109/ICMA.2017.8015804

  21. 21.

    Godoy, J., Haber Guerra, R.E., Muñoz, J.J., Matía, F., García, Á.: Smart Sensing of Pavement Temperature Based on Low-Cost Sensors and V2I Communications. http://digital.csic.es/handle/10261/167928 (2018). Accessed 27 June 2019

  22. 22.

    Goswami, D., Schneider, R., Masrur, A., Lukasiewycz, M., Chakraborty, S., Voit, H., Annaswamy, A.: Challenges in automotive cyber-physical systems design. In: 2012 International Conference on Embedded Computer Systems (SAMOS), pp. 346–354 (2012). https://doi.org/10.1109/SAMOS.2012.6404199

  23. 23.

    Grazioli, F., Kusmenko, E., Roth, A., Rumpe, B., von Wenckstern, M.: Simulation framework for executing component and connector models of self-driving vehicles. In: MODELS (2017)

  24. 24.

    Guérineau, B., Bricogne, M., Durupt, A., Rivest, L.: Mechatronics vs. cyber physical systems: towards a conceptual framework for a suitable design methodology. In: 2016 11th France-Japan 9th Europe-Asia Congress on Mechatronics (MECATRONICS) /17th International Conference on Research and Education in Mechatronics (REM), pp. 314–320 (2016). https://doi.org/10.1109/MECATRONICS.2016.7547161

  25. 25.

    Han, S., Kang, J., Jo, Y., Lee, D., Choi, J.: Robust ego-motion estimation and map matching technique for autonomous vehicle localization with high definition digital map. In: 2018 International Conference on Information and Communication Technology Convergence (ICTC), pp. 630–635 (2018). https://doi.org/10.1109/ICTC.2018.8539518

  26. 26.

    Harrison, R., Vera, D., Ahmad, B.: Engineering methods and tools for cyber-physical automation systems. Proc. IEEE 104(5), 973–985 (2016). https://doi.org/10.1109/JPROC.2015.2510665

    Article  Google Scholar 

  27. 27.

    Izquierdo-Reyes, J., Ramirez-Mendoza, R., Bustamante-Bello, M.: Advanced driver monitoring for assistance system (ADMAS). Int. J. Interact. Des. Manuf: IJIDeM (2016). https://doi.org/10.1007/s12008-016-0349-9

    Article  Google Scholar 

  28. 28.

    Kyutoku, H., Kawanishi, Y., Deguchi, D., Ide, I., Kato, K., Murase, H.: Estimating the scene-wise reliability of lidar pedestrian detectors. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 3511–3516 (2018). https://doi.org/10.1109/ITSC.2018.8569994

  29. 29.

    Lattarulo, R., Pérez, J., Dendaluce, M.: A complete framework for developing and testing automated driving controllers. In: ELSEVIER 20th IFAC World Congress, pp. 258–263 (2017). https://doi.org/10.1016/j.ifacol.2017.08.043

    Article  Google Scholar 

  30. 30.

    MathWorld, W.: Convolution. http://mathworld.wolfram.com/Convolution.html (2019). Accessed 27 June 2019

  31. 31.

    Meadows, E.S., Rawlings, J.B.: Nonlinear Process Control, chap. Model Predictive Control. University of South Carolina, USA (2005)

  32. 32.

    Metamoto: Simulation as a Service, is Now Available. https://www.metamoto.com/ (2019)

  33. 33.

    Michaels, L., Pagerit, S., Rousseau, A., Sharer, P., Halbach, S., Vijayagopal, R., Kropinski, M., Matthews, G., Kao, M., Matthews, O., Steele, M., Will, A.: Model-based systems engineering and control system development via virtual hardware-in-the-loop simulation. In: SAE Technical Paper. SAE International. https://doi.org/10.4271/2010-01-2325 (2010). Accessed 27 June 2019

  34. 34.

    Microsoft: Airsim. https://github.com/Microsoft/AirSim (2019). Accessed 27 June 2019

  35. 35.

    NVIDIA: Drive agx. https://www.nvidia.com/en-us/self-driving-cars/drive-platform/hardware/ (2019) Accessed 27 June 2019

  36. 36.

    NVIDIA: Drive Constellation. https://www.nvidia.com/en-us/self-driving-cars/drive-constellation/ (2019)

  37. 37.

    Rawat, D.B., Bajracharya, C.: Vehicular Cyber Physical Systems: Adaptive Connectivity and Security, 1st edn. Springer, Berlin (2016)

    Google Scholar 

  38. 38.

    ROS: Robot Operative System. http://www.ros.org/ (2019)

  39. 39.

    Skymind.ai: A Beginner’s Guide to Convolutional Neural Networks (cnns). https://skymind.ai/wiki/convolutional-network (2019). Accessed 27 June 2019

  40. 40.

    Udacity: Self Driving Car Engineer Nanodegree. https://www.udacity.com/course/self-driving-car-engineer-nanodegree--nd013 (2019). Accessed 27 June 2019

Download references

Acknowledgements

Funding was provided by Tecnologico de Monterrey (Grant No. A01333649) and Consejo Nacional de Ciencia y Tecnología (Grant No. CVU 850538).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ricardo A. Ramirez-Mendoza.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Curiel-Ramirez, L.A., Ramirez-Mendoza, R.A., Izquierdo-Reyes, J. et al. Hardware in the loop framework proposal for a semi-autonomous car architecture in a closed route environment. Int J Interact Des Manuf 13, 1647–1658 (2019). https://doi.org/10.1007/s12008-019-00619-x

Download citation

Keywords

  • Semi-autonomous driving
  • Autonomous vehicles
  • Machine learning
  • Hardware in the loop
  • Design framework
  • Simulation platform
  • Intelligent transportation systems