Skip to main content
SpringerLink
Log in

Simulation-Based Safety Testing of Automated Driving Systems

  • Conference paper
  • First Online:
Product-Focused Software Process Improvement (PROFES 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14484))

  • 219 Accesses

Abstract

An Automated Driving System (ADS) must undergo comprehensive safety testing before receiving a road permit. Since it is not clear what exactly constitutes sufficient safety for an ADS, one could assume that an ADS is safe enough if it is at least as safe as a Human Driven Vehicle (HDV). Simulation-based testing is a cost-effective way to check the safety of an ADS. My goal is to develop an approach to compare the safety behavior of ADS and HDV using simulation. This comparison aims to quantify the advantages and disadvantages of ADS compared to HDV. Additionally, I aim to develop a process for selecting specific scenarios that contribute to building trust in the accuracy and reliability of simulation results. This involves defining performance criteria against which the behavior of an ADS in the simulator is compared to that of a HDV. Furthermore, I aim to translate the performance advantages or disadvantages observed in simulated ADS behavior into real-world safety-critical traffic scenarios.

Supported by Estonian Research Council grant PRG1226, Bolt Technology OÜ grant, and the Estonian state stipend for doctoral studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The ADS under test.

  2. 2.

    https://carla.org/.

  3. 3.

    https://carla-scenariorunner.readthedocs.io/en/latest/.

  4. 4.

    https://carla.readthedocs.io/en/latest/python_api/.

  5. 5.

    https://carla.readthedocs.io/projects/ros-bridge/en/latest/.

  6. 6.

    https://github.com/UT-ADL/autoware_mini.

  7. 7.

    List of scenarios - Land Transport Authority of Singapore.

  8. 8.

    List of scenarios - US Department of Transportation - Table 1.

References

  1. Althoff, M., Dolan, J.M.: Online verification of automated road vehicles using reachability analysis. IEEE Trans. Robot. 30(4), 903–918 (2014)

    Article  Google Scholar 

  2. Ben Abdessalem, R., Nejati, S., Briand, L.C., Stifter, T.: Testing advanced driver assistance systems using multi-objective search and neural networks. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pp. 63–74 (2016)

    Google Scholar 

  3. Broggi, A., et al.: Extensive tests of autonomous driving technologies. IEEE Trans. Intell. Transp. Syst. 14(3), 1403–1415 (2013)

    Article  Google Scholar 

  4. Broggi, A., Cerri, P., Felisa, M., Laghi, M.C., Mazzei, L., Porta, P.P.: The VisLab intercontinental autonomous challenge: an extensive test for a platoon of intelligent vehicles. Int. J. Veh. Auton. Syst. 10(3), 147–164 (2012)

    Article  Google Scholar 

  5. Chen, Z., He, F., Yin, Y., Du, Y.: Optimal design of autonomous vehicle zones in transportation networks. Transp. Res. Part B: Methodological 99, 44–61 (2017)

    Article  Google Scholar 

  6. Duarte, F., Ratti, C.: The impact of autonomous vehicles on cities: a review. J. Urban Technol. 25(4), 3–18 (2018)

    Article  Google Scholar 

  7. Funke, J., et al.: Up to the limits: autonomous Audi TTS. In: 2012 IEEE Intelligent Vehicles Symposium, pp. 541–547. IEEE (2012)

    Google Scholar 

  8. Jha, S., et al.: Kayotee: a fault injection-based system to assess the safety and reliability of autonomous vehicles to faults and errors. arXiv preprint arXiv:1907.01024 (2019)

  9. Nieuwenhuijze, M.R., van Keulen, T., Öncü, S., Bonsen, B., Nijmeijer, H.: Cooperative driving with a heavy-duty truck in mixed traffic: experimental results. IEEE Trans. Intell. Transp. Syst. 13(3), 1026–1032 (2012)

    Article  Google Scholar 

  10. O’Kelly, M., Sinha, A., Namkoong, H., Tedrake, R., Duchi, J.C.: Scalable end-to-end autonomous vehicle testing via rare-event simulation. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  11. Ploeg, J., Scheepers, B.T., Van Nunen, E., Van de Wouw, N., Nijmeijer, H.: Design and experimental evaluation of cooperative adaptive cruise control. In: 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 260–265. IEEE (2011)

    Google Scholar 

  12. Wicker, M., Huang, X., Kwiatkowska, M.: Feature-guided black-box safety testing of deep neural networks. In: Beyer, D., Huisman, M. (eds.) TACAS 2018. LNCS, vol. 10805, pp. 408–426. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-89960-2_22

    Chapter  Google Scholar 

Download references

Acknowledgements

This research was partly funded by the Austrian BMK, BMAW, and State of Upper Austria under the SCCH competence center INTEGRATE [(FFG grant 892418)], the Estonian Research Council (grant PRG1226), Bolt Technology OÜ, and the Estonian state stipend for doctoral studies.

Author information

Authors and Affiliations

  1. University of Tartu, Narva mnt 18, 51009, Tartu, Estonia

    Fauzia Khan, Hina Anwar & Dietmar Pfahl

Authors
  1. Fauzia Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hina Anwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dietmar Pfahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fauzia Khan .

Editor information

Editors and Affiliations

  1. FHV Vorarlberg University of Applied Science, Dornbirn, Austria

    Regine Kadgien

  2. Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany

    Andreas Jedlitschka

  3. FHV Vorarlberg University of Applied Science, Dornbirn, Austria

    Andrea Janes

  4. University of Oulu, Oulu, Finland

    Valentina Lenarduzzi

  5. University of Oulu, Oulu, Finland

    Xiaozhou Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khan, F., Anwar, H., Pfahl, D. (2024). Simulation-Based Safety Testing of Automated Driving Systems. In: Kadgien, R., Jedlitschka, A., Janes, A., Lenarduzzi, V., Li, X. (eds) Product-Focused Software Process Improvement. PROFES 2023. Lecture Notes in Computer Science, vol 14484. Springer, Cham. https://doi.org/10.1007/978-3-031-49269-3_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49269-3_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49268-6

  • Online ISBN: 978-3-031-49269-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation