Skip to main content
SpringerLink
Log in

Assessing Dependability of Autonomous Vehicles

  • Chapter
  • First Online:
System Dependability and Analytics

Part of the book series: Springer Series in Reliability Engineering ((RELIABILITY))

  • 469 Accesses

Abstract

Autonomous vehicles (AVs) such as self-driving cars and unmanned aerial vehicles are complex systems that use artificial intelligence (AI) and machine learning (ML) to make real-time navigational decisions. Ensuring the dependability of AVs in terms of robustness, correctness, reliability, and safety is critical for their mass deployment and public adoption. However, it is challenging to assess and ensure the dependability of these systems due to their complexity both in terms of software and hardware and in terms of the inherent stochasticity and uncertainty in the sensor data and ML/AI algorithms. In this chapter, we design and develop novel assessment techniques to rigorously validate the AV system, including its runtime operational characteristics. The developed assessment techniques address the challenges mentioned above and significantly outperform the current state-of-the-art assessment techniques. We demonstrate our developed techniques and scientific contributions using self-driving cars as a motivating example.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    A transfer of control from the autonomous system to the human driver in the case of a failure is called a disengagement. Disengagements can be initiated either manually by the driver or autonomously by the car.

  2. 2.

    We use the terms “perturbation” and “fault” interchangeably.

  3. 3.

    In this work, we use simulation traces consisting of the recorded inputs and outputs of the system software modules at each timestep obtained while simulating a driving scenario in a Physics-based simulation engine.

  4. 4.

    We use the shorthand \(\delta > 0\) to mean both lateral and longitudinal \(\delta \)s.

References

  1. Erlien SM (2015) Shared vehicle control using safe driving envelopes for obstacle avoidance and stability. PhD thesis, Stanford University

    Google Scholar 

  2. Jongsang S, Boemjun K, Kyongsu Y (2016) Design and evaluation of a driving mode decision algorithm for automated driving vehicle on a motorway. IFAC-PapersOnLine 49(11):115–120

    Article  Google Scholar 

  3. Nvidia. Nvidia Drive. https://developer.nvidia.com/driveworks

  4. Apollo Open Platform. http://apollo.auto. Accessed: 2018-09-02

  5. Openpilot git repo

    Google Scholar 

  6. Alvarez S (2018) Research group demos why Tesla Autopilot could crash into a stationary vehicle. https://www.teslarati.com/tesla-research-group-autopilot-crash-demo/

  7. Economist T (2018) Why Uber’s self-driving car killed a pedestrian. The Economist May 29, 2018 https://www.economist.com/the-economist-explains/2018/05/29/why-ubers-self-driving-car-killed-a-pedestrian

  8. Chung K, Kalbarczyk ZT, Iyer RK (2019) Availability attacks on computing systems through alteration of environmental control: Smart malware approach. In: Proceedings of the 10th ACM/IEEE international conference on cyber-physical systems, ICCPS ’19, pp 1–12, New York, NY, USA, 2019. Association for Computing Machinery

    Google Scholar 

  9. SAE International (2016) Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles

    Google Scholar 

  10. Mujica F (2014) Scalable electronics driving autonomous vehicle technologies. Texas Instruments White Paper

    Google Scholar 

  11. Banerjee SS, Jha S, Cyriac J, Kalbarczyk ZT, Iyer RK (2018) Hands off the wheel in autonomous vehicles?: A systems perspective on over a million miles of field data. In: 2018 48th annual IEEE/IFIP international conference on dependable systems and networks (DSN), pp 586–597

    Google Scholar 

  12. Leveson N (2011) Engineering a safer world: systems thinking applied to safety. MIT Press

    Google Scholar 

  13. California Department of Motor Vehicles. Testing of autonomous vehicles. https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/testing. Accessed: 27 Nov 2017

  14. Umit O, Christoph S, Keith R (2007) Systems for safety and autonomous behavior in cars: The DARPA grand challenge experience. Proc IEEE 95(2):397–412

    Article  Google Scholar 

  15. Martin B, Karl I, Sanjiv S (eds) (2009) The DARPA Urban Challenge. Springer, Berlin Heidelberg

    Google Scholar 

  16. Urmson C, Anhalt J, Bagnell D, Baker C, Bittner R, Clark MN, Dolan J, Duggins D, Galatali T, Geyer C, Gittleman M, Harbaugh S, Hebert M, Howard TM, Kolski S, Kelly A, Likhachev M, McNaughton M, Miller N, Peterson K, Pilnick B, Rajkumar R, Rybski P, Salesky B, Seo Y-W, Singh S, Snider J, Stentz A, Whittaker WR, Wolkowicki Z, Ziglar J, Bae H, Brown T, Demitrish D, Litkouhi B, Nickolaou J, Sadekar V, Zhang W, Struble J, Taylor M, Darms M, Ferguson D (2008) Autonomous driving in urban environments: boss and the urban challenge. J Field Rob 25(8):425–466

    Article  Google Scholar 

  17. Stanek G, Langer D, Müller-Bessler B, Huhnke B (2010) Junior 3: a test platform for advanced driver assistance systems. In: Intelligent vehicles symposium (IV), 2010 IEEE, pp 143–149

    Google Scholar 

  18. Levinson J, Askeland J, Becker J, Dolson J, Held D, Kammel S, Kolter JZ, Langer D, Pink O, Pratt V, Sokolsky M, Stanek G, Stavens D, Teichman A, Werling M, Thrun S (2011) Towards fully autonomous driving: systems and algorithms. In: 2011 IEEE Intelligent vehicles symposium (IV), pp 163–168

    Google Scholar 

  19. Chatham A (2013) Google’s self-driving cars: the technology, capabilities, and challenges. In: 2013 embedded linux conference, pp 20–24

    Google Scholar 

  20. Chris U (2012) Realizing self-driving vehicles. In: IEEE intelligent vehicles symposium (IV). Alcalá des Henares, Spanien, p 2012

    Google Scholar 

  21. Paden B, Čáp M, Yong SZ, Yershov D, Frazzoli E (2016) A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans Intell Veh 1(1):33–55

    Article  Google Scholar 

  22. Fan C, Qi B, Mitra S, Viswanathan M (2017) DryVR: data-driven verification and compositional reasoning for automotive systems. In: Computer aided verification. Springer International Publishing, pp 441–461

    Google Scholar 

  23. Waymo (2017) On the road to fully self-driving. Waymo safety report https://assets.documentcloud.org/documents/4107762/Waymo-Safety-Report-2017.pdf. Accessed: 27 Nov 2017

  24. Pearl J (2018) Theoretical impediments to machine learning with seven sparks from the causal revolution

    Google Scholar 

  25. Jha S, Banerjee S, Tsai T, Hari SKS, Sullivan MB, Kalbarczyk ZT, Keckler SW, Iyer RK (2019) ML-based fault injection for autonomous vehicles: a case for bayesian fault injection. In: 2019 49th annual IEEE/IFIP international conference on dependable systems and networks (DSN), pp 112–124

    Google Scholar 

  26. Pearl J (2014) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann

    Google Scholar 

  27. Erlien SM, Fujita S, Gerdes JC (2013) Safe driving envelopes for shared control of ground vehicles. IFAC Proc 46(21):831–836

    Google Scholar 

  28. Anderson SJ, Karumanchi SB, Iagnemma K (2012) Constraint-based planning and control for safe, semi-autonomous operation of vehicles. In: 2012 IEEE intelligent vehicles symposium, pp 383–388

    Google Scholar 

  29. Li G, Li Y, Jha S, Tsai T, Sullivan M, Hari SKS, Kalbarczyk Z, Iyer R (2020) Av-fuzzer: finding safety violations in autonomous driving systems. In: 2020 IEEE 31st international symposium on software reliability engineering (ISSRE), pp 25–36

    Google Scholar 

  30. Jha S, Cui S, Banerjee S, Cyriac J, Tsai T, Kalbarczyk Z, Iyer RK (2020) Ml-driven malware that targets AV safety. In: 2020 50th annual IEEE/IFIP international conference on dependable systems and networks (DSN), pp 113–124

    Google Scholar 

  31. Apollo. https://github.com/ApolloAuto/apollo

  32. Jia Y, Lu Y, Shen J, Chen QA, Chen H, Zhong Z, Wei T (2020) Fooling detection alone is not enough: adversarial attack against multiple object tracking. In: International conference on learning representations

    Google Scholar 

  33. Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017) CARLA: an open urban driving simulator. In: Levine S, Vanhoucke V, Goldberg K (eds) Proceedings of the 1st annual conference on robot learning, volume 78 of Proceedings of machine learning research, pp 1–16. PMLR, 13–15 Nov 2017

    Google Scholar 

  34. Anderson James M, Nidhi K, Stanley Karlyn D, Paul S, Constantine S, Oluwatola Tobi A (2016) Autonomous vehicle technology: a guide for policymakers. RAND Corporation, Santa Monica, CA

    Book  Google Scholar 

  35. Kalra N, Paddock SM (2016) Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp Res Part A: Pol Pract 94:182–193

    Google Scholar 

  36. Laura Blumenthal Marjory, S, Anderson James M, Nidhi K, (2018) Measuring automated vehicle safety: forging a framework. RAND Corporation, Santa Monica, CA

    Google Scholar 

  37. Pei K, Cao Y, Yang J, Jana S (2017) DeepXplore: automated whitebox testing of deep learning systems. In: Proceeding of the 26th symposium on operating systems principles, pp 1–18

    Google Scholar 

  38. Rubaiyat AHM, Qin Y, Alemzadeh H (2018) Experimental resilience assessment of an open-source driving agent. In: 2018 IEEE 23rd pacific rim international symposium on dependable computing (PRDC). IEEE, pp 54–63

    Google Scholar 

  39. Jha S, Banerjee SS, Cyriac J, Kalbarczyk ZT, Iyer RK (2018) AVFI: fault injection for autonomous vehicles. In: 2018 48th annual IEEE/IFIP international conference on dependable systems and networks workshops (DSN-W), pp 55–56

    Google Scholar 

  40. Li G, Hari SKS, Sullivan M, Tsai T, Pattabiraman K, Emer J, Keckler SW (2017) Understanding error propagation in deep learning neural network (DNN) accelerators and applications. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, SC ’17, New York, NY, USA, 2017. Association for Computing Machinery

    Google Scholar 

  41. Jha S, Tsai T, Hari S, Sullivan M, Kalbarczyk Z, Keckler SW, Iyer RK (2018) Kayotee: a fault injection-based system to assess the safety and reliability of autonomous vehicles to faults and errors. In: Third IEEE international workshop on automotive reliability and Test. IEEE

    Google Scholar 

  42. Philip K, Michael W (2018) Toward a framework for highly automated vehicle safety validation. Technical report, SAE Technical Paper

    Google Scholar 

  43. Lu J, Sibai H, Fabry E, Forsyth D (2017) No need to worry about adversarial examples in object detection in autonomous vehicles. arXiv preprint arXiv:1707.03501

  44. Pei K, Cao Y, Yang J, Jana S (2017) Towards practical verification of machine learning: the case of computer vision systems. arXiv preprint arXiv:1712.01785

  45. Lakkaraju H, Kamar E, Caruana R, Horvitz E (2017) Identifying unknown unknowns in the open world: representations and policies for guided exploration. In: AAAI, vol 1, pp 2

    Google Scholar 

  46. Saurabh J, Yan M, Zbigniew K, Iyer RK (2021) Assessing risk in dynamic environments for safe driving. Watch out for the risky actors

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM T. J. Watson Research, Yorktown Heights, USA

    Saurabh Jha

Authors
  1. Saurabh Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saurabh Jha .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Long Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Karthik Pattabiraman

  3. Nokia Bell Labs, São Paulo, Brazil

    Catello Di Martino

  4. Mayo Clinic, Rochester, MN, USA

    Arjun Athreya

  5. Purdue University, West Lafayette, IN, USA

    Saurabh Bagchi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jha, S. (2023). Assessing Dependability of Autonomous Vehicles. In: Wang, L., Pattabiraman, K., Di Martino, C., Athreya, A., Bagchi, S. (eds) System Dependability and Analytics. Springer Series in Reliability Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-02063-6_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-02063-6_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-02062-9

  • Online ISBN: 978-3-031-02063-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation