Abstract
Producing automated vehicles (AVs) that are, and can be shown to be, safe is an ongoing challenge. This position paper draws on recent work to discuss alternative approaches to assessing AV safety, noting how AI can be a positive or negative influence it. It features suggestions to promote AV safety, drawing from practice and policy, and it ends with a speculation about a special new role for AI.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Even conventional vehicles are cyberphysical systems with many cyber components and vulnerabilities, creating risks arising from malice in addition to safety risks arising from circumstances or inadvertence.
- 2.
We interviewed AV developers in industry, officials from different levels of government, independent safety researchers, and safety advocates.
- 3.
Gameability has been a concern with disengagements, which are used in different ways by different developers, and it has been raised in connection with proposals to have AVs perform on prespecified tests.
- 4.
Our report discussed how measures from different settings and stages can be aggregated into a framework that could be used across the industry.
- 5.
An early innovation of this kind is the Responsibility Sensitive Safety (RSS) model introduced by Mobileye.
- 6.
The IEEE P2846 working group focuses on a Formal Model for Safety Considerations in Automated Vehicle Decision Making. See: https://sagroups.ieee.org/2846/.
- 7.
As we explained in [3], drawing from psychology research, people are more willing to accept human error than machine error.
- 8.
ISO 26262 for functional safety and ISO/PAS 21448 for safety of the intended functionality motivate this caution.
- 9.
Investigations into the Boeing 737 MAX, Tesla, and Uber crashes have underscored this concern most recently.
- 10.
Also, a legally appropriate explanation might not require technical detail but rather might motivate technical development to support legal expectations: “To build AI systems that can provide explanation in terms of human-interpretable terms, we must both list those terms and allow the AI system access to examples to learn them. System designers should design systems to learn these human-interpretable terms, and also store data from each decision so that is possible to reconstruct and probe a decision post-hoc if needed.” [6].
References
Fraade-Blanar, L., Blumenthal, M.S., et al.: Measuring automated vehicle safety: Forging a Framework. RAND Corporation, Santa Monica, CA (2018)
Kalra, N., Paddock, S.: Driving to safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? RAND Corporation, Santa Monica, CA (2018)
Blumenthal, M., Fraade-Blanar, L., et al.: Safe enough: Approaches to Assessing Acceptable Safety for Automated Vehicles. RAND Corporation, Santa Monica, CA (2020)
NHTSA Advanced notice of proposed rulemaking. https://www.federalregister.gov/documents/2020/12/03/2020-25930/framework-for-automated-driving-system-safety. Accessed 15 June 2021
UL 4600 Homepage. https://ul.org/UL4600. Accessed 15 June 2021
Doshi-Velez, F., et al.: Accountability of AI under the law: the role of explanation, arXiv (2017)
NHTSA Voluntary safety self-assessments. https://www.nhtsa.gov/automated-driving-systems/voluntary-safety-self-assessment. Accessed 15 June 2021
California department of motor vehicles disengagement. https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/. Accessed 15 June 2021
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Blumenthal, M.S. (2021). Learning from AV Safety: Hope and Humility Shape Policy and Progress. In: Habli, I., Sujan, M., Gerasimou, S., Schoitsch, E., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2021 Workshops. SAFECOMP 2021. Lecture Notes in Computer Science(), vol 12853. Springer, Cham. https://doi.org/10.1007/978-3-030-83906-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-83906-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-83905-5
Online ISBN: 978-3-030-83906-2
eBook Packages: Computer ScienceComputer Science (R0)