Skip to main content
Book cover

International Conference on Computer Safety, Reliability, and Security

SAFECOMP 2021: Computer Safety, Reliability, and Security. SAFECOMP 2021 Workshops pp 284–290Cite as

Learning from AV Safety: Hope and Humility Shape Policy and Progress

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 12853)

Abstract

Producing automated vehicles (AVs) that are, and can be shown to be, safe is an ongoing challenge. This position paper draws on recent work to discuss alternative approaches to assessing AV safety, noting how AI can be a positive or negative influence it. It features suggestions to promote AV safety, drawing from practice and policy, and it ends with a speculation about a special new role for AI.

Keywords

  • Automated vehicle
  • Safety
  • AI
  • Measurement
  • Policy

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-83906-2_23
  • Chapter length: 7 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-83906-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Notes

  1. 1.

    Even conventional vehicles are cyberphysical systems with many cyber components and vulnerabilities, creating risks arising from malice in addition to safety risks arising from circumstances or inadvertence.

  2. 2.

    We interviewed AV developers in industry, officials from different levels of government, independent safety researchers, and safety advocates.

  3. 3.

    Gameability has been a concern with disengagements, which are used in different ways by different developers, and it has been raised in connection with proposals to have AVs perform on prespecified tests.

  4. 4.

    Our report discussed how measures from different settings and stages can be aggregated into a framework that could be used across the industry.

  5. 5.

    An early innovation of this kind is the Responsibility Sensitive Safety (RSS) model introduced by Mobileye.

  6. 6.

    The IEEE P2846 working group focuses on a Formal Model for Safety Considerations in Automated Vehicle Decision Making. See: https://sagroups.ieee.org/2846/.

  7. 7.

    As we explained in [3], drawing from psychology research, people are more willing to accept human error than machine error.

  8. 8.

    ISO 26262 for functional safety and ISO/PAS 21448 for safety of the intended functionality motivate this caution.

  9. 9.

    Investigations into the Boeing 737 MAX, Tesla, and Uber crashes have underscored this concern most recently.

  10. 10.

    Also, a legally appropriate explanation might not require technical detail but rather might motivate technical development to support legal expectations: “To build AI systems that can provide explanation in terms of human-interpretable terms, we must both list those terms and allow the AI system access to examples to learn them. System designers should design systems to learn these human-interpretable terms, and also store data from each decision so that is possible to reconstruct and probe a decision post-hoc if needed.” [6].

References

  1. Fraade-Blanar, L., Blumenthal, M.S., et al.: Measuring automated vehicle safety: Forging a Framework. RAND Corporation, Santa Monica, CA (2018)

    CrossRef  Google Scholar 

  2. Kalra, N., Paddock, S.: Driving to safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? RAND Corporation, Santa Monica, CA (2018)

    Google Scholar 

  3. Blumenthal, M., Fraade-Blanar, L., et al.: Safe enough: Approaches to Assessing Acceptable Safety for Automated Vehicles. RAND Corporation, Santa Monica, CA (2020)

    CrossRef  Google Scholar 

  4. NHTSA Advanced notice of proposed rulemaking. https://www.federalregister.gov/documents/2020/12/03/2020-25930/framework-for-automated-driving-system-safety. Accessed 15 June 2021

  5. UL 4600 Homepage. https://ul.org/UL4600. Accessed 15 June 2021

  6. Doshi-Velez, F., et al.: Accountability of AI under the law: the role of explanation, arXiv (2017)

    Google Scholar 

  7. NHTSA Voluntary safety self-assessments. https://www.nhtsa.gov/automated-driving-systems/voluntary-safety-self-assessment. Accessed 15 June 2021

  8. California department of motor vehicles disengagement. https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/. Accessed 15 June 2021

Download references

Author information

Authors and Affiliations

  1. RAND Corporation, Arlington, VA, USA

    Marjory S. Blumenthal

Authors
  1. Marjory S. Blumenthal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marjory S. Blumenthal .

Editor information

Editors and Affiliations

  1. University of York, York, UK

    Ibrahim Habli

  2. Human Factors Everywhere Ltd., Woking, UK

    Mark Sujan

  3. University of York, York, UK

    Dr. Simos Gerasimou

  4. AIT Austrian Institute of Technology GmbH, Vienna, Austria

    Erwin Schoitsch

  5. Thales Deutschland GmbH, Ditzingen, Germany

    Friedemann Bitsch

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Blumenthal, M.S. (2021). Learning from AV Safety: Hope and Humility Shape Policy and Progress. In: Habli, I., Sujan, M., Gerasimou, S., Schoitsch, E., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2021 Workshops. SAFECOMP 2021. Lecture Notes in Computer Science(), vol 12853. Springer, Cham. https://doi.org/10.1007/978-3-030-83906-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-83906-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-83905-5

  • Online ISBN: 978-3-030-83906-2

  • eBook Packages: Computer ScienceComputer Science (R0)