Abstract
Several standards from the domain of safety critical systems, in order to support the argumentation of the safety assurance of a system under development, recommend the construction of a safety case. This activity is guided by the objectives to be met, recommended or required by the standards along the safety lifecycle. Ongoing attempts to use Machine Learning (ML) for safety critical functionality revealed certain deficits. For instance, the widely recognized standard for functional safety of automotive systems, ISO 26262, which can be used as a basis to construct a safety case, does not reason about ML. To this end, the goal of this work is to provide a pattern for arguing about the correct implementation of safety requirements in system components based on ML. The pattern is integrated within an overall encompassing approach for safety case generation for automotive systems and its applicability is showcased on a pedestrian avoidance system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
Bundesministerium für Wirtschaft und Energie, Project KI Absicherung (eng. AI Safety) https://www.ki-absicherung.vdali.de/.
References
McDermid, J., Jia, Y., Habli, I.: Towards a framework for safety assurance of autonomous systems. In: Proceedings of the Workshop on Artificial Intelligence Safety, vol. 2419. CEUR-WS.org (2019)
ISO 26262: Road vehicles - Functional safety (2011)
ISO/PAS 21448: Road vehicles - Safety of the intended functionality (2019)
Bloomfield, R., Bishop, P.: Safety and assurance cases: past, present and possible future – an Adelard perspective. In: Dale, C., Anderson, T. (eds.) Making Systems Safer, pp. 51–67. Springer, London (2010). https://doi.org/10.1007/978-1-84996-086-1_4
The Assurance Case Working Group (ACWG) - Goal Structuring Notation Community Standard Version 2, January 2018
Carlan, C., Gallina, B.: Enhancing state-of-the-art safety case patterns to support change impact analysis. In: 30th European Safety and Reliability Conference, November 2020. http://www.es.mdh.se/publications/5789-
Picardi, C., Hawkins, R., Paterson, C., Habli, I.: A pattern for arguing the assurance of machine learning in medical diagnosis systems. In: Romanovsky, A., Troubitsyna, E., Bitsch, F. (eds.) SAFECOMP 2019. LNCS, vol. 11698, pp. 165–179. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26601-1_12
Picardi, C., Paterson, C., Hawkins, R., Calinescu, R., Habli, I.: Assurance argument patterns and processes for machine learning in safety-related systems. In: Proceedings of the Workshop on Artificial Intelligence Safety, Series CEUR Workshop Proceedings, vol. 2560, pp. 23–30. CEUR-WS.org (2020)
Burton, S., Gauerhof, L., Sethy, B.B., Habli, I., Hawkins, R.: Confidence arguments for evidence of performance in machine learning for highly automated driving functions. In: Romanovsky, A., Troubitsyna, E., Gashi, I., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2019. LNCS, vol. 11699, pp. 365–377. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26250-1_30
Gauerhof, L., Munk, P., Burton, S.: Structuring validation targets of a machine learning function applied to automated driving. In: Gallina, B., Skavhaug, A., Bitsch, F. (eds.) SAFECOMP 2018. LNCS, vol. 11093, pp. 45–58. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99130-6_4
Salay, R., Czarnecki, K.: Using machine learning safely in automotive software: an assessment and adaption of software process requirements in ISO 26262. CoRR, abs/1808.01614 (2018). http://arxiv.org/abs/1808.01614
Rudolph, A., Voget, S., Mottok, J.: A consistent safety case argumentation for artificial intelligence in safety related automotive systems. In: ERTS 2018, Series Proceedings of 9th European Congress on Embedded Real Time Software and Systems, ERTS 2018, January 2018
Putzer, H.J., Wozniak, E.: A structured approach to trustworthy autonomous/cognitive systems. arXiv preprint arXiv:2002.08210 (2020)
DKE: Dke deutsche kommission elektrotechnik elektronik informationstechnik in din und vde - “dke/ak 801.0.8 spezifikation und entwurf autonomer/kognitiver systeme" (2020)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wozniak, E., Cârlan, C., Acar-Celik, E., Putzer, H.J. (2020). A Safety Case Pattern for Systems with Machine Learning Components. In: Casimiro, A., Ortmeier, F., Schoitsch, E., Bitsch, F., Ferreira, P. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2020 Workshops. SAFECOMP 2020. Lecture Notes in Computer Science(), vol 12235. Springer, Cham. https://doi.org/10.1007/978-3-030-55583-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-55583-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55582-5
Online ISBN: 978-3-030-55583-2
eBook Packages: Computer ScienceComputer Science (R0)