A Concept of Scenario Space Exploration with Criticality Coverage Guarantees

Abstract

Assuring the safety of an automated driving system is difficult, because a large, heterogeneous set of traffic situations has to be handled by the system. Systematic testing of the full system at the end of the development seems necessary to be able to reach the required level of assurance. In our approach, the set of potentially relevant, concrete test cases result by parameter instantiation from finitely many more abstract, so called logical scenarios. For nearly all interesting automation systems, even virtual testing via simulation can cover only a tiny fraction of this set of concrete test cases.

Here we present an approach by which a selection of test cases can be shown to be sufficient to assert the system’s safety. For that, we make reasonable assumptions about the system’s inner workings, and about the way safety of a traffic situation can be captured mathematically. Based on these assumptions a criterion for test coverage is derived. This criterion can be used in a simulation procedure exploring the scenario space as a stop condition. If some additional conditions are met, the criterion is shown to imply sufficient coverage to assert safety of the system under test. We discuss the extent and limitation of the resulting guarantee.

We plan to elaborate, implement, and demonstrate this procedure in the context of research projects which develop and apply simulation tools for the verification and validation of automated driving systems.

This research was partially funded by the German Federal Ministry for Economic Affairs and Energy, Grant No. 19A18017 B (SET Level 4 to 5), based on a decision by the Parliament of the Federal Republic of Germany. The responsibility for the content lies with the author.