Slow Down, Move Over: A Case Study in Formal Verification, Refinement, and Testing of the Responsibility-Sensitive Safety Model for Self-Driving Cars

Abstract

Technology advances give us the hope of driving without human error, reducing vehicle emissions and simplifying an everyday task with the future of self-driving cars. Making sure these vehicles are safe is very important to the continuation of this field. In this paper, we formalize the Responsibility-Sensitive Safety model (RSS) for self-driving cars and prove the safety and optimality of this model in the longitudinal direction. We utilize the hybrid systems theorem prover KeYmaera X to formalize RSS as a hybrid system with its nondeterministic control choices and continuous motion model, and prove absence of collisions. We then illustrate the practicality of RSS through refinement proofs that turn the verified nondeterministic control envelopes into deterministic ones and further verified compilation to Python. The refinement and compilation are safety-preserving; as a result, safety proofs of the formal model transfer to the compiled code, while counterexamples discovered in testing the code of an unverified model transfer back. The resulting Python code allows to test the behavior of cars following the motion model of RSS in simulation, to measure agreement between the model and simulation with monitors that are derived from the formal model, and to report counterexamples from simulation back to the formal model.

1 Introduction and Motivation

As technology advances, it becomes both more appealing and feasible to automate everyday tasks. In doing this, there is a higher need for robust formal verification and testing strategies to ensure that these automated systems are safe. Formal verification ensures that all situations encountered in a model of physics are safe, while testing provides evidence that the formal model works as expected and can give insight to bugs in a yet-to-be-verified model. To this end, a testing strategy analyzes some (representative) situations that an automated system may encounter. This paper develops a formal framework to link modeling, offline verification, online monitoring, and testing, sketched in Fig. 1 and illustrated throughout the paper with an example in autonomous driving.

Fig. 1.

Verification, refinement, and compilation for safety and counterexample transfer. All steps are accompanied with a correctness proof.

We use hybrid systems to describe the dynamic components of a car, with the goal of creating discrete computer controllers which prevent collisions and other unsafe situations. Analyzing such a system requires two components: a differential equation modeling the cars’ motion and a car control algorithm. For the differential equation, we follow [12] and describe the longitudinal motion of a car in response to acceleration/deceleration inputs. For the discrete controller, we follow the Responsibility-Sensitive Safety (RSS) model [23]. Its discrete control choices influence the continuous motion of the car, and are given in terms of a nondeterministic control envelope that allows for many different ways of deterministic control implementation. In the resulting hybrid systems model, we prove the safety (defined by absence of collision) of all possible implementations within the control envelope. This proof step replaces the need to test through simulation whether the model is safe, i.e. checking if cars collide in any implementation of the system, but simulation can help in debugging unverified models. We also want the ability to analyze concrete (deterministic) controller implementations and motion models and determine whether they follow the general verified model. We do this through refinement proofs in differential refinement logic [13] to a deterministic hybrid program followed by a verified compilation to Python which allows us to further analyze the behavior of a system. This way, we can analyze concrete behavior for concrete parameter choices in the formal model, and simulate behavior for analysis in select driving situations. In order to transfer the safety proof to all encountered situations, the remaining question after a safety proof is then to detect at runtime when the proof applies and when it does not. We answer this by applying ModelPlex [17] to generate monitors that are able to validate unverified code and motion model alike. We utilize the same monitors in testing and simulation to measure the robustness of control choices, or in other words, the safety-criticality of test scenarios. The proof, refinement, and compilation framework is set up in such a way that, if a proof step fails, compiling the formal model and analyzing it for counterexamples through testing is still possible.

2 Background

2.1 Responsibility-Sensitive Safety

We briefly summarize Responsibility-Sensitive Safety (RSS, [23]) that we use as a case study throughout this paper. The general purpose of RSS is to describe a safe algorithm for the interaction of (semi-)autonomous cars, with safety being defined as the avoidance of collision. RSS defines a lane-based coordinate system which describes the position of cars with a longitudinal and lateral coordinate relative to the lane they are in on a road. The longitudinal coordinate measures the distance relative to a starting position along the road, which abstracts from road geometry. The lateral position describes the distance from the center of the road. The lane-based coordinate system simplifies the motion model into a linear differential equation system as in [12].

Table 1. RSS variables and model parameters

The general idea behind the RSS control envelope is that two cars should maintain a safe distance between them, which is a function of the current speeds of the two cars. If at a safe distance, they are allowed to drive at any acceleration or deceleration within their cars’ maximum acceleration and maximum braking bounds. Otherwise, if not at a safe distance, they must respond with a predefined evasion maneuver called proper response within a specified reaction time horizon in order to remain safe. The safe distance is specified as a function of (relative) vehicle speed and vehicle parameters, as summarized in Table 1 [23].

2.2 Differential Dynamic Logic and Notation

We use differential dynamic logic (dL) [20] to create models of car control and motion, and to express correctness and optimality properties of RSS. Differential dynamic logic supports modeling hybrid systems complete with support for continuous dynamics expressed using differential equations. Within the RSS model, we utilize discrete dynamics to describe the control options in terms of instantly changing the acceleration or deceleration that the car is following. We use differential equations to model the resulting motion change in speed and position of the cars with respect to this chosen acceleration. dL comes with a proof calculus [20], which is supported in the automated theorem prover KeYmaera X  [5]. For the scope of this paper, the following informal understanding, summarized in Table 2 suffices. Assignment of a variable $x \, {:}{=}\, \theta$ instantly assigns the value of $\theta$ to x, nondeterministic assignment $x \, {:}{=}\, *$ assigns any real value to x (e.g. choosing an arbitrary acceleration value). Continuous evolution $\{x{'} = \theta ~ \& ~Q\}$ evolves x along the differential equation $x{'}=\theta$ for an arbitrary duration while Q is true throughout (e.g. change position of cars along kinematic equations, but prevent moving backwards). The test $?P$ checks that formula P is true and continues running if it is, and if not aborts the program (possibly backtracking to other nondeterministic options). Sequential composition $\alpha \texttt {;} \beta$ first runs hybrid program $\alpha$ and subsequently runs hybrid program $\beta$ from the resulting states of $\alpha$ (e.g. first choose acceleration, then follow motion). A nondeterministic choice of operations allows for the choice of two separate hybrid programs, $\alpha \,\cup \,\beta$ (e.g. running $\alpha$ if there is a safe distance between two cars and running emergency braking $\beta$ otherwise). Nondeterministic repetition $\alpha ^*$ repeats hybrid program $\alpha$ some $n \ge 0$ times (e.g. repeatedly running control and motion).

Table 2. Syntax and informal semantics of hybrid programs

The set of dL formulas is generated by the following grammar, where $\sim \,\in \{<,\le ,=,\ne ,\ge ,>\}$ and terms f, g in $+,-,\cdot ,/$ are over the reals, contain number literals, variables and n-ary rigid function symbols, plus support interpreted minimum, maximum, absolute value and trigonometric functions [6]:

$$\begin{aligned} P,Q {:}{=}f \sim g \mid P \wedge Q \mid P \vee Q \mid P \rightarrow Q \mid P \leftrightarrow Q \mid \forall x\,P \mid \exists x\,P \mid \left[ \alpha \right] P \mid \langle \alpha \, \rangle \, P . \end{aligned}$$

We use $f \sim g \sim h$ as shorthand for $f \sim g \wedge g \sim h$. Arithmetic operations, comparisons, Boolean connectives, and quantifiers are as usual. Formulas of the form $\left[ \alpha \right] P$ and $\langle \alpha \, \rangle \, P$ express properties about programs: $\left[ \alpha \right] P$ is true when in all states reachable through transitions of program $\alpha$ the property P is true (used for safety); $\langle \alpha \, \rangle \, P$ is true when there exists a state reachable by the program $\alpha$ in which P is true (used for liveness, optimality, and runtime monitoring).

The semantics of dL  [20] is a Kripke semantics in which the states of the Kripke model are the states of the hybrid system. A state is a map $\omega : \mathcal {V}\rightarrow \mathbb {R}$, assigning a real value $\omega (x)$ to each variable $x \in \mathcal {V}$ in the set of variables $\mathcal {V}$. We write $\llbracket \,P\,\rrbracket$ to denote the set of states in which formula P is true, $\omega \in \llbracket \,P\,\rrbracket$ or equivalently $\omega \models P$ if formula P is true at state $\omega$, $\models P$ for P being valid (true in all states), and $\omega \,\llbracket \,e\,\rrbracket$ to denote the real value of term $e$ in state $\omega$. The semantics of hybrid programs is expressed as a transition relation $\llbracket \alpha \, \rrbracket \subseteq \mathcal {S} \times \mathcal {S}$ of initial and final states in $\mathcal {S}$ [20] and we write $(\omega ,\nu ) \in \llbracket \alpha \, \rrbracket$ to express that program $\alpha$ reaches state $\nu$ when starting in state $\omega$. dL is decidable relative to either an oracle for the continuous first-order logic of differential equations or an oracle for the discrete fragment of dL [19, Theorem 11].

2.3 Differential Refinement Logic

Differential refinement logic dRL [13] defines a relationship between two hybrid programs $\alpha \le \beta$ that tells us that all the states reachable from program $\alpha$ are also reachable by operations of $\beta$. This allows us to transfer safety properties between programs: safety properties about a hybrid program also hold for all refinements of this program. In this paper, we refine the RSS control envelopes to create deterministic controllers, which are guaranteed to inherit the verified safety properties of the original control envelopes.

2.4 ModelPlex

A proof of $\left[ \alpha \right] P$ gives us confidence in the safety of the model $\alpha$. The remaining question is now whether $\alpha$ represents a useful real (driving) behavior, and whether an implementation is faithful to the conditions of the verified model. ModelPlex [17] combines an offline safety proof of the shape $A \rightarrow \left[ \alpha \right] P$ with a runtime monitor checking whether two concrete states $\omega ,\nu$ are connected by a program $\alpha$, i.e., semantically whether $(\omega ,\nu ) \in \llbracket \alpha \, \rrbracket$. The safety proof witnesses that, starting from states satisfying $A$, all states reachable by model $\alpha$ satisfy $P$, while satisfying the runtime monitor witnesses that the two concrete states $\omega ,\nu$ are connected by the program $\alpha$, and so state $\nu$ inherits the safety proof, i.e., $\nu \, \models \, P$ if $\omega \, \models \, A$. The semantic runtime monitor is equivalently phrased in dL as a monitor specification $\langle \alpha \, \rangle \, \bigwedge _{x\in \textsf {BV}(\alpha )}(x=x^+)$ with fresh variables $x^+$ not occurring in $\alpha$ [17]. The dL monitor specification allows ModelPlex, in contrast to online reachability analysis, to shift computation offline by using theorem proving in order to translate a hybrid systems model into a formula $\phi \in \mathcal {M}$ where $\mathcal {M}$ is the set of quantifier-free, modality-free formulas over real arithmetic. Note that the dL monitor specification is not provable offline, since it introduces fresh variables $x^+$. Instead, the proof can be finished at runtime for two concrete states (a state $\omega$ providing values for $x$ and a state $\nu$ providing values for $x^+$) by plugging in concrete measurements for all variables of the runtime monitor. When a runtime monitor $\phi$ is satisfied over states $\omega ,\nu$, we write $(\omega ,\nu ) \in \llbracket \phi \, \rrbracket$.

3 Formalization of RSS Safety and Optimality

In this section we begin our case study on RSS. We first formalize the RSS guidelines and verify the safety of the model. We then propose a way to verify that the RSS model drives optimally. The models and proofs [25] are formalized in the KeYmaera X ASCII syntax [15]. After this we create deterministic controllers which we show are refinements of the general verified model and utilize these to illustrate RSS behavior in simulation.

3.1 Formalization and Verification of RSS Safety

Models in the subsequent sections follow the same structure. To capture this common structure, we introduce a safety template in Model 1. All subsequent safety proofs for the longitudinal motion regardless of direction (same or opposite direction of motion), follow this template, but with specific formulas and programs to fill in the placeholders for the safety distance safeDist, evolution domain constraint edc, free driving program freeDriving, and evasion maneuver program properResponse.

The template defines preconditions (Line 1–4) under which all runs of the hybrid program in Line 6–11 are expected to establish the safety condition in Line 12. The preconditions init set parameter bounds for acceleration, deceleration, and reaction time $\rho$, make sure that the cars are oriented correctly (driving in the expected direction and car 1 is closer to the lane origin than car 2), and are at a safe distance initially. The $\texttt {ctrl}$ program implements the RSS control envelope by setting the accelerations of the cars based on whether the distance between them is safe: the nondeterministic choice in Line 8 expresses the two options of executing freeDriving when at a safe distance and evasion maneuver properResponse otherwise (we use notation $(a_1,a_2) \, {:}{=}\, \texttt {prg}$ to emphasize that the program prg chooses acceleration values $a_1$ and $a_2$). At the boundary, either choice is possible. The $\texttt {motion}$ program uses a differential equation to model the continuous dynamics of driving in response to the chosen acceleration, modeled through kinematics. Subsequent sections utilize this template to create specific dL models to prove several aspects of RSS safety. First we start with cars driving in the same longitudinal direction.

3.2 Longitudinal Safety

In the longitudinal direction, two cars may either drive in the same direction (a follower car following at a safe distance behind a lead car), or in opposite directions (a narrow road with two-way traffic, or a lead car backing up). We describe both models as instances of the template Model 1, using the sign of their respective longitudinal velocities to determine driving direction. We analyze these driving situations in isolation, under the assumption that a gear change when fully stopped is necessary to transition between these situations, which also allows for switching between the formal models that guide runtime safety.

Same Longitudinal Direction. For two cars driving in the same longitudinal direction, RSS [23] defines a safe distance between these cars with respect to their velocities: if the cars were to drive with worst-case behavior (rear car with the maximum possible acceleration and lead car with the maximum possible deceleration) for the entire reaction time $\rho$, there would still be enough space for the rear car to come to a full stop before the stopping point of the lead car, i.e., before colliding, see Eq. (1).

$$\begin{aligned} \texttt {safeDist}_s \triangleq \max (v_1\rho + \frac{1}{2}a_\text {maxAccel}+ \frac{(v_1 + \rho a_\text {maxAccel})^2}{2a_\text {minBrake}} - \frac{v_2^2}{2a_\text {maxBrake}}, 0) \end{aligned}$$

(1)

Whenever the cars are not satisfying the safe distance Eq. (1), the cars must execute the proper response behavior as an evasion maneuver. In a leader and follower setup, the follower car is responsible for not hitting the lead car. As a consequence, the proper response allows the lead car to continue with any acceleration or deceleration that does not exceed maximum braking, but requires the follower car to decelerate at rate at least $a_\text {minBrake}$ [23]. We will now fill in Model 1 with formulas and programs Eqs. (2)–(4), which gives [12, Model 1].

$$\begin{aligned} \texttt {freeDriving}_s&\triangleq a_1 \, {:}{=}\, *; ?{-}a_\text {maxBrake}\le a_1 \le a_\text {maxAccel}; \nonumber \\&~~~~ a_2 \, {:}{=}\, *; ?{-}a_\text {maxBrake}\le a_2 \le a_\text {maxAccel} \end{aligned}$$

(2)

$$\begin{aligned} \texttt {properResponse}_s&\triangleq \bigl (a_1 \, {:}{=}\, *; ?a_1\le -a_\text {minBrake}\,\cup \,?v_1=0; a_1 \, {:}{=}\, 0\bigl ); \nonumber \\&~~~~ \bigl ( a_2 \, {:}{=}\, *; ?a_2\ge a_\text {maxBrake}\,\cup \,?v_2=0; a_2 \, {:}{=}\, 0\bigl ) \end{aligned}$$

(3)

$$\begin{aligned} \texttt {edc}_s&\triangleq v_1 \ge 0 \wedge v_2 \ge 0 \end{aligned}$$

(4)

The definition of $\texttt {properResponse}$ includes a choice of $a_{1, 2} \, {:}{=}\, 0$ if $v_{1, 2} = 0$ in order to allow time to pass in the ODE if one or both cars are stopped.

Theorem 1

(Same Longitudinal Safety). Model 1 with safeDist as per Eq. (1), freeDriving as per Eq. (2), properResponse as per Eq. (3), and edc as per Eq. (4) is valid.

Proof

The key insight to this proof is the loop invariant $J \equiv x_1 \le x_2 \wedge x_1 + \frac{v_1^2}{2a_\text {minBrake}} \le x_2 + \frac{v_1^2}{2a_\text {maxBrake}}$ for proving inductive safety using the loop invariant (LI) rule.

$$\frac{\varGamma \vdash J, \varDelta \quad J \vdash P \quad J \vdash [\alpha ]J }{\varGamma \vdash [\alpha *]P, \varDelta }LI$$

The loop invariant J summarizes that the cars maintain sufficient distance to stop before colliding. The resulting real arithmetic proof obligations become tractable by relating terms of the safe distance formula to terms in the solution of the differential equations using a series of cuts. The details of this proof are in [24].    $\square$

Opposite Longitudinal Direction. The responsibility for collision avoidance is shared among the cars when they drive towards each other (in opposite longitudinal direction on the same lane). Similar to the same longitudinal direction, the safe distance to be maintained in opposite direction driving considers the worst-case behavior for the full reaction time $\rho$ followed by the collision avoidance proper response. In opposite direction, the worst-case behavior is accelerating towards each other with acceleration of $a_\text {maxAccel}$ for $a_1$ and $-a_\text {maxAccel}$ for $a_2$ in the opposing direction. For two cars driving with opposite longitudinal velocity signs, let $v_{1, \rho } = v_1 + \rho a_\text {maxAccel}$ and $v_{2, \rho } = |v_2| + \rho a_\text {maxAccel}$. Then, [23] defines the safe distance in longitudinal opposite direction as in Eq. (5).

$$\begin{aligned} \texttt {safeDist}_o = \frac{v_1 + v_{1, \rho }}{2}\rho + \frac{(v_{1, \rho })^2}{2a_\text {minBrake}} + \frac{v_2 + v_{2, \rho }}{2} \rho + \frac{(v_{2, \rho })^2}{2a_\text {minBrake}} \end{aligned}$$

(5)

In this case, we assume the cars drive towards each other (car 1 driving “forward”, i.e., away from the lane origin, car 2 driving “backward”, i.e., towards the lane origin), and want both of them to decelerate in the proper response. This differs from the same longitudinal direction case, where only the follower car reacts. In this case, the car driving forward reacts to imminent collision with deceleration $a_1 \le -a_\text {minBrake}$, and the car driving backward reacts with deceleration $a_2 \ge a_\text {minBrake}$, formalized in Eqs. (6)–(8).

$$\begin{aligned} \texttt {freeDriving}_o&\triangleq a_1 \, {:}{=}\, *; ?-a_\text {maxBrake}\le a_1 \le a_\text {maxAccel}; \nonumber \\&~~~~ a_2 \, {:}{=}\, *; ?-a_\text {maxAccel}\le a_2 \le a_\text {maxBrake}\end{aligned}$$

(6)

$$\begin{aligned} \texttt {properResponse}_o&\triangleq \bigl ((a_1 \, {:}{=}\, *; ?a_1\le -a_\text {minBrake})\,\cup \,(?v_1=0; a_1 \, {:}{=}\, 0)\bigl ); \nonumber \\&~~~~ \bigl ((a_2 \, {:}{=}\, *; ?a_2\ge a_\text {minBrake})\,\cup \,(?v_2=0; a_2 \, {:}{=}\, 0)\bigl ) \end{aligned}$$

(7)

$$\begin{aligned} \texttt {edc}_o&\triangleq v_1 \ge 0 \wedge v_2 \le 0 \end{aligned}$$

(8)

Theorem 2

(Opposite Longitudinal Safety). Model 1 with $\texttt {safeDist}_o$ Eq. (5), $\texttt {freeDriving}_o$ Eq. (6), $\texttt {properResponse}_o$ Eq. (7), and $\texttt {edc}_o$ Eq. (8) is valid.

Proof

The proof of this model follows a similar structure to the proof of Theorem 1 in the same longitudinal direction. The key insight again is a loop invariant: $J \equiv x_1\le x_2 \wedge x_2 - \frac{v_2^2}{2a_\text {minBrake}} \ge x_1 + \frac{v_1^2}{2a_\text {minBrake}}$. Details of this proof are in [24].    $\square$

3.3 Formalization and Verification of RSS Optimality

This section analyzes whether the safe distance is the minimal safe distance needed to avoid collision. To this end, we encode the rationale behind $\texttt {safeDist}_s$ and $\texttt {safeDist}_o$ as a hybrid program, and ask if we were to follow a smaller safe distance, would there exist collisions (which is a liveness property of the shape $P \rightarrow \langle \alpha \, \rangle \, Q$). Recall that in both cases, the safe distance was determined by one iteration of worst-case behavior of duration $\rho$, followed by executing the proper response until both cars are stopped. Model 2 captures this intuition behind the safe distance formulas in a hybrid program: first, the worst-case behavior in Line 5–7 is executed, followed by any number and duration of proper response in Line 8–9. We show that starting from an initial separation of $\texttt {safeDist}-\varepsilon$ for any arbitrarily small $\varepsilon$, leads to collisions, i.e., we can reach states in which $x_1>x_2$.

Theorem 3

(Same Longitudinal Optimality). Model 2 with safeDist as per Eq. (1), freeDriving as per Eq. (2), properResponse as per Eq. (3), and edc as per Eq. (4) is valid.

Proof

The proof strategy for this case simulates worst case behavior, choosing $a_1=a_\text {maxAccel}$, $a_2=-a_\text {minBrake}$. Unrolling the loop twice enables both cars to come to a complete stop and stay stopped by picking $a_{1,2}=0$ once stopped. The choice of accelerations and loop unrolling results in a collision. Note that two iterations are necessary in the case that one car stops before the other one does, in order to allow for the stopped car to change its deceleration to $a_{1,2}=0$ so that the differential equation can be followed without violating the evolution domain constraint $v_{1,2} \ge 0$ until the second car stops. The proof strategy again simplifies arithmetic proof obligations by matching terms in the safe distance formula with terms in the solution of the differential equation. Since the distance between the cars is initially equal to safe distance reduced by an arbitrarily small $\varepsilon$, using Theorem 1 we conclude that the minimum safe distance between the two cars is characterized by Eq. (1).    $\square$

Theorem 4

(Opposite Longitudinal Optimality). Model 2 with safeDist as per Eq. (5), freeDriving as per Eq. (6), properResponse as per Eq. (7), and edc as per Eq. (8) is valid.

Proof

The proof strategy follows Theorem 3: we again choose worst case behaviors of both cars, namely both cars driving toward each other at maximum acceleration.    $\square$

4 Refinement from dL to Python

The previous section proves correctness of control decisions in all situations using dL under an assumed model of car dynamics. Increased model fidelity often comes at the price of higher verification effort. For example, if we wanted to consider tire friction, the dynamical motion model would no longer have symbolic closed-form solutions, but would require advanced differential equation reasoning [21]. In order to determine whether a model is sufficiently realistic, and to test concrete implementation behavior in specific situations (e.g., how abruptly a car would brake when encountering obstacles), we now discuss refinement to deterministic implementation and simulation. This provides a way of showing fidelity evidence for models to complement the safety evidence, and to debug unverified models.

In formal models, nondeterminism is beneficial to model a large variety of concrete implementations at an abstract level: nondeterministic choice of acceleration in the controllers, nondeterministic number of loop iterations, and nondeterministic duration of ODEs within each loop iteration capture variation in total driving time and behavior. For execution and testing purposes, deterministic behavior is beneficial to produce repeatable, predictable, and comprehensible results. In order to generate control code, integrate with testing, and ensure that a simulation of a system follows its formal model, we select a deterministic subset of hybrid programs to introduce an intermediate language that can then be directly compiled to Python. This language, det-HP, supports assignment, deterministic choice (defined from nondeterministic choice and tests), and loops (defined from nondeterministic repetition and tests). The deterministic subset det-HP of hybrid programs is generated by the grammar below, where $\theta$ is an arithmetic term in $+,-,\cdot ,/$ and P is a quantifier-free formula in real arithmetic:

$$\begin{aligned} \alpha ,\beta \, {:}{=}\, x \, {:}{=}\, \theta \mid \alpha ;\beta \mid \underbrace{(?P;\alpha )\,\cup \,(?\lnot P;\beta )}_{\text {if}~P~\alpha ~\text {else}~\beta } \mid \underbrace{(?P;\alpha )^*;?\lnot P}_{\text {while}~P~\alpha } \end{aligned}$$

Starting from the formal models in Sect. 3, we need a way of resolving the following points of nondeterminism when examining the behavior of these models: First, the nondeterministic choice between free driving and proper response at the boundary of the safe distance region needs to be resolved (e.g., aggressively by favoring freeDriving, or conservatively by favoring properResponse). Second, the nondeterministic choice of acceleration value in both freeDriving and properResponse must be resolved with a deterministic computation of acceleration within the acceleration bounds (e.g., a bang-bang controller implementation will only choose the values of $a_\text {maxAccel}, a_\text {minBrake},$ and $a_\text {maxBrake}$). Third, the nondeterministic duration of differential equations and hence the nondeterministic control cycle time must be resolved (e.g., conservatively by picking the maximum reaction time $\rho$, but not violating the remaining conditions of the evolution domain constraint). Fourth, the nondeterministic number of repetitions must be resolved (e.g., by picking a test length or a finite amount of time to be followed). Note that the model parameter choices and initial values for variables are not sources of nondeterminism as they are symbolically defined and preset.

We now illustrate det-HP with a deterministic controller freeDriving-det that chooses the same acceleration for every iteration of $\texttt {freeDriving}$ below:

$$\begin{aligned} \texttt {freeDriving-det} \triangleq a_1 \, {:}{=}\, a_\text {maxAccel}; a_2 \, {:}{=}\, -a_\text {maxBrake} \end{aligned}$$

In order to inherit Theorem 1, we use dRL to prove that freeDriving-det reaches a subset of the states of $\texttt {freeDriving}_s$ Eq. (2).

Theorem 5

(Refinement). The deterministic free driving controller refines the RSS free driving control envelope, i.e., $\texttt {freeDriving-det} \le \texttt {freeDriving}_s$ Eq. (2) is valid under assumptions $\varGamma \equiv a_\text {maxAccel}>0 \wedge a_\text {maxBrake}> 0$.

Proof

We prove by refinement in dRL. Let $P \equiv -a_\text {maxBrake}\le a_1 \le a_\text {maxAccel}$ and $Q \equiv -a_\text {maxBrake}\le a_2 \le a_\text {maxAccel}$:

In this proof we utilize the test, sequential and assignment rules [13] to show that each step that the deterministic controller takes is within the bounds that the nondeterministic controller sets. The main insight is that deterministic assignments are refinements of unguarded nondeterministic assignments.    $\square$

The intermediate language det-HP serves as a stepping stone towards compilation to Python. We introduce the compilation rules in Fig. 2, where we use the following subset of the Python expression and statement syntax [8] (for simplicity, we will ignore the difference between $\mathbb {R}$ and Float, which can be addressed with interval arithmetic [3]):

The simplified syntax uses arithmetic operators $\circ \in \{\texttt {+},\texttt {-},\texttt {*},\texttt {/},\texttt {**}\}$ and comparison operators $\bullet \in \{\texttt {<},\texttt {<=},\texttt {>=},\texttt {>},\texttt {==},\texttt {!=}\}$.

Fig. 2.

$\texttt {det-HP}$ to Python compilation rules

The compilation from deterministic hybrid programs to the restricted Python syntax preserves safety, see Lemmas 1–2 and Theorem 6. We abbreviate arithmetic evaluation [8] as $\textsf {eval}(\textsf {Py}(\theta ))_{\nu } \rightarrow u$ to real value u and Boolean evaluation as $\nu (\textsf {bool}~\textsf {Py}(\theta )) \rightarrow v$ to Boolean value v.

Lemma 1 (Term compilation is correct)

Assuming Float = $\mathbb {R}$, terms evaluate equivalently, i.e., if $\textsf {eval}(\textsf {Py}(\theta ))_{\nu } \rightarrow u$ then $\nu \,\llbracket \,\theta \,\rrbracket = u$.

Proof

By structural induction over dL terms, see [24].    $\square$

Lemma 2 (Formula compilation is correct)

Formulas evaluate equivalently, i.e., $\nu \models \phi$ iff $\nu (\textsf {bool}~\textsf {Py}(\phi )) \rightarrow \texttt {true}$.

Proof

By structural induction over dL formulas, see [24].    $\square$

Theorem 6

(Compilation of det-HP to Python is correct). All states that are reachable with the compiled Python program are also reachable by the source HP program, i.e. $(\textsf {Py}(\alpha ), \nu ) \rightarrow (\perp , \omega )$ then $(\nu ,\omega ) \in \llbracket \alpha \, \rrbracket$.

Proof

By structural induction over hybrid programs, see [24].    $\square$

Theorem 6 ensures that any state reached in a compiled Python program will also be reachable in the source $\texttt {det-HP}$ program. Together with the refinement step from our original model, we therefore know that any counterexample found in the compiled simulation will also be a counterexample of the original formal dL model.

5 Testing and Monitoring by Example

The previous sections introduced safety verification and refinement proofs that eliminate the need for correctness testing. These two steps, if completed, assure that the system is safe under all implementations. In this section, we consider testing and simulation as a method to provide insight for correcting the formal model and/or an implementation when either of these steps fails, and as a way to assess the fidelity of the formal model and experiment with model parameter selection once the proofs are completed.

5.1 Monitor Structure

In order to assess whether the simulation and the formal model agree and flag differences, we utilize ModelPlex [17]. ModelPlex monitors are formally verified and derived automatically from a hybrid systems model; they flag whenever states encountered at runtime (e.g., in simulation) disagree with the expectations of the formal model. For this, the monitors collect the relationship between starting and final states along all possible execution paths of the formal model. For example, a monitor derived from the discrete statements of Model 1 as instantiated in Theorem 1 is illustrated below (variables of the initial state are x, y, z, variables of the final state are $x^+,y^+,z^+$).

$$\begin{aligned}&(\texttt {safeDist}_s(v_1,v_2) \le x_2-x_1 \wedge -a_\text {maxBrake}\le a_1^+ \le a_\text {maxAccel}\\&\qquad \wedge -a_\text {maxBrake}\le a_2^+ \le a_\text {maxAccel}\wedge t^+=0 \wedge \texttt {edc}_s)\\ \vee&(\texttt {safeDist}_s(v_1,v_2) \ge x_2-x_1 \wedge (a_1^+ \le -a_\text {minBrake}\vee (v_1=0 \wedge a_1^+=0))\\&\qquad \wedge (a_2^+ \ge a_\text {maxBrake}\vee (v_2=0 \wedge a_2^+=0)) \wedge t^+=0 \wedge \texttt {edc}_s) \end{aligned}$$

Fig. 3.

Leader-follower same direction test scenario, with rear car control implementation violating the conditions of RSS. Plots from top to bottom are: ModelPlex monitor verdict, car positions $x_1,x_2$, car speeds $v_1,v_2$, and accelerations $a_1,a_2$.

5.2 Examples of Monitor Usage

Monitoring for Faulty Implementation. Our first example illustrates the use of testing and monitoring when the refinement proof cannot be completed due to an incorrect implementation. With user-defined test scenarios, ModelPlex monitors in simulation highlight the vulnerabilities of the controller implementation as an intermediate step towards finding implementation fixes and completing the refinement proof. In this case, counterexamples found in simulation represent control decisions of the unverified controller implementation that violate the expectations of the formal model. In Fig. 3, a front and a rear car are driving in the same direction, while the rear car follows a faulty controller. The user-defined test scenario is an example of a boundary test by picking worst case behavior where the rear car is accelerating at maximum acceleration whenever allowed by its control guards. The ModelPlex monitor derived from Model 1 with the formulas and programs of Theorem 1 fails at the point of violating RSS: executing the proper response instead of the faulty controller at time time of the monitor alarm would have prevented collision, but following the faulty controller makes the rear car exceed the front car’s position (i.e., cause a collision). The monitor verdict (“ModelPlex ID” in Fig. 3) flags the violated condition of the monitor, which directly relates to a path through the formal model: the faulty controller allowed to continue accelerating forward for one more time step when it should have been following the proper response already. In this case, a boundary test reveals the controller implementation flaw, but other implementation bugs are more subtle and require extensive search. For future work, we envision integration of monitors with falsification techniques.

Monitoring for Modeling Flaws. With a setup similar to monitoring for faulty implementations, we address modeling flaws in unverified models. User-defined test scenarios allow us to examine the behavior of the formal model through monitors: the formal model is faulty when finding scenarios that end in collision without a prior ModelPlex monitor alarm. In this case, the insight for fixing the bug is not merely the (missing) monitor alarm itself, but the sequence of control actions that lead up to the collision.

Fig. 4.

Car behavior in opposite direction model at $\rho =1$ (left) and $\rho = 6$ (right).

Parameter Selection. The last example we include illustrates how to use tests for parameter selection once the formal model and the refinement to a concrete controller implementation is proved. The concrete choices for the symbolic parameters, such as $a_\text {maxBrake}$, $a_\text {minBrake}$, $a_\text {maxAccel}$, and $\rho$, heavily influence the specific behavior of the controller implementation. Simulation with specific parameter choices is a useful technique to inspect the expected real-world behavior, and completes the circle of interaction between formal methods and testing. Formal methods guarantee correctness in a model of physics, refinement proofs and correct compilation guarantee correctness of implementations, and simulation and testing allow for inspecting the fidelity of the formal model and the concrete behavior of the implementation under specific parameter choices. In Fig. 4 we consider parameter selection and its effect on the behavior of cars in the opposite direction scenario. The controllers of both cars in free driving accelerate towards each other at maximum acceleration and in proper response they both brake at minimum rate. With a “short” response horizon of $\rho =1$, the final distance between the cars once fully stopped is much lower than when a response $\rho =6$.

6 Related Work

RSS introduced in [23] has been deployed by the Mobileye research group as the basis for their self-driving car and analyzed extensively from different aspects for its safety and reliablity: [18] considers parameter analysis and concrete behavior of RSS on German highways; [9] extends RSS fidelity by applying Newtonian mechanics to analyze car behavior on slopes and in varying road conditions.

Related work within formal verification of driving models is extensive: [16] utilize a formula equivalent to the same direction safe distance formula introduced by RSS for an adaptive cruise control system, [14] analyze collision avoidance while following Dubin’s paths, [4] and [10] formally verify waypoint navigation, [1] verify swerving maneuvers to avoid collision, [22] develops a formally verified motion planner, [2, 11] use reachability analysis in highway settings and introduce a toolbox to create simulated driving environments to determine reachable sets, [7] is close to our work in that it uses falsification to find mistakes in motion plans of autonomous vehicles. We complement these approaches by providing proofs of optimality of the RSS safety conditions and proofs of opposite driving direction, combined with refinement proofs and correct compilation to Python for testing.

7 Conclusion and Future Work

In this paper we formalized and proved safety as well as optimality of the Responsibility-Sensitive Safety control envelopes in both longitudinal directions. The safety proofs discover loop invariants, which concisely document design properties of the system. These properties of the longitudinal cases do not transfer to the lateral case in a straightforward way, because velocity direction changes do not require an intermediate stop such as a gear change; formalizing and verifying the lateral case is an area of future research.

To complement the safety guarantees of formal verification with practical tools to test and debug unverified models, as well as to assess the fidelity of verified models in specific driving situations and under specific parameter choices, we introduce refinement proofs and correct compilation from nondeterministic models to deterministic implementations in Python. Formal methods guarantee correctness in a model of physics, refinement proofs and correct compilation guarantee correctness of implementations, and simulation and testing allow for inspecting the fidelity of the formal model and the concrete behavior of the implementation under specific parameter choices. Future research includes combination with falsification methods to provide a systematic automated approach to testing of unverified models in differential dynamic logic.