Statistical verification of autonomous system controllers under timing uncertainties

Abstract

Software in autonomous systems like autonomous cars, robots or drones is often implemented on resource-constrained embedded systems with heterogeneous architectures. At the heart of such software are multiple feedback control loops, whose dynamics not only depend on the control strategy being used, but also on the timing behavior the control software experiences. But performing timing analysis for safety critical control software tasks, particularly on heterogeneous computing platforms, is challenging. Consequently, a number of recent papers have addressed the problem of stability analysis of feedback control loops in the presence of timing uncertainties (cf., deadline misses). In this paper, we address a different class of safety properties, viz., whether the system trajectory with timing uncertainties deviates too much from the nominal trajectory. Verifying such quantitative safety properties involves performing a reachability analysis that is computationally intractable, or is too conservative. To alleviate these problems we propose to provide statistical guarantees over the behavior of control systems with timing uncertainties. More specifically, we present a Bayesian hypothesis testing method that estimates deviations from a nominal or ideal behavior. We show that our analysis can provide, with high confidence, tighter estimates of the deviation from nominal behavior than using known reachability analysis methods. We also illustrate the scalability of our techniques by obtaining bounds in cases where reachability analysis fails, thereby establishing the practicality of our proposed method.

Appendices

Appendix 1

A case study: the impact of prior selection

The methodology presented in Sect. 4 assumes a uniform prior distribution. Specifically, we assumed

$$\begin{aligned} \mathcal {P}\! rob \left[ \mathcal {T},x[0], {\tau }_{ nom },\textbf{d}_ ub \right] = \theta \sim \text{ Uniform }(0,1). \end{aligned}$$

However, the number of samples required to conclude that $\theta \ge 0.99$ with sufficiently high probability depends heavily on the choice of prior. Our choice of a uniform prior feels safe in that it does not assume any prior knowledge of the underlying system. A variety of more nuanced procedures exist for selecting so-called objective priors Ghosh (2011) that avoid some of the pitfalls of using a uniform prior. Plugging these priors into our procedure is straightforward, and a full discussion of objective priors is beyond the scope of this paper.

To illustrate the effect of prior knowledge on our testing procedure, we now consider how the choice of prior impacts the number of samples required by our hypothesis testing procedure. Recall that our hypothesis testing procedure boiled down to estimating the value of $\theta$ given that each sampled trajectory would independently obey $\textbf{d}_ ub$ with probability $\theta$. Specifically, we take K samples and then try to estimate $\theta$ given that $\text {Binomial}(K,\theta )=K$. It is well known that the beta distribution is a conjugate prior of binomial likelihood functions—any beta prior distribution and a binomial likelihood function will induce a beta posterior distribution on $\theta$Chen and Novick (1984). Hence, to examine the impact of the choice of prior, we consider two alternate beta distributions as priors.

We compare these choices of prior to the uniform prior used in the paper. For this comparison, we again assume that the user-defined parameter $c=0.99$. That is, we would like to say whether a given $\textbf{d}_ ub$ is at least a 99th percentile of the distribution of trajectory deviations. We consider beta prior distributions with modes at $\theta =0.25$, and $\theta =0.99$ respectively.⁴ These priors reflect two common cases. First, the prior with a peak at $\theta =0.99$ reflects the case where one strongly believes that $\theta$ is close to 0.99, but is also agnostic as to whether the true value of $\theta$ is above or below 0.99. Second, the prior with a peak a $\theta =0.25$ reflects the case where one chooses a prior conservatively to ensure the safety of the system. These priors are shown in Fig. 11.

Fig. 11

A variety of prior distributions that can be plugged into our hypothesis testing procedure. We use beta prior distributions because they emit simple posterior distributions when used with a binomial likelihood function. We consider the uniform prior used in Sect. 4 as well as a prior with a mode of $\theta =0.25$ and a prior with a mode of $\theta =0.99$

Based on each of these priors, we can compute the posterior distribution induced by sampling K trajectories which all obey a given value of $\textbf{d}_ ub$. Figure 12 shows the probability of a type-I error, $\alpha$, as a function of the number of samples, K. We see that the choice of prior significantly impacts the number of samples required by our hypothesis testing procedure. First, we can see that using a prior with a peak at $\theta =0.25$ has the intended effect. Meeting a given level of $\alpha$ when using this prior requires roughly twice as many samples as were required when using a uniform prior. Surprisingly, the prior with a peak at $\theta =0.99$ does not have the opposite effect. Given a strong prior belief that $\theta$ is close to 0.99, we might expect to require fewer samples than when using a uniform prior. Figure 12 shows that, although $\alpha$ is slightly lower for small values of K, we actually require more samples when using the prior with a peak at.99 than were required when using a uniform prior. The issue is that this prior assigns significant probability density to values slightly above and slightly below 0.99. A large number of samples is then required to decide whether $\theta$ is actually above 0.99 or just slightly below 0.99. Hence, while the uniform prior is in some sense non-informative, both of the alternate priors shown here are more conservative in the number of samples they require before allowing us to accept a given value of $\textbf{d}_ ub$.

Fig. 12

Type-I error, $\alpha$, as a function of the sample size K under a variety of beta prior distributions. Using a prior with a mode of 0.25 has the expected effect of requiring more samples than are required when using a uniform prior. Surprisingly, using a prior with a mode of 0.99 also requires more samples than are required when using a uniform prior