\(\mathsf {StocHy}\): Automated Verification and Synthesis of Stochastic Processes
Abstract
\(\mathsf {StocHy}\) is a software tool for the quantitative analysis of discretetime stochastic hybrid systems (shs). \(\mathsf {StocHy}\) accepts a highlevel description of stochastic models and constructs an equivalent shs model. The tool allows to (i) simulate the shs evolution over a given time horizon; and to automatically construct formal abstractions of the shs. Abstractions are then employed for (ii) formal verification or (iii) control (policy, strategy) synthesis. \(\mathsf {StocHy}\) allows for modular modelling, and has separate simulation, verification and synthesis engines, which are implemented as independent libraries. This allows for libraries to be easily used and for extensions to be easily built. The tool is implemented in c++ and employs manipulations based on vector calculus, the use of sparse matrices, the symbolic construction of probabilistic kernels, and multithreading. Experiments show \(\mathsf {StocHy}\)’s markedly improved performance when compared to existing abstractionbased approaches: in particular, \(\mathsf {StocHy}\) beats stateoftheart tools in terms of precision (abstraction error) and computational effort, and finally attains scalability to largesized models (12 continuous dimensions). \(\mathsf {StocHy}\) is available at www.gitlab.com/natchi92/StocHy. Data or code related to this paper is available at: [31].
1 Introduction
Stochastic hybrid systems (shs) are a rich mathematical modelling framework capable of describing systems with complex dynamics, where uncertainty and hybrid (that is, both continuous and discrete) components are relevant. Whilst earlier instances of shs have a long history, shs proper have been thoroughly investigated only from the mid 2000s, and have been most recently applied to the study of complex systems, both engineered and natural. Amongst engineering case studies, shs have been used for modelling and analysis of micro grids [29], smart buildings [23], avionics [7], automation of medical devices [3]. A benchmark for shs is also described in [10]. However, a wider adoption of shs in realworld applications is stymied by a few factors: (i) the complexity associated with modelling shs; (ii) the generality of their mathematical framework, which requires an arsenal of advanced and diverse techniques to analyse them; and (iii) the undecidability of verification/synthesis problems over shs and the curse of dimensionality associated with their approximations.
This paper introduces a new software tool  \(\mathsf {StocHy}\)  which is aimed at simplifying both the modelling of shs and their analysis, and which targets the wider adoption of shs, also by nonexpert users. With focus on the three limiting factors above, \(\mathsf {StocHy}\) allows to describe shs by parsing or extending wellknown and used statespace models and generates a standard shs model automatically and formats it to be analysed. \(\mathsf {StocHy}\) can (i) perform verification tasks, e.g., compute the probability of staying within a certain region of the state space from a given set of initial conditions; (ii) automatically synthesise policies (strategies) maximising this probability, and (iii) simulate the shs evolution over time. \(\mathsf {StocHy}\) is implemented in c++ and modular making it both extendible and portable.
Related work. There exist only a few tools that can handle (classes of) shs. Of much inspiration for this contribution, \(\textsc {faust}^2\) [28] generates abstractions for uncountablestate discretetime stochastic processes, natively supporting shs models with a single discrete mode and finite actions, and performs verification of reachabilitylike properties and corresponding synthesis of policies. \(\textsc {faust}^2\) is naïvely implemented in matlab and lacks in scalability to large models. The modest toolset [18] allows to model and to analyse classes of continuoustime shs, particularly probabilistic hybrid automata (pha) that combine probabilistic discrete transitions with deterministic evolution of the continuous variables. The tool for stochastic and dynamically coloured petri nets (sdcpn) [13] supports compositional modelling of pha and focuses on simulation via Monte Carlo techniques. The existing tools highlight the need for a new software that allows for (i) straightforward and general shs modelling construction and (ii) scalable automated analysis.
 formal verification of shs via either of two abstraction techniques:

for discretetime, continuousspace models with additive disturbances, and possibly with multiple discrete modes, we employ formal abstractions as general Markov chains or Markov decision processes [28]; \(\mathsf {StocHy}\) improves techniques in the \(\textsc {faust}^2\) tool by simplifying the input model description, by employing sparse matrices to manipulate the transition probabilities and by reducing the computational time needed to generate the abstractions.

for models with a finite number of actions, we employ interval Markov decision processes and the model checking framework in [22]; \(\mathsf {StocHy}\) provides a novel abstraction algorithm allowing for efficient computation of the abstract model, by means of an adaptive and sequential refining of the underlying abstraction. We show that we are able to generate significantly smaller abstraction errors and to verify models with up to 12 continuous variables.

 control (strategy, policy) synthesis via formal abstractions, employing:

stochastic dynamic programming; \(\mathsf {StocHy}\) exploits the use of symbolic kernels.

robust synthesis using interval Markov decision processes; \(\mathsf {StocHy}\) automates the synthesis algorithm with the abstraction procedure and the temporal property of interest, and exploits the use of sparse matrices;


simulation of complex stochastic processes, such as shs, by means of Monte Carlo techniques; \(\mathsf {StocHy}\) automatically generates statistics from the simulations in the form of histograms, visualising the evolution of both the continuous random variables and the discrete modes.
This contribution is structured as follows: Sect. 2 crisply presents the theoretical underpinnings (modelling and analysis) for the tool. We provide an overview of the implementation of \(\mathsf {StocHy}\) in Sect. 3. We highlight features and use of \(\mathsf {StocHy}\) by a set of experimental evaluations in Sect. 4: we provide four different case studies that highlight the applicability, ease of use, and scalability of \(\mathsf {StocHy}\). Details on executing all the case studies are detailed in this paper and within a Wiki page that accompanies the \(\mathsf {StocHy}\) distribution.
2 Theory: Models, Abstractions, Simulations
2.1 Models  DiscreteTime Stochastic Hybrid Systems
\(\mathsf {StocHy}\) supports the modelling of the following general class of shs [1, 4].
Definition 1

\(\mathcal {Q} = \{q_{1},q_{2}, \dots ,q_{m}\}\), \(m\in \mathbb {N},\) represents a finite set of modes (locations);

\(n\in \mathbb {N}\) is the dimension of the continuous space \(\mathbb {R}^{n}\) of each mode; the hybrid state space is then given by \(\mathcal {D}= \cup _{q\in \mathcal {Q}}\{q\} \times \mathbb {R}^{n} \);

\(\mathcal {U}\) is a continuous set of actions, e.g. \(\mathbb {R}^{v}\);

\(T_{q}: \mathcal {Q} \times \mathcal {D}\times \mathcal {U} \rightarrow [0,1]\) is a discrete stochastic kernel on Q given \(\mathcal {D}\times \mathcal {U}\), which assigns to each \(s=(q,x) \in \mathcal {D}\) and \(u \in \mathcal {U}\), a probability distribution over \(\mathcal {Q}: T_{q}(\cdot s,u)\);

\(T_{x}: \mathcal B(\mathbb {R}^{n}) \times \mathcal {D}\times \mathcal {U} \rightarrow [0,1]\) is a Borelmeasurable stochastic kernel on \(\mathbb {R}^{n}\) given \(\mathcal {D}\times \mathcal {U}\), which assigns to each \(s \in \mathcal {D}\) and \(u \in \mathcal {U}\) a probability measure on the Borel space \((\mathbb {R}^{n}, \mathcal B(\mathbb {R}^{n})): T_{x}( \cdot s,u)\).
In this model the discrete component takes values in a finite set \(\mathcal {Q}\) of modes (a.k.a. locations), each endowed with a continuous domain (the Euclidean space \(\mathbb {R}^{n}\)). As such, a point s over the hybrid state space \(\mathcal {D}\) is pair (q, x), where \(q \in \mathcal {Q}\) and \(x \in \mathbb {R}^{n}\). The semantics of transitions at any point over a discrete time domain, are as follows: given a point \(s \in \mathcal {D}\), the discrete state is chosen from \(T_{q}\), and depending on the selected mode \(q \in \mathcal {Q}\) the continuous state is updated according to the probabilistic law \(T_{x}\). Nondeterminism in the form of actions can affect both discrete and continuous transitions.
Remark 1
A rigorous characterisation of shs can be found in [1], which introduces a general class of models with probabilistic resets and a hybrid actions space. Whilst we can deal with general shs models, in the case studies of this paper we focus on special instances, as described next. \(\square \)
Remark 2
(Special instance). In Case Study 2 (see Sect. 4.2) we look at models where actions are associated to a deterministic selection of locations, namely \(T_{q}: \mathcal {U} \rightarrow \mathcal {Q}\) and \(\mathcal {U}\) is a finite set of actions. \(\square \)
Remark 3
Definition 2

\(\mathcal {Q} = \{q_{1},q_{2}, \dots ,q_{m}\}\), \(m\in \mathbb {N},\) represents a finite set of modes;

\(\mathcal {U}\) is a finite set of actions;

\(T_{q}: \mathcal {Q} \times \mathcal {Q} \times \mathcal {U} \rightarrow [0,1]\) is a discrete stochastic kernel that assigns, to each \(q \in \mathcal {Q}\) and \(u \in \mathcal {U}\), a probability distribution over \(\mathcal {Q}: T_{q}(\cdot q,u)\).
Whenever the set of actions is trivial or a policy is synthesised and used (cf. discussion in Sect. 2.2) the mdp reduces to a Markov chain (mc), and a kernel \(T_{q}: \mathcal {Q} \times \mathcal {Q} \rightarrow [0,1]\) assigns to each \(q \in \mathcal {Q}\) a distribution over \(\mathcal {Q}\) as \(T_q(\cdot q)\).
Definition 3

\(\mathcal {Q}\) and \(\mathcal {U}\) are as in Definition 2;

\(\check{P}\) and \(\hat{P}: \mathcal {Q} \times \mathcal {U} \times \mathcal {Q} \rightarrow [0,1]\) is a function that assigns to each \(q \in \mathcal {Q}\) a lower (upper) bound probability distribution over \(\mathcal {Q}: \check{P}(\cdot q,u)\) \((\hat{P}(\cdot q,u)\) respectively).
2.2 Formal Verification and Strategy Synthesis via Abstractions
Formal verification and strategy synthesis over shs are in general not decidable [4, 30], and can be tackled via quantitative finite abstractions. These are precise approximations that come in two main different flavours: abstractions into mdp [4, 28] and into imdp [22]. Once the finite abstractions are obtained, and with focus on specifications expressed in (nonnested) pctl or fragments of ltl [5], formal verification or strategy synthesis can be performed via probabilistic model checking tools, such as prism [21], storm [12], iscasMc [17]. We overview next the two alternative abstractions, as implemented in \(\mathsf {StocHy}\).
Abstractions into Markov decision processes. Following [27], mdp are generated by either (i) uniformly gridding the state space and computing an abstraction error, which depends on the continuity of the underlying continuous dynamics and on the chosen grid; or (ii) generating the grid adaptively and sequentially, by splitting the cells with the largest local abstraction error until a desired global abstraction error is achieved. The two approaches display an intuitive tradeoff, where the first in general requires more memory but less time, whereas the second generates smaller abstractions. Either way, the probability to transit from each cell in the grid into any other cell characterises the mdp matrix \(T_q\). Further details can be found in [28]. \(\mathsf {StocHy}\) newly provides a c++ implementation and employs sparse matrix representation and manipulation, in order to attain faster generation of the abstraction and use in formal verification or strategy synthesis.
Verification via mdp (when the action set is trivial) is performed to check the abstraction against nonnested, boundeduntil specifications in pctl [5] or cosafe linear temporal logic (csltl) [20].
Strategy synthesis via mdp is defined as follows. Consider, the class of deterministic and memoryless Markov strategies \(\pi = (\mu _0, \mu _1, \dots )\) where \(\mu _k: \mathcal {Q} \rightarrow \mathcal {U}\). We compute the strategy \(\pi ^\star \) that maximises the probability of satisfying a formula, with algorithms discussed in [28].
Abstraction into Interval Markov decision processes (imdp) is based on a procedure in [11] performed using a uniform grid and with a finite set of actions \(\mathcal {U}\) (see Remark 2). \(\mathsf {StocHy}\) newly provides the option to generate a grid using adaptive/sequential refinements (similar to the case in the paragraph above) [27], which is performed as follows: (i) define a required minimal maximum abstraction error \(\varepsilon _{max}\); (ii) generate a coarse abstraction using the Algorithm in [11] and compute the local error \(\varepsilon _q\) that is associated to each abstract state q; (iii) split all cells where \(\varepsilon _q > \varepsilon _{max}\) along the main axis of each dimension, and update the probability bounds (and errors); and (iv) repeat this process until \(\forall q, \; \varepsilon _{q} < \varepsilon _{max}\).
Verification via imdp is run over properties in csltl or boundedLTL (bltl) form using the imdp model checking algorithm in [22].
Synthesis via imdp [11] is carried out by extending the notions of strategies of mdp to depend on memory, that is on prefixes of paths.
2.3 Analysis via Monte Carlo Simulations
Monte Carlo techniques generate numerical sampled trajectories representing the evaluation of a stochastic process over a predetermined time horizon. Given a sufficient number of trajectories, one can approximate the statistical properties of the solution process with a required confidence level. This approach has been adopted for simulation of different types of shs. [19] applies sequential Monte Carlo simulation to shs to reason about rareevent probabilities. [13] performs Monte Carlo simulations of classes of shs described as Petri nets. [8] proposes a methodology for efficient Monte Carlo simulations of continuoustime shs. In this work, we analyse a shs model using Monte Carlo simulations following the approach in [4]. Additionally, we generate histogram plots at each time step, providing further insight on the evolution of the solution process.
3 Overview of \(\mathsf {StocHy}\)
Installation. \(\mathsf {StocHy}\) is set up using the provided get_dep file found within the distribution package, which will automatically install all the required dependencies. The executable run.sh builds and runs \(\mathsf {StocHy}\). This basic installation setup has been successfully tested on machines running Ubuntu 18.04.1 LTS GNU and Linux operating systems.
Input interface. The user interacts with \(\mathsf {StocHy}\) via the main file and must specify (i) a highlevel description of the model dynamics and (ii) the task to be performed. The description of model dynamics can take the form of a list of the transition probabilities between the discrete modes, and of the statespace models for the continuous variables in each mode; alternatively, a description can be obtained by specifying a path to a matlab file containing the model description in statespace form together with the transition probability matrix. Tasks can be of three kinds (each admitting specific parameters): simulation, verification, or synthesis. The general structure of the input interface is illustrated via an example in Listing 1.1: here the user is interested in simulating a shs with two discrete modes \(\mathcal {Q} = \{q_0,q_1\}\) and two continuous variables evolve according to (3). The model is autonomous and has no control actions. The relationship between the discrete modes is defined by a fixed transition probability (line 1). The evolution of the continuous dynamics are defined in lines 2–14. The initial condition for both the discrete modes and the continuous variables are set in lines 16–21 (this is needed for simulation tasks). The equivalent shs model is then set up by instantiating an object of type Open image in new window (line 23). Next, the task is defined in line 27 (simulation with a time horizon \(K= 32\), as specified in line 25 and using the simulator library, as set in line 26). We combine the model and task specification together in line 29. Finally, \(\mathsf {StocHy}\) carries out the simulation using the function performTask (line 31).
Modularity. \(\mathsf {StocHy}\) comprises independent libraries for different tasks, namely (i) \(\textsc {faust}^2\), (ii) imdp, and (iii) simulator. Each of the libraries is separate and depends only on the model structure that has been entered. This allows for seamless extensions of individual submodules with new or existing tools and methods. The function performTask acts as multiplexer for calling any of the libraries depending on the input model and task specification.
Data structures. \(\mathsf {StocHy}\) makes use of multiple techniques to minimise computational overhead. It employs vector algebra for efficient handling of linear operations, and whenever possible it stores and manipulates matrices as sparse structures. It uses the linear algebra library Armadillo [24, 25], which applies multithreading and a sophisticated expression evaluator that has been shown to speed up matrix manipulations in c++ when compared to other libraries. \(\textsc {faust}^2\) based abstractions define the underlying kernel functions symbolically using the library GiNaC [6], for easy evaluation of the stochastic kernels.
Output interface. We provide outputs as text files for all three libraries, which are stored within the results folder. We also provide additional python scripts for generating plots as needed. For abstractions based on \(\textsc {faust}^2\), the user has the additional option to export the generated mdp or mc to prism format, to interface with the popular model checker [21] (\(\mathsf {StocHy}\) prompts the user this option following the completion of the verification or synthesis task). As a future extension, we plan to export the generated abstraction models to the model checker storm [12] and to the modelling format jani [9].
4 \(\mathsf {StocHy}\): Experimental Evaluation
We apply \(\mathsf {StocHy}\) on four different case studies highlighting different models and tasks to be performed. All the experiments are run on a standard laptop, with an Intel Core i78550U CPU at 1.80 GHz \(\times \) 8 and with 8 GB of RAM.
4.1 Case Study 1  Formal Verification
We are interested in verifying whether the continuous variables remain within the safe set \(X_{safe} =[405, 540] \times [18,24]\) over 45 min (\(K = 3\)). This property can be encoded as a bltl property, \(\varphi _1 := \square ^{\le K} X_{safe},\) where \(\square \) is the “always” temporal operator considered over a finite horizon. The semantics of bltl is defined over finite traces, denoted by \(\zeta = \{\zeta _j\}_{j=0}^{K}\). A trace \(\zeta \) satisfies \(\varphi _1\) if \(\forall j \le K, \zeta _j \in X_{safe}\), and we quantify the probability that traces generated by the shs satisfy \(\varphi _1\).
When tackled with the method based on \(\textsc {faust}^2\) that hinges on the computation of Lipschitz constants, this verification task is numerically tricky, in view of difference in dimensionality of the range of \(x_1\) and \(x_2\) within the safe set \(X_{safe}\) and the variance associated with each dimension \(G_{q_0}= [{\begin{matrix} \sigma _{1} &{} 0 \\ 0 &{} \sigma _{2} \\ \end{matrix}}] = [{\begin{matrix} 40.096 &{}0 \\ 0 &{} 0.511\\ \end{matrix}}]\). In order to mitigate this, \(\mathsf {StocHy}\) automatically rescales the state space so all the dynamics evolve in a comparable range.
Implementation. \(\mathsf {StocHy}\) provides two verification methods, one based on \(\textsc {faust}^2\) and the second based on imdp. We parse the model from file cs1.mat (see line 2 of Listings 1.2(a) and 1.3(b), corresponding to the two methods). cs1.mat sets parameter values to (6) and uses a \(\varDelta =\) 15 [min]. As anticipated, we employ both techniques over the same model description:

for \(\textsc {faust}^2\) we specify the safe set (\(X_{safe}\)), the maximum allowable error, the grid type (whether uniform or adaptive grid), the time horizon, together with the type of property of interest (safety or reachavoid). This is carried out in lines 5–21 in Listing 1.2(a).

for the imdp method, we define the safe set (\(X_{safe}\)), the grid size, the relative tolerance, the time horizon and the property type. This can be done by defining the task specification using lines 5–21 in Listing 1.3(b).

for \(\textsc {faust}^2\), \(\mathsf {StocHy}\) generates four text files within the results folder: representative_points.txt contains the partitioned state space; transition_matrix.txt consists of the transition probabilities of the generated abstract mc; problem_solution.txt contains the sat probability for each state of the mc; and e.txt stores the global maximum abstraction error.

for imdp, \(\mathsf {StocHy}\) generates three text files in the same folder: stepsmin.txt stores \(\check{P}\) of the abstract imdp; stepsmax.txt stores \(\hat{P}\); and solution.txt contains the sat probability and the errors \(\varepsilon _q\) for each abstract state q.
Case study 1: Comparison of verification results for \(\varphi _1\) when using \(\textsc {faust}^2\) vs imdp.
Tool method  Impl. platform  \({{\mathcal {Q}}}\) [states]  Time [s]  Error \(\varepsilon _{\max }\) 

\(\textsc {faust}^2\)  matlab  576  186.746  1 
\(\textsc {faust}^2\)  c++  576  51.420  1 
imdp  c++  576  87.430  0.236 
\(\textsc {faust}^2\)  matlab  1089  629.037  1 
\(\textsc {faust}^2\)  c++  1089  78.140  1 
imdp  c++  1089  387.940  0.174 
\(\textsc {faust}^2\)  matlab  2304  2633.155  1 
\(\textsc {faust}^2\)  c++  2304  165.811  1 
imdp  c++  2304  1552.950  0.121 
\(\textsc {faust}^2\)  matlab  3481  7523.771  1 
\(\textsc {faust}^2\)  c++  3481  946.294  1 
imdp  c++  3481  3623.090  0.098 
\(\textsc {faust}^2\)  matlab  4225  10022.850  0.900 
\(\textsc {faust}^2\)  c++  4225  3313.990  0.900 
imdp  c++  4225  4854.580  0.089 
4.2 Case Study 2  Strategy Synthesis
and \(i\in \{0,1\}\). Consider the continuous domain shown in Fig. 2a over both discrete locations. We plan to synthesise the optimal switching strategy \(\pi ^\star \) that maximises the probability of reaching the green region, whilst avoiding the purple one, over an unbounded time horizon, given any initial condition within the domain. This can be expressed with the ltl formula, \(\varphi _2 := (\lnot purple) \;\mathsf {U}\;green,\) where \(\mathsf {U}\) is the “until” temporal operator, and the atomic propositions \(\{purple, green\}\) denote regions within the set \(X = [1.5, 1.5]^2\) (see Fig. 2a).
Implementation. We define the model dynamics following lines 3–14 in Listing 1.1, while we use Listing 1.3 to specify the synthesis task and together with its associated parameters. The ltl property \(\varphi _2\) is over an unbounded time horizon, which leads to employing the imdp method for synthesis (recall that the \(\textsc {faust}^2\) implementation can only handle timebounded properties, and its abstraction error monotonically increases with the time horizon of the formula). In order to encode the task we set the variable safe to correspond to X the grid size to 0.12 and the relative tolerance to 0.06 along both dimensions (cf. lines 5–10 in Listing 1.3). We set the time horizon K = 1 to represent an unbounded time horizon, let p = 4 to trigger the synthesis engine over the given specification and make lb = 3 to use imdp method (cf. lines 12–19 in Listing 1.3). This task specification partitions the set X into the underlying imdp via uniform gridding. Alternatively, the user has the option to make use of the adaptivesequential algorithm by defining a new variable eps_max which characterise the maximum allowable abstraction error and then specify the task using taskSpec_t mySpec(lb,K,p,boundary,eps_max,grid,rtol);. Next, we define two files (phi1.txt and phi2.txt) containing the coordinates within the gridded domain (see Fig. 2a) associated with the atomic propositions purple and green, respectively. This allows for automatic labelling of the statespace over which synthesis is to be performed. Running the main file, \(\mathsf {StocHy}\) generates a Solution.txt file within the results folder. This contains the synthesised \(\pi ^\star \) policy, the lower bound for the probabilities of satisfying \(\varphi _2\), and the local errors \(\varepsilon _q\) for any region q.
Outcomes. The case study generates an abstraction with a total of 2410 states, a maximum probability of 1, a maximum abstraction error of 0.21, and it requires a total time of 1639.3 [s]. In this case, we witness a slightly larger abstraction error via the imdp method then in the previous case study. This is due the nondiagonal covariance matrix \(G_{q_0}\) which introduces a rotation in X within mode \(q_0\). When labelling the states associated with the regions purple and green, an additional error is introduced due to the over and underapproximation of states associated with each of the two regions. We further show the simulation of a trajectory under \(\pi ^\star \) with a starting point of \((0.5,1)\) in \(q_0\), within Fig. 2a.
4.3 Case Study 3  Scaling in Continuous Dimension of Model
We now focus on the continuous dynamics by considering a stochastic process with \(\mathcal {Q} =\{q_0\}\) (single mode) and dynamics evolving according to (3), characterised by \(A_{q_0} = 0.8\mathbf {I}_{d}\), \(F_{q_0} = \mathbf 0 _d\) and \(G_{q_0}= 0.2\mathbf {I}_{d}\), where d corresponds to the number of continuous variables. We are interested in checking the ltl specification \(\varphi _3 := \square X_{safe}\), where \(X_{safe}= [1,1]^{d}\), as the continuous dimension d of the model varies. Here “\(\square \)” is the “always” temporal operator and a trace \(\zeta \) satisfies \(\varphi _3\) if \(\forall k \ge 0,\; \zeta _k \in X_{safe}\). In view of the focus on scalability for this Case Study 3, we disregard discussing the computed probabilities, which we instead covered in Sect. 4.1.
Case study 3: Verification results of the imdpbased approach over \(\varphi _3\), for varying dimension d of the stochastic process.
Dimensions \([\text {d}]\)  2  3  4  5  6  7  8  9  10  11  12 
\({\mathcal {Q}}\) [states]  4  14  30  62  126  254  510  1022  2046  4094  8190 
Time taken [s]  0.004  0.06  0.21  0.90  4.16  19.08  79.63  319.25  1601.31  5705.47  21134.23 
Error \((\varepsilon _{max})\)  4.15e5  3.34e5  2.28e5  9.70e5  8.81e6  1.10e6  2.95e6  4.50e7  1.06e7  4.90e8  4.89e8 
Outcomes. From Table 2 we can deduce that by employing the imdp method within \(\mathsf {StocHy}\), the generated abstract models have manageable state spaces, thanks to the tight error bounds that is obtained. Notice that since the number of cells per dimension is increased with the dimension d of the model, the associated abstraction error \(\varepsilon _{max}\) is decreased. The small error is also due to the underlying contractive dynamics of the process. This is a key fact leading to scalability over the continuous dimension d of the model: \(\mathsf {StocHy}\) displays a significant improvement in scalability over the state of the art [28] and allows abstracting stochastic models with relevant dimensionality. Furthermore, \(\mathsf {StocHy}\) is capable to handle specifications over infinite horizons (such as the considered until formula).
4.4 Case Study 4  Simulations
Outcomes. The generated simulation plots are shown in Fig. 4, which depicts: (i) a sample trace for each continuous variable (the evolution of \(x_1\) is shown in Fig. 4a, \(x_2\) in Fig. 4b) and for the discrete modes (see Fig. 4c); and (ii) histograms depicting the range of values the continuous variables can be in during each time step and the associated count (see Fig. 4c for \(x_1\) and Fig. 4e for \(x_2\)); and a histogram showing the likelihood of being in a discrete mode within each time step (see Fig. 4f). The total time taken to generate the simulations is 48.6 [s].
5 Conclusions and Extensions
We have presented \(\mathsf {StocHy}\), a new software tool for the quantitative analysis of stochastic hybrid systems. There is a plethora of enticing extensions that we are planning to explore. In the short term, we intend to: (i) interface with other model checking tools such as storm [12] and the modest toolset [16]; (ii) embed algorithms for policy refinement, so we can generate policies for models having numerous continuous input variables [15]; (iii) benchmarking the tool against a set of shs models [10]. In the longer term, we plan to extend \(\mathsf {StocHy}\) such that (i) it can employ a graphical userinterface; (ii) it can allow analysis of continuoustime shs; and (iii) it can make use of data structures such as multiterminal binary decision diagrams [14] to reduce the memory requirements during the construction of the abstract mdp or imdp.
Notes
Acknowledgements
The author’s would also like to thank Kurt Degiorgio, Sadegh Soudjani, Sofie Haesaert, Luca Laurenti, Morteza Lahijanian, Gareth Molyneux and Viraj Brian Wijesuriya. This work is in part funded by the Alan Turing Institute, London, and by Malta’s ENDEAVOUR Scholarships Scheme.
References
 1.Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)MathSciNetCrossRefGoogle Scholar
 2.Abate, A.: Formal verification of complex systems: modelbased and datadriven methods. In: Proceedings of the 15th ACMIEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2017, Vienna, Austria, 29 September–02 October 2017, pp. 91–93 (2017)Google Scholar
 3.Abate, A., et al.: ARCHCOMP18 category report: stochastic modelling. EPiC Ser. Comput. 54, 71–103 (2018)CrossRefGoogle Scholar
 4.Abate, A., Katoen, J.P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. Eur. J. Control 16(6), 624–641 (2010)MathSciNetCrossRefGoogle Scholar
 5.Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press, Cambridge (2008)zbMATHGoogle Scholar
 6.Bauer, C., Frink, A., Kreckel, R.: Introduction to the GiNaC framework for symbolic computation within the C++ programming language. J. Symbolic Comput. 33(1), 1–12 (2002)MathSciNetCrossRefGoogle Scholar
 7.Blom, H., Lygeros, J. (eds.): Stochastic Hybrid Systems: Theory and Safety Critical Applications. LNCIS, vol. 337. Springer, Heidelberg (2006). https://doi.org/10.1007/11587392CrossRefzbMATHGoogle Scholar
 8.Bouissou, M., Elmqvist, H., Otter, M., Benveniste, A.: Efficient Monte Carlo simulation of stochastic hybrid systems. In: Proceedings of the 10th International Modelica Conference, Lund, Sweden, 10–12 March 2014, no. 96, pp. 715–725. Linköping University Electronic Press (2014)Google Scholar
 9.Budde, C.E., Dehnert, C., Hahn, E.M., Hartmanns, A., Junges, S., Turrini, A.: JANI: quantitative model and tool interaction. In: Legay, A., Margaria, T. (eds.) TACAS 2017. LNCS, vol. 10206, pp. 151–168. Springer, Heidelberg (2017). https://doi.org/10.1007/9783662545805_9CrossRefGoogle Scholar
 10.Cauchi, N., Abate, A.: Benchmarks for cyberphysical systems: a modular model library for building automation systems. IFACPapersOnLine 51(16), 49–54 (2018). 6th IFAC Conference on Analysis and Design of Hybrid Systems ADHS 2018Google Scholar
 11.Cauchi, N., Laurenti, L., Lahijanian, M., Abate, A., Kwiatkowska, M., Cardelli, L.: Efficiency through uncertainty: scalable formal synthesis for stochastic hybrid systems. In: 22nd ACM International Conference on Hybrid Systems: Computation and Control (HSCC) (2019). arXiv:1901.01576
 12.Dehnert, C., Junges, S., Katoen, J.P., Volk, M.: A storm is coming: a modern probabilistic model checker. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10427, pp. 592–600. Springer, Cham (2017). https://doi.org/10.1007/9783319633909_31CrossRefGoogle Scholar
 13.Everdij, M.H., Blom, H.A.: Hybrid Petri Nets with diffusion that have intomappings with generalised stochastic hybrid processes. In: Blom, H.A.P., Lygeros, J. (eds.) Stochastic Hybrid Systems. LNCIS, vol. 337, pp. 31–63. Springer, Heidelberg (2006). https://doi.org/10.1007/11587392_2CrossRefzbMATHGoogle Scholar
 14.Fujita, M., McGeer, P.C., Yang, J.Y.: Multiterminal binary decision diagrams: an efficient data structure for matrix representation. Formal Methods Syst. Des. 10(2–3), 149–169 (1997)CrossRefGoogle Scholar
 15.Haesaert, S., Cauchi, N., Abate, A.: Certified policy synthesis for general Markov decision processes: an application in building automation systems. Perform. Eval. 117, 75–103 (2017)CrossRefGoogle Scholar
 16.Hahn, E.M., Hartmanns, A., Hermanns, H., Katoen, J.P.: A compositional modelling and analysis framework for stochastic hybrid systems. Formal Methods Syst. Des. 43(2), 191–232 (2013)CrossRefGoogle Scholar
 17.Hahn, E.M., Li, Y., Schewe, S., Turrini, A., Zhang, L.: iscasMc: a webbased probabilistic model checker. In: Jones, C., Pihlajasaari, P., Sun, J. (eds.) FM 2014. LNCS, vol. 8442, pp. 312–317. Springer, Cham (2014). https://doi.org/10.1007/9783319064109_22CrossRefGoogle Scholar
 18.Hartmanns, A., Hermanns, H.: The modest toolset: an integrated environment for quantitative modelling and verification. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 593–598. Springer, Heidelberg (2014). https://doi.org/10.1007/9783642548628_51CrossRefGoogle Scholar
 19.Krystul, J., Blom, H.A.: Sequential Monte Carlo simulation of rare event probability in stochastic hybrid systems. IFAC Proc. Volumes 38(1), 176–181 (2005)CrossRefGoogle Scholar
 20.Kupferman, O., Vardi, M.Y.: Model checking of safety properties. Formal Methods Syst. Des. 19(3), 291–314 (2001)CrossRefGoogle Scholar
 21.Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic realtime systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). https://doi.org/10.1007/9783642221101_47CrossRefGoogle Scholar
 22.Lahijanian, M., Andersson, S.B., Belta, C.: Formal verification and synthesis for discretetime stochastic systems. IEEE Trans. Autom. Control 60(8), 2031–2045 (2015)MathSciNetCrossRefGoogle Scholar
 23.Larsen, K.G., Mikučionis, M., Muñiz, M., Srba, J., Taankvist, J.H.: Online and compositional learning of controllers with application to floor heating. In: Chechik, M., Raskin, J.F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 244–259. Springer, Heidelberg (2016). https://doi.org/10.1007/9783662496749_14CrossRefGoogle Scholar
 24.Sanderson, C., Curtin, R.: Armadillo: a templatebased C++ library for linear algebra. J. Open Source Softw. 1, 26–32 (2016)CrossRefGoogle Scholar
 25.Sanderson, C., Curtin, R.: A userfriendly hybrid sparse matrix class in C++. In: Davenport, J.H., Kauers, M., Labahn, G., Urban, J. (eds.) ICMS 2018. LNCS, vol. 10931, pp. 422–430. Springer, Cham (2018). https://doi.org/10.1007/9783319964188_50CrossRefGoogle Scholar
 26.Škulj, D.: Discrete time Markov chains with interval probabilities. Int. J. Approx. Reason. 50(8), 1314–1329 (2009)MathSciNetCrossRefGoogle Scholar
 27.Soudjani, S.E.Z.: Formal abstractions for automated verification and synthesis of stochastic systems. Ph.D. thesis, TU Delft (2014)Google Scholar
 28.Soudjani, S.E.Z., Gevaerts, C., Abate, A.: FAUST\(^2\): formal abstractions of uncountableSTate STochastic processes. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 272–286. Springer, Heidelberg (2015). https://doi.org/10.1007/9783662466810_23CrossRefGoogle Scholar
 29.Střelec, M., Macek, K., Abate, A.: Modeling and simulation of a microgrid as a stochastic hybrid system. In: 2012 3rd IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe), pp. 1–9, October 2012Google Scholar
 30.Summers, S., Lygeros, J.: Verification of discrete time stochastic hybrid systems: a stochastic reachavoid decision problem. Automatica 46(12), 1951–1961 (2010)MathSciNetCrossRefGoogle Scholar
 31.Cauchi, N., Abate, A.: Artifact and instructions to generate experimental results for TACAS 2019 paper: StocHy: automated verification and synthesis of stochastic processes (artifact). Figshare (2019). https://doi.org/10.6084/m9.figshare.7819487.v1
Copyright information
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.