StocHy: automated verification and synthesis of stochastic processes

StocHy is a software tool for the quantitative analysis of discrete-time stochastic hybrid systems (SHS). StocHy accepts a high-level description of stochastic models and constructs an equivalent SHS model. The tool allows to (i) simulate the SHS evolution over a given time horizon; and to automatically construct formal abstractions of the SHS. Abstractions are then employed for (ii) formal verification or (iii) control (policy, strategy) synthesis. StocHy allows for modular modelling, and has separate simulation, verification and synthesis engines, which are implemented as independent libraries. This allows for libraries to be easily used and for extensions to be easily built. The tool is implemented in C++ and employs manipulations based on vector calculus, the use of sparse matrices, the symbolic construction of probabilistic kernels, and multi-threading. Experiments show StocHy's markedly improved performance when compared to existing abstraction-based approaches: in particular, StocHy beats state-of-the-art tools in terms of precision (abstraction error) and computational effort, and finally attains scalability to large-sized models (12 continuous dimensions). StocHy is available at www.gitlab.com/natchi92/StocHy.


Introduction
Stochastic hybrid systems (shs) are a rich mathematical modelling framework capable of describing systems with complex dynamics, where uncertainty and hybrid (that is, both continuous and discrete) components are relevant.Whilst earlier instances of shs have a long history, shs proper have been thoroughly investigated only from the mid 2000s, and have been most recently applied to the study of complex systems, both engineered and natural.Amongst the first class, shs have been used for modelling and analysis of smart grids [27], automation of medical devices [3], avionics [7], prognostics and health management [29].
However, a wider adoption of shs in real-world applications is stymied by a few factors: (i) the complexity associated with modelling shs; (ii) the generality of their mathematical framework, which requires an arsenal of advanced and diverse techniques to analyse them; and (iii) the undecidability of verification/synthesis problems over shs and the curse of dimensionality associated with their approximations.This paper introduces a new software tool -StocHywhich is aimed at simplifying both the modelling of shs and their analysis, and targets the wider adoption of shs by non-expert users.With focus on the three limiting factors above, StocHy allows to describe shs by parsing or extending well-known and -used state-space models and generates a standard shs model automatically and formats it to be analysed.StocHy can (i) perform verification tasks, e.g., compute the probability of staying within a certain region of the state space from a given set of initial conditions; (ii) automatically synthesise policies (strategies) maximising this probability, and (iii) simulate the shs evolution over time.StocHy is implemented in c++ and modular making it both extendible and portable.
Related work.There exist only a few tools that can handle (classes of) shs.Of much inspiration for this contribution, faust 2 [26] generates abstractions for uncountable-state discrete-time stochastic processes, natively supporting shs models with a single discrete mode and finite actions, and performs verification of reachability-like properties and corresponding synthesis of policies.faust 2 is naïvely implemented in matlab and lacks in scalability to large models.modest toolset [17] allows to model and to analyse classes of continuous-time shs, particularly probabilistic hybrid automata (pha) that combine probabilistic discrete transitions with deterministic evolution of the continuous variables.The tool for stochastic and dynamically coloured petri nets (sdcpn) [12] supports compositional modelling of pha and focuses on simulation via Monte Carlo techniques.The existing tools highlight the need for a new software that allows for (i) straightforward and general shs modelling construction and (ii) scalable automated analysis.

Contributions. The StocHy tool newly enables
formal verification of shs via either of two abstraction techniques: • for discrete-time, continuous-space models with additive disturbances, and possibly with multiple discrete modes, we employ formal abstractions as general Markov chains or Markov decision processes [26]; StocHy improves techniques in the faust 2 tool by simplifying the input model description, by employing sparse matrices to manipulate the transition probabilities and by reducing the computational time needed to generate the abstractions.• for models with a finite number of actions, we employ interval Markov decision processes and the model checking framework in [21]; StocHy provides a novel abstraction algorithm allowing for efficient computation of the abstract model, by means of an adaptive and sequential refining of the underlying abstraction.We show that we are able to generate significantly smaller abstraction errors and to verify models with up to 12 continuous variables.
control (strategy, policy) synthesis via formal abstractions, employing: • stochastic dynamic programming; StocHy exploits the use of symbolic kernels.
• robust synthesis using interval Markov decision processes; StocHy automates the synthesis algorithm with the abstraction procedure and the temporal property of interest, and exploits the use of sparse matrices; simulation of complex stochastic processes, such as shs, by means of Monte Carlo techniques; StocHy automatically generates statistics from the simulations in the form of histograms, visualising the evolution of both the continuous random variables and the discrete modes.
This contribution is structured as follows: Sec. 2 crisply presents the theoretical underpinnings (modelling and analysis) for the tool.We provide an overview of the implementation of StocHy in Sec. 3. We highlight features and use of StocHy by a set of experimental evaluations in Sec.4: we provide four different case studies that highlight the applicability, ease of use, and scalability of StocHy.Details on executing all the case studies are detailed in this paper and within a Wiki page that accompanies the StocHy distribution.

Models -Stochastic Hybrid Systems
StocHy supports the modelling of the following general class of shs [1,4].
-Q = {q 1 , q 2 , . . ., q m }, m ∈ N, represents a finite set of modes (locations); n ∈ N is the dimension of the continuous space R n of each mode; the hybrid state space is then given by D= ∪ q∈Q {q} × R n ; -U is a continuous set of actions, e.g.
which assigns to each s = (q, x) ∈ D and u ∈ U, a probability distribution over In this model the discrete component takes values in a finite set Q of modes (a.k.a.locations), each endowed with a continuous domain (the Euclidean space R n ).As such, a point s over the hybrid state space D is pair (q, x), where q ∈ Q and x ∈ R n .The semantics of transitions at any point over a discrete time domain, are as follows: given a point s ∈ D, the discrete state is chosen from T q , and depending on the selected mode q ∈ Q the continuous state is updated according to the probabilistic law T x .Non-determinism in the form of actions can affect both discrete and continuous transitions.
Remark 1.A rigorous characterisation of shs can be found in [1], which introduces a general class of models with probabilistic resets and a hybrid actions space.Whilst in principle we can deal with general shs models, in the case studies of this paper we focus on special instances, as described next.
Remark 2 (Special instance).In Case Study 2 (see Sec. 4.2) we look at models where actions are associated to a deterministic selection of locations, namely T q : U → Q and U is a finite set of actions.
Remark 3 (Special instance).In Case Study 4 (Section 4.4) we consider nonlinear dynamical models with bilinear terms, which are characterised for any q ∈ Q by where k ∈ N represents the discrete time index, A q , B q , G q are appropriately sized matrices, N q,i represents the bilinear influence of the i−th input component u i , and w k = w ∼ N (•; 0, 1) and N (•; η, ν) denotes a Gaussian density function with mean η and covariance matrix ν 2 .This expresses the continuous kernel In Case Study 1-2-3 (Sec.4.1-4.3),we look at the special instance from [21], where the dynamics are autonomous (no actions) and linear: here where in Case Studies 1, 3 Q is a single element.
Definition 2. A Markov decision process ( mdp) [5] is a discrete-time model defined as the tuple H = (Q, U, T q ), where -Q = {q 1 , q 2 , . . ., q m }, m ∈ N, represents a finite set of modes; -U is a finite set of actions; -T q : Q × Q × U → [0, 1] is a discrete stochastic kernel that assigns, to each q ∈ Q and u ∈ U, a probability distribution over Q : T q (•|q, u).
Whenever the set of actions is trivial or a policy is synthesised and used (cf.discussion in Sec.2.2) the mdp reduces to a Markov chain (mc), and a kernel Definition 3.An interval Markov decision process ( imdp) [24] extends the syntax of an mdp by allowing for uncertain T q , and is defined as the tuple H = (Q, U, P , P ), where -Q and U are as in Def.2; -P and P : is a function that assigns to each q ∈ Q a lower (upper) bound probability distribution over Q : P (•|q, u) ( P (•|q, u) respectively).
For all q, q ∈ Q and u ∈ U, it holds that P (q |q, u) ≤ P (q |q, u) and, q ∈Q P (q |q, u) ≤ 1 ≤ q ∈Q P (q |q, u).

Formal Verification and Strategy Synthesis via Abstractions
Formal verification and strategy synthesis over shs are in general not decidable [4,28], and can be tackled via quantitative finite abstractions.These are precise approximations that come in two main different flavours: abstractions into mdp [4,26] and into imdp [21].Once the finite abstractions are obtained, and with focus on specifications expressed in (non-nested) pctl or fragments of ltl [5], formal verification or strategy synthesis can be performed via probabilistic model checking tools, such as prism [20], storm [11], iscasMc [16].We overview next the two alternative abstractions, as implemented in StocHy.
Abstractions into Markov decision processes Following [25], mdp are generated by either (i) uniformly gridding the state space and computing an abstraction error, which depends on the continuity of the underlying continuous dynamics and on the chosen grid; or (ii) generating the grid adaptively and sequentially, by splitting the cells with the largest local abstraction error until a desired global abstraction error is achieved.The two approaches display an intuitive trade-off, where the first in general requires more memory but less time, whereas the second generates smaller abstractions.Either way, the probability to transit from each cell in the grid into any other cell characterises the mdp matrix T q .Further details can be found in [26].StocHy newly provides a c++ implementation and employs sparse matrix representation and manipulation, in order to attain faster generation of the abstraction and use in formal verification or strategy synthesis.Verification via mdp (when the action set is trivial) is performed to check the abstraction against non-nested, bounded-until specifications in pctl [5] or cosafe linear temporal logic (csltl) [19].
Strategy synthesis via mdp is defined as follows.Consider, the class of deterministic and memoryless Markov strategies π = (µ 0 , µ 1 , . . . ) where µ k : Q → U. We compute the strategy π that maximises the probability of satisfying a formula, with algorithms discussed in [26].
Abstraction into Interval Markov decision processes (imdp) is based on a procedure in [10] performed using a uniform grid and with a finite set of actions U (see Remark 2).StocHy newly provides the option to generate a grid using adaptive/sequential refinements (similar to the case in the paragraph above) [25], which is performed as follows: (i) define a required minimal maximum abstraction error ε max ; (ii) generate a coarse abstraction using the Algorithm in [10] and compute the local error ε q that is associated to each abstract state q; (iii) split all cells where ε q > ε max along the main axis of each dimension, and update the probability bounds (and errors); and (iv) repeat this process until ∀q, ε q < ε max .Verification via imdp is run over properties in csltl or bounded-LTL (bltl) form using the imdp model checking algorithm in [21].Synthesis via imdp [10] is carried out by extending the notions of strategies of mdp to depend on memory, that is on prefixes of paths.

Analysis via Monte Carlo simulations
Monte Carlo techniques generate numerical sampled trajectories representing the evaluation of a stochastic process over a predetermined time horizon.Given a sufficient number of trajectories, one can approximate the statistical properties of the solution process with a required confidence level.This approach has been adopted for simulation of different types of shs.[18] applies sequential Monte Carlo simulation to shs to reason about rare-event probabilities.[12] performs Monte Carlo simulations of classes of shs described as Petri nets.[8] proposes a methodology for efficient Monte Carlo simulations of continuous-time shs.In this work, we analyse a shs model using Monte Carlo simulations following the approach in [4].Additionally, we generate histogram plots at each time step, providing further insight on the evolution of the solution process.

Overview of StocHy
Installation StocHy is set up using the provided get dep file found within the distribution package, which will automatically install all the required dependencies.The executable run.shbuilds and runs StocHy.This basic installation setup has been successfully tested on machines running Ubuntu 18.04.1 LTS GNU and Linux operating systems.

Input interface
The user interacts with StocHy via the main file and must specify (i) a high-level description of the model dynamics and (ii) the task to be performed.The description of model dynamics can take the form of a list of the transition probabilities between the discrete modes, and of the state-space models for the continuous variables in each mode; alternatively, a description can be obtained by specifying a path to a matlab file containing the model description in state-space form together with the transition probability matrix.Tasks can be of three kinds (each admitting specific parameters): simulation, verification, or synthesis.The general structure of the input interface is illustrated via an example in Listing 1.1: here the user is interested in simulating a shs with two discrete modes Q = {q 0 , q 1 } and two continuous variables evolve according to (3).The model is autonomous and has no control actions.The relationship between the discrete modes is defined by a fixed transition probability (line 1).The evolution of the continuous dynamics are defined in lines 2-14.The initial condition for both the discrete modes and the continuous variables are set in lines 16-21 (this is needed for simulation tasks).The equivalent shs model is then set up by instantiating an object of type shs t<arma::mat,int> (line 23).Next, the task is defined in line 27 (simulation with a time horizon K = 32, as specified in line 25 and using the simulator library, as set in line 26).We combine the model and task specification together in line 29.Finally, StocHy carries out the simulation using the function performTask (line 31).
Modularity StocHy comprises independent libraries for different tasks, namely (i) faust 2 , (ii) imdp, and (iii) simulator.Each of the libraries is separate and depends only on the model structure that has been entered.This allows for seamless extensions of individual sub-modules with new or existing tools and methods.The function performTask acts as multiplexer for calling any of the libraries depending on the input model and task specification.
Data structures StocHy makes use of multiple techniques to minimise computational overhead.It employs vector algebra for efficient handling of linear operations, and whenever possible it stores and manipulates matrices as sparse structures.It uses the linear algebra library Armadillo [22,23], which applies multi-threading and a sophisticated expression evaluator that has been shown to speed up matrix manipulations in c++ when compared to other libraries.faust 2 based abstractions define the underlying kernel functions symbolically using the library GiNaC [6], for easy evaluation of the stochastic kernels.
Output interface We provide outputs as text files for all three libraries, which are stored within the results folder.We also provide additional python scripts for generating plots as needed.For abstractions based on faust 2 , the user has the additional option to export the generated mdp or mc to prism format, to interface with the popular model checker [20] (StocHy prompts the user this option following the completion of the verification or synthesis task).As a future extension, we plan to export the generated abstraction models to the model checker storm [11] and to the modelling format jani [9].

StocHy: Experimental Evaluation
We apply StocHy on four different case studies highlighting different models and tasks to be performed.All the experiments are run on a standard laptop, with an Intel Core i7-8550U CPU at 1.80GHz × 8 and with 8 GB of RAM.

Case Study 1 -Formal Verification
We consider the shs model first presented in [2].The model takes the form of ( 1), and has one discrete mode and two continuous variables representing the level of CO 2 (x 1 ) and the ambient temperature (x 2 ), respectively.The continuous variables evolve according to We are interested in verifying whether the continuous variables remain within the safe set X saf e = [405, 540] × [18,24] over 45 minutes (K = 3).This property can be encoded as a bltl property, ϕ 1 := ≤K X saf e , where is the "always" temporal operator considered over a finite horizon.The semantics of bltl is defined over finite traces, denoted by ζ = {ζ j } K j=0 .A trace ζ satisfies ϕ 1 if ∀j ≤ K, ζ j ∈ X saf e , and we quantify the probability that traces generated by the shs satisfy ϕ 1 .
Case study 1: Listings explaining task specification for (a) faust When tackled with the method based on faust 2 that hinges on the computation of Lipschitz constants, this verification task is numerically tricky, in view of difference in dimensionality of the range of x 1 and x 2 within the safe set X saf e and the variance associated with each dimension G q0 = [ σ1 0 0 σ2 ] = [ 40.096 0 0 0.511 ].In order to mitigate this, we rescale the state space so all the dynamics evolve in a comparable range and also apply the abstraction based on imdp.More precisely (this is done externally to StocHy), we consider an affine map x = Jy with J = [ 22.5 0 0 1 ], which results in the safe set X saf e to [18,24] 2 and in G q0 = [ 1.782 0 0 0.511 ].Consequently, the generated cell partitions are more uniform, with finer partitioning along x 2 .The dynamics of the new state space are provided in the file cs1.mat.
Implementation StocHy provides two verification methods, one based on faust 2 and the second based on imdp.We parse the model from file cs1.mat (see line 2 of Listings 1.2(a) and 1.3(b), corresponding to the two methods).cs1.mat sets parameter values to (6) and uses a ∆ = 15 [min].As anticipated, we employ both techniques over the same model description: for faust 2 we specify the safe set (X saf e ), the maximum allowable error, the grid type (whether uniform or adaptive grid), the time horizon, together with the type of property of interest (safety or reach-avoid).This is carried out in lines 5-21 in Listing 1.2(a).for the imdp method, we define the safe set (X saf e ), the grid size, the relative tolerance, the time horizon and the property type.This can be done by defining the task specification using lines 5-21 in Listing 1.3 (b).
Finally, to run either of the methods on the defined input model, we combine the model and the task specification using inputSpec t<arma::mat,int> myInput(myShs,mySpec), then run the command performTask(myInput).The verification results for both methods are stored in the results directory: for faust 2 , StocHy generates four text files within the results folder: representative points.txtcontains the partitioned state space; transition matrix.txtconsists of the transition probabilities of the generated abstract mc; problem solution.txtcontains the sat probability for each state of the mc; and e.txt stores the global maximum abstraction error.for imdp, StocHy generates three text files in the same folder: stepsmin.txtstores P of the abstract imdp; stepsmax.txtstores P ; and solution.txtcontains the sat probability and the errors ε q for each abstract state q.Outcomes We perform the verification task using both faust 2 and imdp, over different sizes of the abstraction grid.We employ uniform gridding for both methods.We further compare the outcomes of StocHy against those of the faust 2 tool, which is implemented in matlab [26].Note that the imdp consists of |Q| + 1 states, where the additional state is the sink state q u = D\X saf e .
green purple green Fig. 2: Case study 2: (a) Gridded domain together with a superimposed simulation of trajectory initialised at (−0.5, −1) within q 0 , under the synthesised optimal switching strategy π * .Lower probabilities of satisfying ϕ 2 for mode q 0 (b) and for mode q 1 (c), as computed by StocHy.
The results are shown in Table 1.We saturate (conservative) errors output that are greater than 1 to this value.We show the probability of satisfying the formula obtained from imdp for a grid size of 3481 states in Fig. 1 -similar probabilities are obtained for the remaining grid sizes.As evident from Table 1, the new imdp method outperforms the approach using faust 2 in terms of the maximum error associated to the abstraction (faust 2 generates an abstraction error < 1 only with 4225 states).Comparing the faust 2 within StocHy and the original faust 2 implementation (running in matlab), StocHy offers computational speed-up for the same grid size.This is due to the faster computation of the transition probabilities, through StocHy's use of matrix manipulations.faust 2 within StocHy also simplifies the input of the dynamical model description: in the original faust 2 implementation, the user is asked to manually input the stochastic kernel in the form of symbolic equations in a matlab script.This is not required when using StocHy, which automatically generates the underlying symbolic kernels from the input state-space model descriptions.

Case Study 2 -Strategy Synthesis
We consider a stochastic process with two modes Q = {q 0 , q 1 }, which continuously evolves according to (3) with A q0 = 0.43 0.52 0.65 0.12 , G q0 = 1 0.1 0 0.1 , A q0 = 0.65 0.12 0.52 0.43 , G q1 = 0.2 0 0 0.2 , F qi = 0 0 , and i ∈ {0, 1}.Consider the continuous domain shown in Fig. 2a over both discrete locations.We plan to synthesise the optimal switching strategy π that maximises the probability of reaching the green region, whilst avoiding the purple one, over an unbounded time horizon, given any initial condition within the domain.This requirement can be expressed with the ltl formula, ϕ 2 := (¬purple) U green, where U is the "until " temporal operator, and the atomic propositions {purple, green} denote regions within the set X = [−1.5,1.5] 2 , as shown in Fig. 2a.
Implementation We define the model dynamics following lines 3-14 in Listing 1.1, while we use Listing 1.3 to specify the synthesis task and together with its associated parameters.The ltl property ϕ 2 is over an unbounded time horizon, which leads to employing the imdp method for synthesis (recall that the faust 2 implementation can only handle time-bounded properties, and its abstraction error monotonically increases with the time horizon of the formula).
In order to encode the task we set the variable safe to correspond to X the grid size to 0.12 and the relative tolerance to 0.06 along both dimensions (cf.lines 5-10 in Listing 1.3).We set the time horizon K = -1 to represent an unbounded time horizon, let p = 4 to trigger the synthesis engine over the given specification and make lb = 3 to use imdp method (cf.lines 12-19 in Listing 1.3).This task specification partitions the set X into the underlying imdp via uniform gridding.Alternatively, the user has the option to make use of the adaptive-sequential algorithm by defining a new variable eps max which characterise the maximum allowable abstraction error and then specify the task using taskSpec t mySpec(lb,K,p,boundary,eps max,grid,rtol);.Next, we define two files (phi1 .txtand phi2.txt)containing the coordinates within the gridded domain (see Fig. 2a) associated with the atomic propositions purple and green, respectively.This allows for automatic labelling of the state-space over which synthesis is to be performed.Running the main file, StocHy generates a Solution.txtfile within the results folder.This contains the synthesised π policy, the lower bound for the probabilities of satisfying ϕ 2 , and the local errors ε q for any region q.
Outcomes The case study generates an abstraction with a total of 2410 states, a maximum probability of 1, a maximum abstraction error of 0.21, and it requires a total time of 1639. 3 [s].In this case, we witness a slightly larger abstraction error via the imdp method then in the previous case study.This is due the nondiagonal covariance matrix G q0 which introduces a rotation in X within mode q 0 .When labelling the states associated with the regions purple and green, an additional error is introduced due to the over-and under-approximation of states associated with each of the two regions.We further show the simulation of a trajectory under π with a starting point of (−0.5, −1) in q 0 , within Fig. 2a.

Case Study 3 -Scaling in Continuous Dimension of Model
We now focus on the continuous dynamics by considering a stochastic process with Q = {q 0 } (single mode) and dynamics evolving according to (3), characterised by A q0 = 0.8I d , F q0 = 0 d and G q0 = 0.2I d , where d corresponds to the number of continuous variables.We are interested in checking the ltl specification ϕ 3 := X saf e , where X saf e = [− Table 2: Case study 3: Verification results of the imdp-based approach over ϕ 3 , for varying dimension d of the stochastic process.
3, we disregard discussing the computed probabilities, which we instead covered in Section 4.1.
Implementation Similar to Case Study 2, we follow lines 3-14 in Listing 1.1 to define the model dynamics, while we use Listing 1.3 to specify the verification task using the imdp method.For this example, we employ a uniform grid having a grid size of 1 and relative tolerance of 1 for each dimension (cf.lines 5-10 in Listing 1.3).We set K = -1 to represent an unbounded time horizon, p = 1 to perform verification over a safety property and lb = 3 to use the imdp method (cf.lines 12-19 in Listing 1.3).In Table 2 we list the number of states required for each dimension, the total computational time, and the maximum error associated with each abstraction.
Outcomes From Table 2 we can deduce that by employing the imdp method within StocHy, the generated abstract models have manageable state spaces, thanks to the tight error bounds that is obtained.Notice that since the number of cells per dimension is increased with the dimension d of the model, the associated abstraction error ε max is decreased.The small error is also due to the underlying contractive dynamics of the process.This is a key fact leading to scalability over the continuous dimension d of the model: StocHy displays a significant improvement in scalability over the state of the art [26] and allows abstracting stochastic models with relevant dimensionality.Furthermore, StocHy is capable to handle specifications over infinite horizons (such as the considered until formula).

Case Study 4 -Simulations
For this last case study, we refer to the CO 2 model described in Case Study 1 (Sec.4.1).We extend the CO 2 model to capture (i) the effect of occupants leaving or entering the zone within a time step (ii) the opening or closing of the windows in the zone [2].ρ m is now a control input and is an exogenous signal.This can be described as a shs comprising two-dimensional dynamics, over discrete modes in the set {q 0 = (E, C), q 1 = (F, C), q 2 = (F, O), q 3 = (E, O)} describing possible configurations of the room (empty (E) or full (F), and with windows open (O) or closed (C)).A mc representing the discrete modes and their dynamics is in Figure 3a.The continuous variables evolve according to Eqn. (6), which now captures the effect of switching between discrete modes, as x where the additional terms are:  temperature level of x 2 ∼ N (17, 2) [ • C].We define the initial conditions using Listing 1.4.Line 2 defines the number of Monte Carlo simulations using by the variable monte and sets this to 5000.We instantiate the initial values of the continuous variables using the term x init, while we set the initial discrete mode using the variable q init.This is done using lines 4-17 which defines independent normal distribution for each of the continuous variable from which we sample 5000 points for each of the continuous variables and defines the initial discrete mode to q 0 = (E, C).We define the control signal ρ m in line 20, by parsing the u.txt which contains discrete values of ρ m for each time step (see Fig. 3b).Once the model is defined, we follow Listing 1.1 to perform the simulation.The simulation engine also generates a python script, simPlots.py,which gives the option to visualise the simulation outcomes offline.
Outcomes The generated simulation plots are shown in Fig. 4, which depicts: (i) a sample trace for each continuous variable (the evolution of x 1 is shown in Fig. 4a, x 2 in Fig. 4b) and for the discrete modes (see Fig.

Conclusions and Extensions
We have presented StocHy, a new software tool for the quantitative analysis of stochastic hybrid systems.There is a plethora of enticing extensions that we are planning to explore.In the short term, we intend to: (i) interface with other model checking tools such as storm [11] and the modest toolset [15]; (ii) embed algorithms for policy refinement, so we can generate policies for models having numerous continuous input variables [14].In the longer term, we plan to extend StocHy such that (i) it employs a graphical user-interface; (i) it may allow analysis of continuous-time shs;and (iii) it makes use of data structures such as multi-terminal binary decision diagrams [13] to reduce the memory requirements during the construction of the abstract mdp or imdp.
∆ the sampling time [min], V is the volume of the zone [m 3 ], ρ m is the mass air flow pumped inside the room [m 3 /min], c is the natural drift air flow [m 3 /min], C out is the outside CO 2 level [ppm/min], T set is the desired temperature [ o C], T out is the outside temperature [ • C/min], C z is the zone capacitance [Jm 3 / • C], C pa is the specific heat capacity of air [J/ • C], R is the resistance to heat transfer [ • C/J], and σ (•) is a variance term associated to the noise w k ∼ N (0, 1).
4c); and (ii) histograms depicting the range of values the continuous variables can be in during each time step and the associated count (see Fig.4dfor x 1 and Fig.4e for x2 ); and a histogram showing the likelihood of being in a discrete mode within each time step (see Fig.4f).The total time taken to generate the simulations is 48.6 [s].

Table 1 :
Case study 1: Comparison of verification results for ϕ 1 when using faust 2 vs imdp.
1, 1] d , as the continuous dimension d of the model varies.Here " " is the "always" temporal operator and a trace ζ satisfies ϕ 3 if ∀k ≥ 0, ζ k ∈ X saf e .In view of the focus on scalability for this Case Study (•) is the natural drift air flow that changes depending whether the window is open ( o ) or closed ( c ) [m 3 /min]; C occ is the generated CO 2 level when the zone is occupied (it is multiplied by the indicator function 1 F ) [ppm/min]; T occ is the generated heat due to occupants [ • C/min], which couples the dynamics in (7) as T occ,k = vx 1,k + .