Localized Attractor Computations for Infinite-State Games

Abstract

Infinite-state games are a commonly used model for the synthesis of reactive systems with unbounded data domains. Symbolic methods for solving such games need to be able to construct intricate arguments to establish the existence of winning strategies. Often, large problem instances require prohibitively complex arguments. Therefore, techniques that identify smaller and simpler sub-problems and exploit the respective results for the given game-solving task are highly desirable.

In this paper, we propose the first such technique for infinite-state games. The main idea is to enhance symbolic game-solving with the results of localized attractor computations performed in sub-games. The crux of our approach lies in identifying useful sub-games by computing permissive winning strategy templates in finite abstractions of the infinite-state game. The experimental evaluation of our method demonstrates that it outperforms existing techniques and is applicable to infinite-state games beyond the state of the art.

Authors are ordered randomly, denoted by P. Heim, R. Dimitrova and S. Prakash. The publicly verifiable record of the randomization is available at www.aeaweb.org.

1 Introduction

Games on graphs provide an effective way to formalize the automatic synthesis of correct-by-design software in cyber-physical systems. The prime examples are algorithms that synthesize control software to ensure high-level logical specifications in response to external environmental behavior. These systems typically operate over unbounded data domains. For instance, in smart-home applications [35], they need to regulate real-valued quantities like room temperature and lighting in response to natural conditions, day-time, or energy costs. Also, unbounded data domains are valuable for over-approximating large countable numbers of products in a smart manufacturing line [20]. The tight integration of many specialized machines makes their efficient control challenging. Similar control synthesis problems occur in robotic warehouse systems [18], underwater robots for oil-pipe inspections [25], and electric smart-grid regulation [29].

Algorithmically, the outlined synthesis problems can be formalized via infinite-state games that model the ongoing interaction between the system (with its to-be-designed control software) and its environment over their infinite data domains. Due to their practical relevance and their challenging complexity, there has been an increasing interest in automated techniques for solving infinite-state games to obtain correct-by-design control implementations. As the game-solving problem is in general undecidable in the presence of infinite data domains, this problem is substantially more challenging than its finite-state counterpart.

Fig. 1.

Schematic paper outline; contributions highlighted in blue. (Color figure online)

Within the literature¹, there are two prominent directions to attack this problem. One comprises abstraction-based approaches, where either the overall synthesis problem (e.g. [23, 38]) or the specification (e.g. [8, 14, 27]) are abstracted, resulting in a finite-state game, to which classical techniques apply. The other one are constraint-based techniques [9, 10, 32, 33], that work directly on a symbolic representation of the infinite-state game. Due to the undecidability of the overall synthesis problem, both categories are inherently constrained. While abstraction-based approaches are limited by the abstraction domain they employ, constraint-based techniques typically diverge due to non-terminating fixpoint computations.

To address these limitations, a recent constraint-based technique called attractor acceleration [21] employs ranking arguments to improve the convergence of symbolic game-solving algorithms. While this technique has shown superior performance over the state-of-the art, the utilized ranking arguments become complex, and thus difficult to find, as the size of the games increases. This makes the approach from [21] infeasible in such cases, often resulting in divergence in larger and more complex games.

In this paper, we propose an approach to overcoming the above limitation and thus extending the applicability of synthesis over infinite state games towards realistic applications. The key idea is to utilize efficient abstraction-based pre-computations that localize attractor computations to small and useful sub-games. In that way, acceleration can be applied locally to small sub-games, and the results utilized by the procedure for solving the global game. This often avoids the computationally inefficient attractor acceleration over the complete game. To guide the identification of useful sub-games, our approach computes strategy templates [2] – a concise representation of a possibly infinite number of winning strategies – in finite abstractions of the infinite-state game. Figure 1 shows an overview of our method which also serves as an outline of the paper.

Our experimental evaluation demonstrates the superior performance of our approach compared to the state of the art. Existing tools fail on almost all benchmarks, while our implementation terminates within minutes.

To build up more intuition, we illustrate the main idea of our approach with the following example, which will also serve as our running example.

Fig. 2.

A reactive program game for a sample-collecting robot with locations ${ base , move , mine }$, integer-type program variables ${ pos , done , req , samp }$ and input variable ${ inpReq }$. We use the following abbreviations: $enterBase ~\widehat{=}~( pos = 12 \wedge done = 1)$, $atMine ~\widehat{=}~( pos = 23)$, $haveSamples ~\widehat{=}~(a > 0 \vee b > 0)$, $enough ~\widehat{=}~ samp \ge req$, $sampleA ~\widehat{=}~( samp := samp + a)$, $sampleB ~\widehat{=}~( samp := samp + b)$, and $sampleS ~\widehat{=}~( samp := samp + 1)$. In each round of the game, the environment chooses a value for the input ${ inpReq }$. Based on guards over program variables and inputs, the game transitions to a black square. The system then chooses one of the possible updates to the program variables, thus determining the next location.

Example 1

Figure 2 shows a reactive program game for a sample-collecting robot. The robot moves along tracks, and its position is determined by the integer program variable $pos$. The robot remains in location $base$ until prompted by the environment to collect $inpReq$ many samples. It cannot return to $base$ until the required samples are collected, as enforced by the variable $done$. From the right position, it can enter the $mine$, where it must stay and collect samples from two sites, a and b. However, it has to choose the correct site in each iteration, as they might not have samples all the time (if both do not have samples, it can get one sample itself). Once enough samples are collected, the robot can return to $base$. The requirement on the robot’s strategy is to be at base infinitely often.

Attractor acceleration [21] uses ranking arguments to establish that by iterating some strategy an unbounded number of times through some location, a player in the game can enforce reaching a set of target states. In this example, to reach $samp \ge req$ in location $mine$ (the target) the robot can iteratively increase the value of $samp$ by choosing the right updates (the iterated strategy). This works, since if $samp$ is increased repeatedly, eventually $samp \ge req$ will hold (the ranking argument). Establishing the existence of the iterated strategy (i.e. the robot can increment $samp$) is a game-solving problem, since the behavior of the robot is influenced by the environment. This game-solving problem potentially considers the whole game, since the iterated strategy is not known a priori. In addition, identifying locations where acceleration can be applied and finding the right ranking arguments is challenging. This impacts the scalability and applicability of acceleration, making it infeasible for large games.

Consequently, our method aims to identify small and useful sub-games and cache the results obtained by solving these sub-games. In Example 1, a useful sub-game would be the game restricted to the $mine$ location with the target state $samp \ge req$. Applying the acceleration technique to this sub-game, provides the ranking argument described earlier. These cached results are then utilized to enhance the symbolic game-solving procedure for the entire game.

To identify these small and useful sub-games, we use permissive strategy templates [2] in finite-state abstracted games. They describe a potentially infinite set of winning strategies using local conditions on the transitions of the game. These local conditions (in the abstract game) provide guidance about local behavior in the solution of the infinite-state game without solving it. This local behavior (e.g. incrementing $samp$ in $mine$) induces our sub-games.

2 Preliminaries

Sequences and First-Order Logic. For a set V, $V^*$ and $V^\omega$ denote the sets of finite, respectively infinite, sequences of elements of V, and let $V^\infty = V^* \cup V^\omega$. For $\pi \in V^\infty$, we denote with $|\pi | \in \mathbb {N}\cup \{ \infty \}$ the length of $\pi$, and define $dom (\pi ):=\{0,\ldots , |\pi | - 1\}$. For $\pi = v_0v_1\ldots \in V^\infty$ and $i, j\in dom (\pi )$ with $i<j$, we define $\pi [i]:=v_i$ and $\pi [i,j]:=v_i\ldots v_j$. $last (\pi )$ is the last element of a finite sequence $\pi$.

Let $\mathcal {V}$ be the set of all values of arbitrary types, $Vars$ be the set of all variables, $\mathcal {F}$ be the set of all functions, and $\varSigma _F$ be the set of all function symbols. Let $\mathcal {T}_F$ be the set of all function terms defined by the grammar $\mathcal {T}_F\ni \tau _f :\,\!:= x \,|\, f(\tau _f^1, \dots \tau _f^n)$ for $f \in \varSigma _F$ and $x \in Vars$. A function $\nu : Vars \rightarrow \mathcal {V}$ is called an assignment. The set of all assignments over variables $X \subseteq Vars$ is denoted as $Assignments (X)$. We denote the combination of two assignments $\nu ', \nu ''$ over disjoint sets of variables by $\nu ' \uplus \nu ''$. A function $\mathcal {I}: \varSigma _F\rightarrow \mathcal {F}$ is called an interpretation. The set of all interpretations is denoted as $Interpretations (\varSigma _F)$. The evaluation of function terms $\chi _{\nu ,\mathcal {I}}: \mathcal {T}_F\rightarrow \mathcal {V}$ is defined by $\chi _{\nu ,\mathcal {I}}(x) := \nu (x)$ for $x \in Vars$, $\chi _{\nu ,\mathcal {I}}(f(\tau _0, \dots \tau _n)) := \mathcal {I}(f)(\chi _{\nu ,\mathcal {I}}(\tau _0), \dots \chi _{\nu ,\mathcal {I}}(\tau _n))$ for $f \in \varSigma _F$ and $\tau _0, \dots \tau _n \in \mathcal {T}_F$. We denote the set of all first-order formulas as $FOL$ and by $QF$ the set of all quantifier-free formulas in $FOL$. Let $\varphi$ be a formula and $X = \{x_1,\ldots ,x_n\} \subseteq Vars$ be a set of variables. We write $\varphi (X)$ to denote that the free variables of $\varphi$ are a subset of X. We also denote with $FOL (X)$ and $QF (X)$ the set of formulas (respectively quantifier-free formulas) whose free variables belong to X. For a quantifier $Q \in \{\exists , \forall \}$, we write $Q X.\varphi$ as a shortcut for $Q x_1.\ldots Q x_n.\varphi$. We denote with $\models : Assignments ( Vars ) \times Interpretations (\varSigma _F) \times FOL$ the entailment of first-order logic formulas. A first-order theory $T \subseteq Interpretations (\varSigma _F)$ with $T \ne \emptyset$ restricts the possible interpretations of function and predicate symbols. Given a theory T, for a formula $\varphi (X)$ and assignment $\nu \in Assignments (X)$ we define that $\nu \models _{T} \varphi$ if and only if $\nu , \mathcal {I}\models \varphi$ for all $\mathcal {I}\in T$.

For exposition on first-order logic and first-order theories, see c.f. [7].

Two-Player Graph Games. A game graph is a tuple $G = (V,V_ Env ,V_ Sys ,\rho )$ where $V = V_ Env \uplus V_ Sys$ are the vertices, partitioned between the environment player (player $Env$) and the system player (player $Sys$), and $\rho \subseteq (V_ Env \times V_ Sys ) \cup (V_ Sys \times V_ Env )$ is the transition relation. A play in G is a sequence $\pi \in V^\infty$ where $(\pi [i],\pi [i+1])\in \rho$ for all $i \in dom (\pi )$, and if $\pi$ is finite then $last (\pi )$ is a dead-end.

For $p = Sys$ (or $Env$) we define $1 - p := Env$ (respectively $Sys$). A strategy for player p is a partial function $\sigma : V^*V_{p} \rightarrow V$ where $\sigma (\pi \cdot v) = v'$ implies $(v,v') \in \rho$ and $\sigma$ is defined for all $\pi \cdot v \in V^*V_{p}$ unless v is a dead-end. $Strat _{p}(G)$ denotes the set of all strategies for player p in G. A play $\pi$ is consistent with $\sigma$ for player p if $\pi [i+1] = \sigma (\pi [0,i])$ for every $i \in dom (\pi )$ where $\pi [i] \in V_p$. $Plays _G(v,\sigma )$ is the set of all plays in G starting in v and consistent with strategy $\sigma$.

An objective in G is a set $\varOmega \subseteq V^\infty$. A two-player turn-based game is a pair $(G,\varOmega )$, where G is a game graph and $\varOmega$ is an objective for player $Sys$. A sequence $\pi \in V^\infty$ is winning for player $Sys$ if and only if $\pi \in \varOmega$, and is winning for player $Env$ otherwise. The definitions of different types of common objectives can be found in [34]. The winning region $W_p(G,\varOmega )$ of player p in $(G,\varOmega )$ is the set of all vertices v from which player p has a strategy $\sigma$ such that every play in $Plays _G(v,\sigma )$ is winning for player p. A strategy $\sigma$ of player p is winning if for every $v\in W_p(G,\varOmega )$, every play in $Plays _G(v,\sigma )$ is winning for player p.

Acceleration-Based Solving of Infinite-State Games. We represent infinite-state games using the same formalism as [21], called reactive program games. Intuitively, reactive program games describe symbolically, using $FOL$ formulas and terms, the possible interactions between the system player and the environment player in two-player games over infinite data domains.

Definition 1

(Reactive Program Game Structure [21]). A reactive program game structure is a tuple $\mathcal G = (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ with the following components. T is a first-order theory. $\mathbb {I}\subseteq Vars$ is a finite set of input variables. $\mathbb {X}\subseteq Vars$ is a finite set of program variables where $\mathbb {I}\cap \mathbb {X}= \emptyset$. L is a finite set of game locations. $Inv : L \rightarrow FOL (\mathbb {X})$ maps each location to a location invariant. $\delta \subseteq L \times QF (\mathbb {X}\cup \mathbb {I}) \times (\mathbb {X}\rightarrow \mathcal {T}_F) \times L$ is a finite symbolic transition relation where

• (1) for every $l \in L$ the set of outgoing transition guards $Guards (l) := \{ g \mid \exists u, l'.~(l, g, u , l') \in \delta \}$ is such that $\bigvee _{g \in Guards (l)} g \equiv _{T} \top$, and for all $g_1, g_2 \in Guards (l)$ with $g_1 \ne g_2$ it holds that $g_1 \wedge g_2 \equiv _{T} \bot$,

• (2) for all $l,g, u, l_1, l_2$, if $(l, g, u, l_1) \in \delta$ and $(l, g, u, l_2) \in \delta$, then $l_1 = l_2$, and

• (3) for every $l \in L$ and $\textbf{x} \in Assignments (\mathbb {X})$ such that $\textbf{x} \models _{T} Inv (l)$, and $\textbf{i} \in Assignments (\mathbb {I})$, there exist a transition $(l, g, u, l') \in \delta$ such that $\textbf{x} \uplus \textbf{i}\models _{T} g$ and $\mathbf {x'} \models _{T} Inv (l')$ where $\textbf{x}'(x) = \chi _{\textbf{x}\uplus \textbf{i},\mathcal {I}}(u(x))$ for all $x \in \mathbb {X}$ and $\mathcal {I}\in T$, and

• (4) for every $(l, g, u, l') \in \delta$, $f \in \varSigma _F(u)$, $\mathcal {I}_1, \mathcal {I}_2 \in T$ it holds that $\mathcal {I}_1(f) = \mathcal {I}_2(f)$.

The requirements on $\delta$ imply for each $l\in L$ that: (1) the guards in $Guards (l)$ partition the set $Assignments (\mathbb {X}\cup \mathbb {I})$, (2) each pair of $g \in Guards (l)$ and update u can label at most one outgoing transition from l, (3) if there is an assignment satisfying the invariant at l, then for every input assignment there is a possible transition, and (4) the theory T determines the meaning of functions in updates uniquely. Given locations $l,l' \in L$, we define $Labels (l,l'):=\{(g,u) \mid (l, g, u, l') \in \delta \}$ as the set of labels on transitions from l to $l'$. We define as $\textsf{RPGS}$ the set of all reactive program game structures. The semantics of the reactive program game structure $\mathcal G$ is a (possibly infinite) game graph defined as follows.

Definition 2

(Semantics of Reactive Program Game Structures). Let $\mathcal G = (T,\mathbb {I}, \mathbb {X}, L, Inv , \delta )$ be a reactive program game structure. The semantics of $\mathcal G$ is the game graph $\llbracket \mathcal \rrbracket {G} = (\mathcal {S}, \mathcal {S}_ Env ,\mathcal {S}_ Sys ,\rho )$ where $\mathcal {S}:= \mathcal {S}_ Env \uplus \mathcal {S}_ Sys$ and

• $\mathcal {S}_ Env := \{ (l,\textbf{x}) \in L \times Assignments (\mathbb {X}) \mid \textbf{x} \models _{T} Inv (l)\}$;

• $\mathcal {S}_ Sys :=\mathcal {S}_ Env \times Assignments (\mathbb {I})$;

• $\rho \subseteq (\mathcal {S}_ Env \times \mathcal {S}_ Sys ) \cup (\mathcal {S}_ Sys \times \mathcal {S}_ Env )$ is the smallest relation such that

  • $(s,(s,\textbf{i})) \in \rho$ for every $s \in \mathcal {S}_ Env$ and $\textbf{i} \in Assignments (\mathbb {I})$,

  • $(((l,\textbf{x}), \textbf{i}),(l',\textbf{x}')) \in \rho$ iff $\textbf{x}' \models _{T} Inv (l')$ and there exists $(g,u) \in Labels (l,l')$ such that $\textbf{x}\uplus \textbf{i} \models _{T} g$, $\textbf{x}'(x) = \chi _{\textbf{x}\uplus \textbf{i}, \mathcal {I}}(u(x))$ for every $x \in \mathbb {X}$ and $\mathcal {I}\in T$.

Note that this semantics differs from the original one in [21] where the semantic game structure is not split into environment and system states. We do that in order to consistently use the notion of a game graph. Both semantics are equivalent. We refer to the vertices of $\llbracket G\rrbracket$ as states. We define the function $loc : \mathcal {S}\rightarrow L$ where $loc (s) := l$ for any $s= (l,\textbf{x}) \in \mathcal {S}_ Env$ and any $s = ((l,\textbf{x}),\textbf{i}) \in \mathcal {S}_ Sys$. By abusing notation, we extend the function $loc$ to sequences of states, defining $loc : \mathcal {S}^\infty \rightarrow L^\infty$ where $loc (\pi ) = l_0l_1l_2\ldots$ iff $loc (\pi [i]) = l_i$ for all $i \in dom (\pi )$. For simplicity of the notation, we write $W_p(\mathcal {G},\varOmega )$ instead of $W_p(\llbracket \mathcal {G}\rrbracket ,\varOmega )$. We represent and manipulate possibly infinite sets of states symbolically, using formulas in $FOL$($\mathbb {X}$) to describe sets of assignments to the variables in $\mathbb {X}$. Our symbolic domain $\mathcal {D}:= L \rightarrow FOL (\mathbb {X})$ is the set of functions mapping locations to formulas in $FOL$($\mathbb {X}$). An element $d \in \mathcal {D}$ represents the states $\llbracket d\rrbracket : = \{((l,\textbf{x}) \in \mathcal {S}\mid \textbf{x} \models _{T} d(l)\}.$ With $\{ l_1 \mapsto \varphi _1, \dots , l_n \mapsto \varphi _n \}$ we denote $d \in \mathcal {D}$ s.t. $d(l_i) = \varphi _i$ and $d(l) = \bot$ for $l \not \in \{l_1, \dots , l_n\}$. For brevity, we sometimes refer to elements of $\mathcal {D}$ as sets of states.

Note that the elements of the symbolic domain $\mathcal {D}$ represent subsets of $\mathcal {S}_ Env$, i.e., sets of environment states. Environment states are pairs of location and valuation of the program variables. The system states, on the other hand, correspond to intermediate configurations that additionally store the current input from the environment. This input is not stored further on (unless assigned to program variables). Thus, we restrict the symbolic domain to environment states.

Solving Reactive Program Games. We consider objectives defined over the locations of a reactive program game structure $\mathcal {G}$. That is, we require that if $\pi ',\pi '' \in \mathcal {S}^\infty$ are such that $loc (\pi ') = loc (\pi '')$, then $\pi ' \in \varOmega$ iff $\pi '' \in \varOmega$. We consider the problem of solving reactive program games. Given $\mathcal {G}$ and an objective $\varOmega$ for Player $Sys$ defined over the locations of $\mathcal {G}$, we want to compute $W_{ Sys }(\llbracket \mathcal \rrbracket {G},\varOmega )$.

Attractor Computation and Acceleration. A core building block of many algorithms for solving two-player games is the computation of attractors. Intuitively, an attractor is the set of states from which a given player p can enforce reaching a given set of target states no matter what the other player does. Formally, for a reactive program game structure $\mathcal {G}$, and $R \subseteq \mathcal {S}$ the player-p attractor for R is

$$ Attr _{\llbracket \mathcal \rrbracket {G},p}(R) := \{s \in \mathcal {S}\mid \exists \sigma \in Strat _{p}(\llbracket \mathcal \rrbracket {G}).\forall \pi \in Plays _{\llbracket \mathcal {G}\rrbracket }(s,\sigma ).\exists n\in \mathbb {N}.\;\pi [n]\in R\}.$$

In this work, we are concerned with the symbolic computation of attractors in reactive program games. Attractors in reactive program games are computed using the so-called enforceable predecessor operator over the symbolic domain $\mathcal {D}$. For $d \in \mathcal {D}$, $CPre _{\mathcal {G},p}(d) \in \mathcal {D}$ represents the states from which player p can enforce reaching $\llbracket d\rrbracket$ in one step in $\mathcal {G}$ (i.e. one move by each player). More precisely,

$$ \begin{array}{ll} \llbracket CPre _{\mathcal {G}, Sys }(d)\rrbracket &{}= \{ s \in \mathcal {S}_ Env \mid \forall s'. \; ((s,s') \in \rho ) \rightarrow \exists s''.\; (s',s'') \in \rho \wedge s ''\in \llbracket d\rrbracket \},~\text {and} \\ \llbracket CPre _{\mathcal {G}, Env }(d)\rrbracket &{}= \{s \in \mathcal {S}_ Env \mid \exists s'. \; ((s,s') \in \rho ) \wedge \forall s''.\; ((s',s'') \in \rho ) \rightarrow s'' \in \llbracket d\rrbracket \}. \end{array} $$

The player-p attractor for $\llbracket d\rrbracket$ can be computed as a fixpoint of the enforceable predecessor operator:

$$ \begin{array}{ll} Attr _{\llbracket \mathcal \rrbracket {G},p}(\llbracket d\rrbracket ) \cap \mathcal {S}_ Env &= \llbracket \mu X. \;d \vee CPre _{\mathcal {G},p}(X)\rrbracket , \end{array} $$

where $\mu$ denotes the least fixpoint. Note that since $\mathcal {S}$ is infinite, an iterative computation of the attractor is not guaranteed to terminate.

In Example 1, consider the computation of the player-$Sys$ attractor for $\llbracket d\rrbracket$ where $d = \{ base \mapsto \top , move \mapsto \top , mine \mapsto \bot \}$. Applying $CPre _{\mathcal {G}, Sys }(d)$ will produce $\{ base \mapsto \top , move \mapsto \top , mine \mapsto samp \ge req \}$ as in one step player-$Sys$ can enforce reaching $move$ if $samp \ge req$ in $mine$. Since in $mine$ the system player can enforce to increment $samp$ by at least one, a second iteration of $CPre _{\mathcal {G}, Sys }(\cdot )$ gives $\{\dots , mine \mapsto samp \ge req - 1\}$, a third $\{\dots , mine \mapsto samp \ge req - 2\}$, and so on. Thus, a naive iterative fixpoint computation does no terminate here. To avoid this non-termination, [21] introduced attractor acceleration. It will compute that, as explained in Sect. 1, the fixpoint is indeed $\{\dots , mine \mapsto \top \}$.

Permissive Strategy Templates. The main objective of this work is to identify small and useful sub-games, for which the results can enhance the symbolic game-solving process. To achieve this, we use a technique called permissive strategy templates [2], designed for finite game graphs. These templates can represent (potentially infinite) sets of winning strategies through local edge conditions. This motivates our construction of sub-games based on templates in Sect. 4.2.

These strategy templates are structured using three local edge conditions: safety, co-live, and live-group templates. Formally, given a game $(G,\varOmega )$ with $G=(V,V_ Env , V_ Sys ,\rho )$ and $E_p=\rho \cap (V_p\times V_{p-1})$, a strategy template for player p is a tuple $(U,D,\mathcal {H})$ consisting of a set of unsafe edges $U\subseteq E_p$, a set of co-live edges $D\subseteq E_p$, and a set of live-groups $\mathcal {H}\subseteq 2^{E_p}$. A strategy template $(U,D,\mathcal {H})$ represents the set of plays $\varPsi = \varPsi _U\cap \varPsi _D\cap \varPsi _\mathcal {H}\subseteq Plays (G)$, where

$$\begin{aligned} \varPsi _U&:= \{ \pi \mid \forall i.~(\pi [i], \pi [i+1]) \not \in U\}, \ \ \varPsi _D:= \{ \pi \mid \exists k.~\forall i > k.~(\pi [i], \pi [i+1]) \not \in D\}, \\ \varPsi _\mathcal {H}&:= \bigcap _{H\in \mathcal {H}} \{ \pi \mid (\forall i.~\exists j > i.~ \pi [j] \in \textsc {src}(H)) \rightarrow (\forall i.~\exists j > i.~ (\pi [j], \pi [j+1]) \in H) \}, \end{aligned}$$

where $\textsc {src}(H)$ contains the sources $\{u \mid (u,v)\in H\}$ of the edges in $H$. A strategy $\sigma$ for player p satisfies a strategy template $\varPsi$ if it is winning in the game $(G,\varPsi )$ for player p. Intuitively, $\sigma$ satisfies a strategy template if every play $\pi$ consistent with $\sigma$ for player p is contained in $\varPsi$, that is, (i) $\pi$ never uses the unsafe edges in $U$ (i.e., $\pi \in \varPsi _U$), (ii) $\pi$ stops using the co-live edges in $D$ eventually (i.e., $\pi \in \varPsi _D$), and (iii) for every live-group $H\in \mathcal {H}$, if $\rho$ visits $\textsc {src}(H)$ infinitely often, then it also uses the edges in $H$ infinitely often (i.e., $\pi \in \varPsi _\mathcal {H}$). Strategy templates can be used as a concise representation of winning strategies as formalized next.

Definition 3

(Winning Strategy Template [2]). A strategy template $\varPsi$ for player p is winning if every strategy satisfying $\varPsi$ is winning for p in $(G,\varOmega )$.

We note that the algorithms for computing winning strategy templates in safety, Büchi, co-Büchi, and parity games, presented in [2], exhibit the same worst-case computation time as standard methods for solving such (finite-state) games.

3 Attractor Computation with Caching

As outlined in Sect. 1, the core of our method consists of the pre-computation of attractor sets for local sub-games and the utilization of the results in the attractor computations performed when solving the complete reactive program game. We call the pre-computed results attractor cache. We use the cache during attractor computations to directly add to the computed attractor sets of states from which, based on the pre-computed information, the respective player can enforce reaching the current attractor subset. In that way, if the local attractor computation requires acceleration, we can avoid performing the acceleration during the attractor computation for the overall game. This section presents the formal definition of an attractor cache and shows how it is used.

Intuitively, an attractor cache is a finite set of tuples or cache entries of the form $(\mathcal {G}, p, src , targ , \mathbb {X}_ ind )$. $\mathcal {G}$ is a reactive program game structure and p the player the cache entry applies to. The sets of states $src , targ \in \mathcal {D}$ are related via enforceable reachability: player p can enforce reaching $\llbracket targ \rrbracket$ from $\llbracket src \rrbracket$ in $\mathcal {G}$. $\mathbb {X}_ ind$ are the so-called independent variables – the enforcement relation must hold independently of and preserve the values of $\mathbb {X}_ ind$. Independent variables are useful when a cache entry only concerns a part of the game structure where these variables are irrelevant. This allows the utilization of the cache entry under different conditions on those variables. We formalize this intuition in the next definition.

Definition 4

(Attractor Cache). A finite set $C \subseteq \textsf{RPGS}\times \{ Sys , Env \} \times \mathcal {D}\times \mathcal {D}\times 2^{\mathbb {X}}$ is called an attractor cache if and only if for all $(\mathcal {G}, p, src , targ , \mathbb {X}_ ind ) \in C$ and all $\varphi \in FOL (\mathbb {X}_ ind )$ it holds that $\llbracket src \wedge \lambda l. ~\varphi \rrbracket \subseteq Attr _{\llbracket \mathcal \rrbracket {G},p}(\llbracket targ \wedge \lambda l. ~\varphi \rrbracket ).$

We use the lambda abstraction $\lambda l. ~\varphi$ to denote the anonymous function that maps each location in L to the formula $\varphi$.

Example 2

Recall the game from Example 1. From every state with location $mine$, player $Sys$ can enforce eventually reaching $samp \ge req$ by choosing at every step the update that increases variable $samp$. As this argument only concerns location $mine$, the program variables $done$ and $pos$ are independent. Since it is not updated, $req$ is also independent (we prove this in the next section). Hence, $C_ ex = \{ (\mathcal {G}_ ex , Sys , src , targ , \mathbb {X}_ ind ) \}$ where $\mathcal {G}_ ex$ is from Fig. 2, $src = \{ mine \mapsto \top \}$, $targ = \{ mine \mapsto samp \ge req \}$, and $\mathbb {X}_ ind = \{ done , pos , req \}$ is an attractor cache.

Algorithm 1

Attractor computation using an attractor cache.

Algorithm 1 shows how we use an attractor cache to enhance accelerated attractor computations. AttractorAccCache extends the procedure AttractorAcc for accelerated symbolic attractor computation presented in [21]. AttractorAccCache takes a cache as an additional argument and at each iteration of the attractor computation checks if some cache entry is applicable. For each such cache entry, if $\llbracket targ \rrbracket$ is a subset of $\llbracket a^n\rrbracket$, we can add $src$ to $a^n$ since we know that $targ$ is enforceable from $src$. However, $a^n$ may constrain the values of $\mathbb {X}_ ind$ making this subset check fail unnecessarily. Therefore, StrengthenTarget computes a formula $\varphi \in FOL (\mathbb {X}_ ind )$ such that $targ$ strengthened with $\varphi$ is a subset of $a^n$. Intuitively, $\varphi$ describes the values of the independent variables that remain unchanged in the cached attractor. Note that $\varphi$ always exists as we could pick $\bot$, which we have to do if $targ$ is truly not a subset of $a^n$.

The next lemma formalizes this intuition and the correctness of AttractorAccCache under the above condition on StrengthenTarget. Note that since the cache is used in the context of attractor computation, the objective $\varOmega$ of the reactive program game is not relevant here.

Lemma 1

(Correctness of Cache Utilization). Let $\mathcal {G}$ be a reactive program game structure, $p \in \{ Sys , Env \}$, $d \in \mathcal {D}$ and C be an attractor cache. Furthermore, suppose that for every $targ \in \mathcal {D}$, $a \in \mathcal {D}$ and every $\mathbb {X}_ ind \subseteq \mathbb {X}$ it holds that if $\textsc {StrengthenTarget}( targ , \mathbb {X}_ ind , a) = \varphi$, then $\varphi \in FOL (\mathbb {X}_ ind )$ and $\llbracket targ \wedge \lambda l. ~\varphi \rrbracket \subseteq \llbracket a\rrbracket$. Then, if the procedure $\textsc {AttractorAccCache}(\mathcal G, p, d, C)$ terminates returning $attr \in \mathcal {D}$, then it holds that $\llbracket attr \rrbracket = Attr _{\llbracket \mathcal \rrbracket {G},p}(\llbracket d\rrbracket ) \cap \mathcal {S}_ Env$.

We realize $\textsc {StrengthenTarget}( targ ,\mathbb {X}_ ind ,a)$ such that it returns the formula $\bigwedge _{l \in L} \big (\forall (\mathbb {X}\backslash \mathbb {X}_ ind ).~ targ (l) \rightarrow a(l)\big )$ which satisfies the condition in Lemma 1.

Example 3

Recall the game from Example 1 and the cache $C_ ex$ from Example 2. Suppose that we are computing the attractor for player $Sys$ to $d = \{ base \mapsto \top \}$, i.e. $\textsc {AttractorAccCache}(\mathcal {G}_ ex , Sys , d, C_ ex )$ without acceleration, i.e., $\textsf{Accelerate}$ returns $\bot$ in line 11 in Algorithm 1. Initially, $a^1 = \{ base \mapsto \top \}$. After one iteration of applying $Cpre$, we get $a^2 = \{ base \mapsto \top , move \mapsto pos = 12 \wedge done = 1\}$. Then we get $a^3 = \{ \dots , mine \mapsto pos = 12 \wedge samp \ge req \}$. In the only entry of $C_ ex$, the target set $targ = \{ mine \mapsto samp \ge req \}$ contains more states in $mine$ (i.e., all possible positions of the robot) then $a^3$ (which asserts $pos = 12$). However, $\textsc {StrengthenTarget}( targ , \mathbb {X}_ ind , a^3)$ as implemented above, will return the strengthening $pos = 12$ (after simplifying the formula), which makes the cache entry with $targ$ applicable. Since $src = \{ mine \mapsto \top \}$, we update $a^3$ to $\{ \dots , mine \mapsto pos = 12\}$ in line 10 of the algorithm.

4 Abstract Template-Based Cache Generation

Section 3 defined attractor caches and showed their utilization for attractor computations via Algorithm 1. We motivated this approach by the observation that there often exist small local sub-games that entail essential attractors, and pre-computing these attractors within the sub-games, caching them and then using them via Algorithm 1 is more efficient then only applying acceleration over the entire game (as in [21]). To formalize this workflow, Sect. 4.1 explains the generation of cache entries from sub-game structures of the given reactive program game, and Sect. 4.2 discusses the identification of helpful sub-game structures via permissive strategy templates in finite-state abstractions of the given game.

4.1 Generating Attractor Caches from Sub-Games

Within this subsection, we consider a sub-game structure $\mathcal {G}'$ which is induced by a subset of locations $L_ sub \subseteq L$ of the original reactive program game structure $\mathcal {G}$, as formalized next. Intuitively, we remove all locations from $\mathcal {G}$ not in $L_ sub$ and redirect their incoming transitions to a new sink location $\textsf{sink}_ sub$.

Definition 5

(Induced Sub-Game Structure). Let $\mathcal {G}= (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ be a reactive program game structure and let $L_ sub \subseteq L$ be a set of locations. The sub-game structure induced by $L_ sub$ is the reactive program game structure

$\textsf{SubGame}(\mathcal {G},L_ sub ) := (T,\mathbb {I},\mathbb {X}',L', Inv ',\delta ')$ where $L' := L_ sub \cup \{\textsf{sink}_{ sub }\}$,

$\mathbb {X}' : = \{ x \in \mathbb {X}\mid x~\text {appears in transitions from or invariants of } L_{ sub } \text { in } \mathcal {G}' \}$,

$Inv '(l) := Inv (l)$ for all $l \in L' \setminus \{\textsf{sink}_{ sub }\}$ and $Inv '(\textsf{sink}_{ sub }) := \top$, and

$\begin{array}{lll} \delta ' &{} := &{} \{(l,g,u,l') \in \delta \mid l,l' \in L' \} \cup \{(\textsf{sink}_{ sub },\top ,\lambda x.\;x,\textsf{sink}_{ sub })\} \cup \\ {} &{}&{} \{(l,g,\lambda x.x,\textsf{sink}_{ sub }) \mid \exists l' \in L.\; (l,g,u,l') \in \delta \wedge l\in L' \wedge l' \not \in L'\}. \end{array}$

Recall that $\mathcal {D}= L \rightarrow FOL (\mathbb {X})$. Let $\mathcal {D}' := L' \rightarrow FOL (\mathbb {X}')$ be the symbolic domain for a sub-game structure with locations $L'$. As $\mathbb {X}' \subseteq \mathbb {X}$, $FOL (\mathbb {X}') \subseteq FOL (\mathbb {X})$ which allows us to extend each element of $\mathcal {D}'$ to an element of $\mathcal {D}$ that agrees on $L'$. Formally, we define $\textsf{extend}_{L} : \mathcal {D}' \rightarrow \mathcal {D}$ such that for $d'\in \mathcal {D}'$ and $l \in L$ we have $\textsf{extend}_{L} (d')(l) := \texttt {if } l \in L' \texttt { then }d'(l) \texttt { else } \bot$.

The computation of an attractor cache from an induced sub-game is detailed in Algorithm 2. Given a reactive program game structure $\mathcal {G}$, a player p, and a subset of locations $L_ sub$, Algorithm 2 first computes the induced sub-game (line 2). The quantifier elimination ([7, Ch. 7]) $\textsf{QElim}$ in line 3 projects the given $d \in \mathcal {D}$ to an element $d'$ of the symbolic domain $\mathcal {D}'$ of the sub-game structure. Then, in line 4, we perform the accelerated attractor computation from [21] with target set $d'$ to obtain the set of states a from which player p can enforce reaching $d'$ in $\mathcal {G}'$. The independent variables are those variables in $\mathbb {X}$ that are not updated in any of the transitions in $\mathcal {G}'$. Formally, we define those as $\textsf{IndependentVars}(\mathcal {G},\mathcal {G}'):=\{x \in \mathbb {X}\mid \forall (l, g, u, l') \in \delta '.~u(x)=x\}$. In order to output an attractor cache for the original game $\mathcal {G}$, we extend the computed source and target sets a and $d'$ via the previously defined function $\textsf{extend}_{L}$ (line 6). Intuitively, the attractor computed over a sub-game $\mathcal {G}'$ is also an attractor for the overall game $\mathcal {G}$ as sub-games are only restricted by location (not by variables). Hence, player p can also enforce reaching the target set in the original game $\mathcal {G}$, if he can do so in $\mathcal {G}'$. This is formalized by the next lemma.

Lemma 2

Let $\mathcal {G}= (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ be a reactive program game structure, and let $\mathcal {G}'= (T,\mathbb {I},\mathbb {X}',L', Inv ',\delta ')$ be an induced sub-game structure with sink location $\textsf{sink}_{ sub }$ constructed as above. Let $src ', targ ' \in \mathcal {D}'$ be such that $targ '(\textsf{sink}_{ sub }) = \bot$ and $\llbracket src '\rrbracket \subseteq Attr _{\llbracket \mathcal \rrbracket {G'},p}(\llbracket targ '\rrbracket )$ for some player $p\in \{ Sys , Env \}$. Furthermore, let $\mathbb {Y} \subseteq \textsf{IndependentVars}(\mathcal {G},\mathcal {G}')$. Then, for every $\varphi \in FOL (\mathbb {Y} )$ it holds that

$$\llbracket \textsf{extend}_L( src ') \wedge \lambda l.\varphi \rrbracket \subseteq Attr _{\llbracket \mathcal \rrbracket {G},p}(\llbracket \textsf{extend}_L( targ ') \wedge \lambda l.\varphi \rrbracket ).$$

This results in the following correctness statement.

Lemma 3

$\textsc {SubgameCache}(\mathcal {G}, p, L_ sub , d)$ returns an attractor cache over $\mathcal {G}$.

Algorithm 2

Cache generation based on an induced sub-game.

Fig. 3.

Induced sub-game structure $\textsf{SubGame}(\mathcal {G}_ ex ,\{ mine \})$ of the reactive program game structure $\mathcal {G}_ ex$ from Fig. 2, with the same abbreviations as in Fig. 2.

Example 4

Consider the reactive program game structure $\mathcal {G}_ ex$ from Example 1. We apply $\textsc {SubgameCache}(\mathcal {G}_ ex , Sys , \{ mine \}, d)$ with $d =\{ mine \mapsto samp \ge req \wedge pos = 12 \wedge done \ne 1 \}$. First, we construct the induced sub-game structure in Figrue 3. Quantifier elimination produces the target set $d' = \{ mine \mapsto samp \ge req \}$. If we compute the attractor in this sub-game to set $d'$, we get $\{ mine \mapsto \top \}$. Note that since the number of steps needed to reach $d'$ depends on the initial value of $samp$ and is hence unbounded, a technique like acceleration [21] is necessary to compute this attractor. As in this sub-game structure only the variable $samp$ is updated, the independent variables are $\mathbb {X}_ ind = \{ done , pos , req \}$. With this we get the cache entry from Example 2.

4.2 Constructing Sub-games from Abstract Strategy Templates

The procedure from the previous subsection yields attractor caches regardless of how the sub-games are chosen. In this section we describe our approach to identifying “useful” sub-game structures. These sub-game structures are induced by so-called helpful edges determined by permissive strategy templates. Since the game graph described by a reactive program game structure is in general infinite, we first construct finite abstract games in which we compute permissive strategy templates for the two players. We start by describing the abstract games.

Finite Abstractions of Reactive Program Games. Here we describe the construction of a game graph $\widehat{G}=(V, V_ Env , V_ Sys , \widehat{\rho })$ from a reactive program game structure $\mathcal {G}= (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ with semantics $\llbracket \mathcal {G}\rrbracket = (\mathcal {S}, \mathcal {S}_ Env ,\mathcal {S}_ Sys , \rho )$. While $\llbracket \mathcal {G}\rrbracket$ is also a game graph, its vertex set is typically infinite. The game graph $\widehat{G}$, which is an abstraction of $\llbracket \mathcal {G}\rrbracket$, has a finite vertex set instead.

We construct the game graph $\widehat{G}$ from $\mathcal {G}$ by performing abstraction with respect to a given abstract domain. The abstract domain consists of two finite sets of quantifier-free first-order formulas which are used to define the vertex sets of the game graph $\widehat{G}$. The conditions that we impose in the definition of abstraction domain given below ensure that it can partition the state space of $\mathcal {G}$.

Definition 6

(Game Abstraction Domain). A game abstraction domain for a reactive program game structure $\mathcal {G}= (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ is a pair of finite sets of quantifier-free first-order formulas $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}}) \in QF (\mathbb {X}) \times QF (\mathbb {X}\cup \mathbb {I})$ such that for $\mathcal {P} = \mathcal P_{\mathbb {X}}$ (resp. $\mathcal {P}= \mathcal {P}_{\mathbb {X}\cup \mathbb {I}}$) and $V = \mathbb {X}$ (resp. $V = \mathbb {X}\cup \mathbb {I}$), $\mathcal {P}$ partitions $Assignments (V)$, i.e. $Assignments (V) = \bigcup _{\varphi \in \mathcal {P}}\{ \textbf{v} \mid \textbf{v} \models _{T} \varphi \}$ and for every $\varphi _1,\varphi _2 \in \mathcal {P}$ with $\varphi _1\wedge \varphi _2$ satisfiable it holds that $\varphi _1 = \varphi _2$.

The abstraction domain we use consists of all conjunctions of atomic predicates (and their negations) that appear in the guards of the reactive program game structure $\mathcal {G}$. Let $GA$ be the set of atomic formulas appearing in the guards of $\mathcal {G}$. We use the abstraction domain $\textsf{AbstractDomain}(\mathcal {G}):=(\mathcal P_{\mathbb {X}}^{ GA },\mathcal {P}_{\mathbb {X}\cup \mathbb {I}}^{ GA })$ where

$$ \begin{array}{ll} \mathcal P_{\mathbb {X}}^{ GA } &{} := \{ \bigwedge \nolimits _{\varphi \in J}\varphi \wedge \bigwedge \nolimits _{\varphi \not \in J} \lnot \varphi \mid J \subseteq GA \cap FOL (\mathbb {X})\}, \\ \mathcal {P}_{\mathbb {X}\cup \mathbb {I}}^{ GA } &{} := \{ \bigwedge \nolimits _{\varphi \in J}\varphi \wedge \bigwedge \nolimits _{\varphi \not \in J} \lnot \varphi \mid J \subseteq GA \cap ( FOL (\mathbb {X}\cup \mathbb {I})\setminus FOL (\mathbb {X}))\}. \end{array} $$

Example 5

In the game structure $\mathcal {G}_ ex$ from Example 1, we get for $\mathcal P_{\mathbb {X}}^{ GA }$ all combinations of $\varphi _1 \wedge \varphi _2 \wedge \varphi _3$, where $\varphi _1 \in \{ req < samp , req \ge samp \}$, $\varphi _2 \in \{ pos = 12, pos = 23, pos \ne 12 \wedge pos \ne 23\}$, and $\varphi _3 \in \{ task = 1, task \ne 1 \}$. For $\mathcal {P}_{\mathbb {X}\cup \mathbb {I}}^{ GA }$ we get all combinations of $\psi _1 \wedge \psi _2 \wedge \psi _3$, where $\psi _1 \in \{a \le 0, a > 0\}$, $\psi _2 \in \{ b \le 0, b > 0\}$, and $\psi _3 \in \{ inpReq \le 0 , inpReq > 0 \}$.

We choose this abstraction domain as a baseline since the predicates appearing in the guards are natural delimiters in the program variable state space. However, the abstraction we define now is independent of this specific domain.

Given a game abstraction domain $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$, we construct two abstract game graphs, $\widehat{G}^\uparrow$ and $\widehat{G}^\downarrow$. They have the same sets of vertices but differ in the transition relations. The transition relation in $\widehat{G}^\uparrow$ overapproximates the transitions originating from states of player $Sys$ and underapproximates the transitions from states of player $Env$. In $\widehat{G}^\downarrow$ the approximation of the two players is reversed.

Definition 7

(Abstract Game Graphs). Let $\mathcal {G}= (T,\mathbb {I}, \mathbb {X}, L, Inv ,\delta )$ be a reactive program game structure and $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$ be an abstraction domain. The game graphs $\widehat{G}^\circ =(V, V_ Env , V_ Sys , \widehat{\rho }^\circ )$ with $\circ \in \{\uparrow ,\downarrow \}$ are the $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$-induced abstractions of $\mathcal {G}$ if $V := V_ Sys \cup V_ Env$, $V_ Env := L \times \mathcal P_{\mathbb {X}}$, and $V_ Sys := L \times \mathcal P_{\mathbb {X}}\times \mathcal {P}_{\mathbb {X}\cup \mathbb {I}}$; and $\widehat{\rho }^\circ \subseteq (V_ Env \times V_ Sys ) \cup (V_ Sys \times V_ Env )$ is the smallest relation such that

• $((l, \varphi ), (l, \varphi , \varphi _I)) \in \widehat{\rho }^\circ \cap (V_ Env \times V_ Sys )$ iff the following formula is valid

  $$ move ^\bullet _\mathbb {X}( Inv (l) \wedge \varphi (\mathbb {X}), \exists \mathbb {I}.~ \varphi _I(\mathbb {X}, \mathbb {I}))$$

     for $\bullet = \downarrow$ if $\circ = \uparrow$ and $\bullet = \uparrow$ if $\circ = \downarrow$,

• $((l, \varphi , \varphi _I), (l', \varphi ')) \in \widehat{\rho }^\circ \cap (V_ Sys \times V_ Env )$ iff the following formula is valid

  $$ move ^\circ _{\mathbb {X}\cup \mathbb {I}}\left( Inv (l) \wedge \varphi (\mathbb {X}) \wedge \varphi _I(\mathbb {X}, \mathbb {I}), \exists (g , u) \in Labels (l, l').~ trans (g,u,l',\varphi ')\right) $$

     for $trans (g,u,l',\varphi '):=g(\mathbb {X}, \mathbb {I}) \wedge \big (\varphi '\wedge Inv (l')\big )(u(\mathbb {X}, \mathbb {I}))$,

where $move ^\uparrow _{V}(\varphi , \varphi ') := \exists V. \varphi (V) \wedge \varphi '(V)$ and $move ^\downarrow _{V}(\varphi , \varphi ') := \forall V. \varphi (V) \rightarrow \varphi '(V)$.

Definition 7 provides us with a procedure AbstractRPG for constructing the pair of abstractions $(\widehat{G}^\uparrow ,\widehat{G}^\downarrow ) := \textsc {AbstractRPG}(\mathcal {G},(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}}))$.

We refer to the vertices in the abstract game graphs as abstract states. By slightly overloading notation, we define the projection from abstract states $v\in V$ to the respective location by $loc : V \rightarrow L$ s.t. $loc ((l,\varphi )) =l$ and $loc ((l,\varphi ,\varphi _I)) =l$. This definition naturally extends to sequences of abstract states $\pi \in V^\infty$ s.t. $loc (\widehat{\pi })[i] = loc (\widehat{\pi }[i])$ for all $i \in dom(\pi )$, and to sets of vertices: $loc : 2^V \rightarrow 2^L$. Given $\mathcal {G}$ with semantics $\llbracket \mathcal {G}\rrbracket = (\mathcal {S}, \mathcal {S}_ Env ,\mathcal {S}_ Sys , \rho )$ and an abstract game graph $\widehat{G}^\circ =(V, V_ Env , V_ Sys , \widehat{\rho }^\circ )$, we define the following functions between their respective state spaces. The concretization function $\gamma : V \rightarrow 2^\mathcal {S}$ is defined s.t.

$$ \begin{array}{ll} \gamma ((l,\varphi )) &{}:= \{ (l, \textbf{x}) \in \mathcal {S}_ Env \mid \textbf{x} \models _{T} \varphi \}~\text {and}\\ \gamma ((l,\varphi ,\varphi _I)) &{}:= \{ ((l, \textbf{x}),\textbf{i}) \in \mathcal {S}_ Sys \mid \textbf{x} \uplus \textbf{i} \models _{T} \varphi \wedge \varphi _I \}. \end{array} $$

The abstraction function $\alpha : \mathcal {S}\rightarrow 2^V$ is defined s.t. $v\in \alpha (s)$ iff $s\in \gamma (v)$. We extend both function from states to (finite or infinite) state sequences $\pi \in \mathcal {S}^\infty$ and $\widehat{\pi }\in V^\infty$ s.t.

$$ \begin{array}{ll} \gamma (\widehat{\pi }) &{}:=\{ \pi \in \mathcal {S}^\infty \mid |\pi |=|\widehat{\pi }| \wedge \forall i\in dom(\pi ).~\pi [i] \in \gamma (\widehat{\pi }[ i])\},~\text {and}\\ \alpha ( \pi ) &{}:=\{ \widehat{\pi }\in V^\infty \mid |\pi |=|\widehat{\pi }| \wedge \forall i\in dom(\pi ).~\widehat{\pi }[i] \in \alpha (\pi [ i])\}. \end{array} $$

Both functions naturally extend to sets of states or infinite sequences of states by letting $\gamma (A) : = \bigcup _{a\in A} \gamma (a)$ for $A \subseteq V$ and $A\subseteq V^\omega$ and $\alpha (C) : = \bigcup _{c\in C} \alpha (c)$ for $C \subseteq \mathcal {S}$ and $C\subseteq \mathcal {S}^\omega$. Note that it follows from the partitioning conditions imposed on $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$ in Definition 6 that $\alpha$ is a total function and always maps states and state sequences to a singleton set. We abuse notation and write $\alpha (s) = v$ (resp. $\alpha (\pi ) = \widehat{\pi }$) instead of $\alpha (s) = \{v\}$ (resp. $\alpha (\pi ) = \{\widehat{\pi }\}$).

Let $\varOmega \subseteq \mathcal {S}^\infty$ be an objective for the semantic game $\llbracket \mathcal {G}\rrbracket$. With the relational functions $\langle \alpha ,\gamma \rangle$ defined before, $\varOmega$ naturally induces an abstract objective $\widehat{\varOmega } := \alpha (\varOmega )\subseteq V^\infty$ over the abstract state space V.

Recall that we consider winning conditions $\varOmega \subseteq \mathcal {S}^\infty$ for $\llbracket \mathcal {G}\rrbracket$ defined over the set L of locations of $\mathcal {G}$. As $\alpha$ preserves the location part of the states, $\widehat{\pi }\in \widehat{\varOmega }$ iff $\gamma (\widehat{\pi }) \subseteq \varOmega$. That is, a sequence of abstract states is winning according to $\widehat{\varOmega }$ iff all the corresponding concrete state sequences are winning according to $\varOmega$.

The next lemma states the correctness property that the abstraction satisfies. More concretely, $\widehat{G}^\uparrow$ overapproximates the winning region of player $Sys$ in the concrete game, and $\widehat{G}^\downarrow$ underapproximates it.

Lemma 4

(Correctness of the Abstraction). Given a reactive program game structure $\mathcal {G}$ with semantics $\llbracket \mathcal {G}\rrbracket = (\mathcal {S}, \mathcal {S}_ Env ,\mathcal {S}_ Sys , \rho )$ and location-based objective $\varOmega$, let $\widehat{G}^\circ =(V, V_ Env , V_ Sys , \widehat{\rho }^\circ )$ with $\circ \in \{\uparrow , \downarrow \}$ be its $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$-induced abstractions with relational functions $\langle \alpha ,\gamma \rangle$. Then it holds that (1) $W_{ Sys }(\llbracket \mathcal {G}\rrbracket ,\varOmega ) \subseteq \gamma (W_{ Sys }(\widehat{G}^\uparrow ,\widehat{\varOmega }))$, and (2) $\gamma (W_{ Sys }(\widehat{G}^\downarrow ,\widehat{\varOmega })) \subseteq W_{ Sys }(\llbracket \mathcal {G}\rrbracket ,\varOmega )$.

By duality, $\widehat{G}^\downarrow$ results in an overapproximation of the winning region of player $Env$ in the concrete game. Given an abstraction $\widehat{G}^\circ$, we denote with $OverapproxP (\widehat{G}^\circ )$ the player whose winning region is overapproximated in $\widehat{G}^\circ$: $OverapproxP (\widehat{G}^\circ ):= Sys$ if $\circ =\uparrow$ and $OverapproxP (\widehat{G}^\circ ):= Env$ if $\circ =\downarrow$.

Abstract Strategy Templates and Their Induced Sub-games. We now describe how we use a permissive strategy template for a player p in an abstract game to identify sub-game structures of the given reactive program game from which to generate attractor caches for player p.

We determine the sub-game structures and local target sets based on so-called helpful edges for player p in the abstract game where p is over-approximated. A helpful edge is a live-edge or an alternative choice to a co-live edge of a permissive strategy template. Intuitively, a helpful edge is an edge that player p might have to take eventually in order to win the abstract game. As our chosen abstraction domain is based on the guards, a helpful edge often corresponds to the change of conditions necessary to enable a guard in the reactive program game. Since reaching this change might require an unbounded number of steps, our method attempts a local attractor computation and potentially acceleration. Identifying helpful edges based on permissive strategy templates rather than on winning strategies has the following advantages. First, templates reflect multiple abstract winning strategies for player p, capturing multiple possibilities to make progress towards the objective. Moreover, they describe local conditions, facilitating the localization our method aims for. Helpful edges are defined as follows.

Definition 8

(Helpful Edge). Given a strategy template $(U,D,\mathcal {H})$ for player p in a game $(G,\varOmega )$ with $G=(V,V_ Env , V_ Sys ,\rho )$, we call an edge $e \in \rho$ helpful for player p w.r.t. the template $(U,D,\mathcal {H})$ if and only if the following holds: There exists a live-group $H \in \mathcal {H}$ such that $e \in H$, or $e \not \in U\cup D$ and there exists a co-live edge $(v_s,v_t) \in D$ with $v_s = \textsc {src}(e)$. We define $\textsf{Helpful}_{G,p}(U,D,\mathcal {H})$ to be the set of helpful edges for player p in G w.r.t. $(U,D,\mathcal {H})$.

For each helpful edge, we define pre- and post-sets which are the abstract environment states before and after that edge. This is formalized as follows.

Definition 9

(Pre- and Post-Sets). Let $\widehat{G}^\circ =(V, V_ Env , V_ Sys , \widehat{\rho }^\circ )$ for some $\circ \in \{\uparrow ,\downarrow \}$ be a $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$-induced abstraction of $\mathcal {G}$, let $p_{ over }:= OverapproxP (\widehat{G}^\circ )$, and $e = (v_s,v_t) \in \textsf{Helpful}_{\widehat{G}^\circ ,p_{ over }}(U,D,\mathcal {H})$ for some template $(U,D,\mathcal {H})$. If $p_{ over } = Env$, we have $e \in V_ Env \times V_ Sys$ and define $\textsf{Pre}(e,p_{ over }) := \{ v_s \}$ and $\textsf{Post}(e,p_{ over }) := \{ v \in V \mid (v_t,v) \in \widehat{\rho }^\circ \}$. If $p_{ over } = Sys$ we have that $e \in V_ Sys \times V_ Env$ and define $\textsf{Pre}(e,p_{ over }) := \{ v \in V \mid (v,v_s) \in \widehat{\rho }^\circ \}$ and $\textsf{Post}(e,p_{ over }) := \{ v_t \}$. Note that in both cases it holds that $\textsf{Pre}(e,p_{ over }), \textsf{Post}(e,p_{ over }) \subseteq V_ Env \subseteq L \times \mathcal P_{\mathbb {X}}$.

Algorithm 3

Generation of a cache based on a strategy template.

As a helpful edge represents potential “progress” for player p, we consider the question of whether player p has a strategy in the concrete game to reach the post-set from the pre-set. This motivates the construction of sub-game structures induced by the locations connecting those two sets in the reactive program game.

Procedure GenerateCache in Algorithm 3 formalizes this idea. It takes an abstract game and a strategy template for the over-approximated player $p_ over$ in this game. For each helpful edge e, it constructs the sub-game structure induced by the set of locations that lie on a simple path in the location graph from the locations of the pre-set to the post-set of e. The optional parameter b allows for heuristically tuning the locality of the sub-games by bounding the paths’ length.

For each sub-game structure, the target sets for the local attractor computations are determined by the post-sets of the helpful edges that induced this sub-game structure (it might be more than one). They are computed by

$$\begin{aligned} \textsc {ConstructTargets}(L_ sub , PostSet ) = T_1 \cup T_2 \cup T_3 \end{aligned}$$

(1)

where the sets $T_1,T_2 \text { and }T_3$ of elements of $\mathcal {D}$ are defined as follows.

• $T_1:=\{d\in \mathcal {D}\mid \exists P. \;(L_ sub ,P) \in PostSet \wedge \forall l \in L.\;d(l) = \bigvee _{(l,\varphi ) \in P} \varphi \}$ consists of targets that are determined by a single post-set.

• $T_2:=\{d_\cup \}$, where for every $l \in L$, $d_\cup (l) = \bigvee _{P \text { s.t. } (L_ sub ,P) \in PostSet } \bigvee _{(l,\varphi ) \in P} \varphi$ is the singleton containing the union of the targets of all post-sets.

• $T_3:=\{d_\top \}$, where for $l \in L$, $d_\top (l)= \exists P, \varphi . (L_ sub ,P) \in PostSet \wedge (\varphi , l) \in P$ contains the target that is $\top$ iff the location appears in some post-set.

Once the targets are constructed, GenerateCache uses SubgameCache from Algorithm 2 to compute the attractor caches for those targets and respective sub-game structures. By Lemma 3, SubgameCache returns attractor caches. As attractor caches are closed under set union, we conclude the following.

Corollary 1

The set C returned by $\textsc {GenerateCache}$ is an attractor cache.

Example 6

The abstractions of $\mathcal {G}_ ex$ from Example 1 and respective templates are too large to depict. One helpful edge for $Sys$ is $e = (( mine , \varphi , \varphi _I), ( mine , \varphi '))$ with , , and $\varphi _I = a > 0 \wedge b \le 0 \wedge inpReq \le 0$. This edge e is in a live group where the other edges are similar with different $\varphi _I$. They correspond to the situation where the value of $samp$ finally becomes greater or equal to $req$. For e, $\textsf{Pre}(e, Sys ) = \{( mine , \varphi )\}$ and $\textsf{Post}(e, Sys ) = \{( mine , \varphi ')\}$ result in $L_ sub = \{ mine \}$ and the target $\{ mine \mapsto \varphi ' \}$. With this, we generate a cache as in Example 4.

5 Game Solving with Abstract Template-Based Caching

Algorithm 4

Game solving with abstract template-based caching.

This section summarizes our approach for reactive progam game solving via Algorithm 4, which combines the procedures introduced in Sect. 3 and Sect. 4 as schematically illustrated in Fig. 1 of Sect. 1. Algorithm 4 starts by computing the abstract domain and both abstractions. For each abstract game, $\textsc {SolveAbstract}$ computes a strategy template [2]. Then, GenerateCache is invoked to construct the respective attractor cache. RPGSolveWithCache solves reactive program games in direct analogy to RPGSolve from [21], but instead of using AttractorAcc, it uses the new algorithm AttractorAccCache which utilizes the attractor cache C. The overall correctness of RPGCacheSolve follows from Lemma 1, Corollary 1, and the correctness of [21].

Theorem 1

(Correctness). Given a reactive program game structure $\mathcal {G}$ and a location-based objective $\varOmega$, for any $b \in \mathbb {N}$, if RPGCacheSolve terminates, then it returns $W_{ Sys }(\llbracket \mathcal {G}\rrbracket ,\varOmega ).$

Remark 1

In addition to using the strategy templates from the abstract games for caching, we can make use of the winning regions in the abstract games, which are computed together with the templates. Thanks to Lemma 4, we know that outside of its winning region in the abstract game the over-approximated player loses for sure. Thus, we can prune parts of the reactive program game that correspond to the abstract states where the over-approximated player loses. As our experiments show that the main performance advantage is gained by caching rather than pruning, we give the formal details for pruning in the extended version [34].

Discussion. The procedure RPGCacheSolve depends on the choice of game abstraction domain $(\mathcal P_{\mathbb {X}},\mathcal {P}_{\mathbb {X}\cup \mathbb {I}})$ and on the construction of the local games performed in GenerateCache. The abstraction based on guards is natural, as it is obtained from the predicates appearing in the game. Acceleration [21] is often needed to establish that some guards can eventually be enabled. Therefore, we choose an abstraction domain that represents precisely the guards in the game.

Helpful edges capture transitions that a player might need to take, hence the game solving procedure has to establish that the player can eventually enable their guards. This might require acceleration, and hence motivates our use of helpful edges to construct the local games. Investigating alternatives to these design choices and their further refinement is a subject of future work.

6 Experimental Evaluation

Table 1. Evaluation Results. ST is the variable domain type (additional to $\mathbb {B}$). |L|, $|\mathbb {X}|$, $|\mathbb {I}|$ are the number of respective game elements. We show the wall-clock running time in seconds for our prototype rpg-STeLA in three settings (one with normal caching, one with additional pruning, one that only prunes), rpgsolve, and MuVal (with clause exchange). TO means timeout after 30 min, MO means out of memory (8GB). We highlight in bold the fastest solving runtime result. The evaluation was performed on a computer equipped with an Intel(R) Core(TM) i5-10600T CPU @ 2.40 GHz.

We implemented Algorithm 4 for solving reactive program games in a prototype tool² rpg-STeLA (Strategy Template-based Localized Acceleration). Our implementation is based on the open-source reactive program game solver rpgsolve from [21]. Specifically, we use rpgsolve for the AttractorAcc and RPGSolveWithCache methods to compute attractors via acceleration and to solve reactive program games utilizing the precomputed cache, respectively. We realize SolveAbstract by using PeSTel  [2], which computes strategy templates in finite games. We do not use the bound b in Algorithm 4.

We compare our tool rpg-STeLA to the solver rpgsolve and the $\mu$CLP solver MuVal  [36]. Those are the only available techniques that can handle unbounded strategy loops, as stated in [21]. Other tools from [5, 8,9,10, 27, 28, 31,32,33] cannot handle those, are outperformed by rpgsolve, or are not available. For MuVal, we encoded the games into $\mu$CLP as outlined in [36] and done in [21].

Benchmarks. We performed the evaluation on three newly introduced sets of benchmarks (described in detail in [34]). They all have unbounded variable ranges, contain unbounded strategy loops, and have Büchi winning conditions. On the literature benchmarks from [6, 21, 27, 31, 39] rpgsolve performs well as [21] shows. Hence, we did not use them as local attractor caches are unnecessary, and they are smaller than our new ones. Our new benchmark categories are:

(1) Complex Global Strategy (Scheduler and Item Processing). These benchmarks consist of a scheduler and an item processing unit. The core feature of these benchmarks is that the system needs to perform tasks that require complex global strategic decisions and local strategic decisions requiring acceleration.

(2) Parametric Benchmarks (Chains). These benchmarks each consist of two parametric chains of local sub-tasks requiring acceleration and local strategic reasoning and more lightweight global strategic reasoning. The number of variables scales differently in both chains, showcasing differences in scalability.

(3) Simple Global Strategy (Robot and Smart Home). These benchmarks represent different tasks for a robot and a smart home. The robot moves along tracks (with one-dimensional discrete position) and must perform tasks like collecting several products. The smart home must, e.g., maintain temperature levels and adjust blinds depending on whether the house is empty or on the current time of day. These benchmarks need acceleration and local strategic reasoning, but their global reasoning is usually simpler and more deterministic.

Analysis. The experimental results in Table 1 demonstrate that local attractor pre-computation and caching have a significant impact on solving complex games. This is evidenced by the performance of rpg-STeLA that is superior to the other two tools. We further see that pruning (without caching) is not sufficient, which underscores the need to use more elaborate local strategic information in the form of an attractor cache. This necessitates the computation of strategy templates, and simply solving an abstract game is insufficient. However, as pruning does not cause significant overhead, it offers an additional optimization.

7 Related Work

A body of methods for solving infinite-state games and synthesizing reactive systems operating over unbounded data domains exists. Abstraction-based approaches reduce the synthesis problem to the finite-state case. Those include abstraction of two-player games [15, 19, 23, 37, 38], which extends ideas from verification, such as abstract interpretation and counterexample-guided abstraction refinement, to games. The temporal logic LTL has recently been extended with data properties, resulting in TSL [14] and its extension with logical theories [13]. Synthesis techniques for those [8, 14, 27] are based on propositional abstraction of the temporal specification and iterative refinement by introducing assumptions. The synthesis task’s main burden in abstraction-based methods falls on the finite-state synthesis procedure. In contrast, we use abstraction not as the core solving mechanism but as a means to derive helpful sub-games. Another class of techniques reason directly over the infinite-state space. Several constraint-based approaches [9, 10, 24] have been proposed for specific types of objectives. [32, 33] lift fixpoint-based methods for finite-state game solving to a symbolic representation of infinite state sets. However, a naive iterative fixpoint computation can be successful on a relatively limited class of games. Recently, [21] proposed a technique that addresses this limitation by accelerating symbolic attractor computations. However, as we demonstrate, their approach has limited scalability when the size of the game structure grows. Our method mitigates this by identifying small helpful sub-games and composing their solutions to solve the game.

There are many approaches for compositional synthesis from LTL specifications [11, 12, 16]. To the best of our knowledge, no techniques for decomposing infinite-state games exist prior to our work.

In verification, acceleration [3, 4, 17] and loop summarization [26] are applied to the loops in given program and can thus be easily combined with subsequent analysis. In contrast, in the setting of games, acceleration relies on establishing the existence of a strategy which needs more guidance.

Permissive strategy templates were introduced in [1] and used in [2] to represent sets of winning strategies for the system player in two-player games. They were used to synthesize hybrid controllers for non-linear dynamical systems [30]. Similar to our work, [30] uses templates over abstractions to localize the compuation of continuous feedback controllers. While this inspired the solution methodology for infinite-state systems developed in this paper, the abstraction methodology and the semantics of the underlying system and its controllers are very different in [30]. Our work is the first which uses permissive strategy templates as a guide for localizing the computation of fixpoints in infinite-state games.

8 Conclusion

We presented a method that extends the applicability of synthesis over infinite-state games towards realistic applications. The key idea is to reduce the game solving problem to smaller and simpler sub-problems by utilizing winning strategy templates computed in finite abstractions of the infinite-state game. The resulting sub-problems are solved using a symbolic method based on attractor acceleration. Thus, in our approach abstraction and symbolic game solving work in concert, using strategy templates as the interface between them. This opens up multiple avenues for future work, such as exploring different abstraction techniques, as well as developing data-flow analysis techniques for reactive program games that can be employed in the context of symbolic game-solving procedures.