Synthesizing optimally resilient controllers

Recently, Dallal, Neider, and Tabuada studied a generalization of the classical game-theoretic model used in program synthesis, which additionally accounts for unmodeled intermittent disturbances. In this extended framework, one is interested in computing optimally resilient strategies, i.e., strategies that are resilient against as many disturbances as possible. Dallal, Neider, and Tabuada showed how to compute such strategies for safety specifications. In this work, we compute optimally resilient strategies for a much wider range of winning conditions and show that they do not require more memory than winning strategies in the classical model. Our algorithms only have a polynomial overhead in comparison to the ones computing winning strategies. In particular, for parity conditions, optimally resilient strategies are positional and can be computed in quasipolynomial time.


Introduction
Reactive synthesis is an exciting and promising approach to solving a crucial problem, whose importance is ever-increasing due to ubiquitous deployment of embedded systems: obtaining correct and verified controllers for safety-critical systems. Instead of an engineer programming a controller by hand and then verifying it against a formal specification, synthesis automatically constructs a correct-by-construction controller from the given specification (or reports that no such controller exists).
Typically, reactive synthesis is modeled as a two-player zero-sum game on a finite graph that is played between the system, which seeks to satisfy the specification, and its environment, which seeks to violate it. Although this model is well understood, there are still multiple obstacles to overcome before synthesis can be realistically applied in practice. These obstacles include not only the high computational complexity of the problem, but also more fundamental ones. Among the most prohibitive issues in this regard is the need for a complete model of the interaction between the system and its environment, including an accurate model of the environment, the actions available to both players, as well as the effects of these actions.
This modeling task often places an insurmountable burden on engineers as the environments in which real-life controllers are intended to operate tend to be highly complex or not fully known at design time. Also, when a controller is deployed in the real world, a common source of errors is a mismatch between the controller's intended result of an action and the actual result. Such situations arise, e.g., in the presence of disturbances, when the effect of an action is not precisely known, or when the intended control action of the controller cannot be executed, e.g., when an actuator malfunctions. By a slight abuse of notation from control theory, such errors are subsumed under the generic term disturbance (cf. [12]).
To obtain controllers that can handle disturbances, one has to yield control over their occurrence to the environment. However, due to the antagonistic setting of the two-player zero-sum game, this would allow the environment to violate the specification by causing disturbances at will. Overcoming this requires the engineer to develop a realistic disturbance model, which is a highly complex task, as such disturbances are assumed to be rare events. Also, incorporating such a model into the game leads to a severe blowup in the size of the game, which can lead to intractability due to the high computational complexity of synthesis.
To overcome these fundamental difficulties, Dallal et al. [12] proposed a conceptually simple, yet powerful extension of infinite games termed "games with unmodeled intermittent disturbances". Such games are played similarly to classical infinite games: two players, called Player 0 and Player 1, move a token through a finite graph, whose vertices are partitioned into vertices under the control of Player 0 and Player 1, respectively; the winner is declared based on a condition on the resulting play. In contrast to classical games, however, the graph is augmented with additional disturbance edges that originate in vertices of Player 0 and may lead to any other vertex. Moreover, the mechanics of how Player 0 moves is modified: whenever she moves the token, her move might be overridden, and the token instead moves along a disturbance edge. This change in outcome implicitly models the occurrence of a disturbance-the intended result of the controller and the actual result differ-but it is not considered to be antagonistic. Instead, the occurrence of a disturbance is treated as a rare event without any assumptions on frequency, distribution, etc. This approach very naturally models the kind of disturbances typically occurring in control engineering [12].
As a non-technical example, consider a scenario with three siblings, Alice, Bob, and Charlie, and their father, Donald. He repeatedly asks Alice to fetch water from a well using a jug made of clay. Alice has three ways to fulfill that task: she may get the water herself or she may delegate it to either Bob or Charlie. In a simple model, the outcome of these strategies is identical: Donald's request for water is fulfilled. This is, however, unrealistic, as this model ignores the various ways that the execution of the strategies may go wrong. By modeling the situation as a game with disturbances, we obtain a more realistic model.
If Alice gets the jug herself, no disturbance can occur: she controls the outcome completely. If she delegates the task to Bob, the older of her brothers, Donald may get angry with her for not fulfilling her duties herself, which should not happen infinitely often. Finally, if she delegates the task to her younger brother Charlie, he might drop and break the jug, which would be disastrous for Alice. These strategies can withstand different numbers of disturbances: the first strategy does not offer any possibility for disturbances, while infinitely many (a single) disturbance cause Alice to lose when using the second (the third) strategy. This model captures the intuition about Donald's and Charlie's behavior: both events occur non-antagonistically and their frequency is unknown.
This non-antagonistic nature of disturbances is different from existing approaches in the literature and causes many interesting phenomena that do not occur in the classical theory of infinite graph-based games. In Fig. 1, we show an example of a parity game with disturbances that already exhibits some of these phenomena. In that parity game, vertices are labeled with non-negative integers, so-called colors, and Player 0 wins if the highest color seen infinitely often is even. For the sake of readability and conciseness, the parity game in Fig. 1 does not model the example given in natural language above, but is rather constructed to showcase properties of games with disturbances.
Consider, for instance, vertex v 2 . In the classical setting without disturbances, Player 0 wins every play reaching v 2 by simply looping in this vertex forever (since the highest color seen infinitely often is even). However, this is no longer true in the presence of disturbances: a disturbance in v 2 causes a play to proceed to vertex v 1 , from which Player 0 can no longer win. In vertex v 7 , Player 0 is in a similar, yet less severe situation: she wins every play with finitely many disturbances but loses if infinitely many disturbances occur. Finally, vertex v 9 falls into a third category: from this vertex, Player 0 wins every play even if infinitely many disturbances occur. In fact, disturbances partition the set of vertices from which Player 0 can guarantee to win into three disjoint regions (indicated as shaded boxes in Fig. 1): (a) vertices from which she can win if at most a fixed finite number of disturbances occur, (b) vertices from which she can win if any finite number of disturbances occurs but not if infinitely many occur, and (c) vertices from which she can win even if infinitely many disturbances occur.
The observation above gives rise to a question that is both theoretically interesting and practically important: if Player 0 can tolerate different numbers of disturbances from different vertices, how should she play to be resilient 1 to as many disturbances as possible, i.e., to tolerate as many disturbances as possible but still win? Put slightly differently, disturbances induce an order on the space of winning strategies ("a winning strategy is better if it is more resilient"), and the natural problem is to compute optimally resilient winning strategies, yielding optimally resilient controllers. Note that this is in contrast to the classical theory of infinite games, where the space of winning strategies is unstructured.
Dallal et al. [12] have solved the problem of computing optimally resilient winning strategies for safety games. Their approach exploits the existence of maximally permissive winning strategies in safety games [2], which allows Player 0 to avoid "harmful" disturbance edges during a play. In games with more expressive winning conditions, however, this is no longer possible, as witnessed by vertex v 4 in the example of Fig. 1: although Player 0 can avoid a disturbance edge by looping in v 4 forever, she needs to move to v 2 eventually in order to see an even color (otherwise she loses), thereby risking to lose if a disturbance occurs. In fact, the problem of constructing optimally resilient winning strategies for games other than safety games has been left open by Dallal, Neider, and Tabuada. In this work, we solve this problem for a large class of infinite games, including parity games.

Our contributions
In Sect. 2, we introduce the concept of resilience, which captures for each vertex how many disturbances need to occur for Player 0 to lose. This generalizes the notion of determinacy and allows us to derive optimally resilient winning strategies.
Our main result is an algorithm for computing the resilience of vertices and optimally resilient winning strategies, which we present in Sect. 3. This algorithm requires the game to have a prefix-independent winning condition, to be determined, and all its subgames to be (classically) solvable. The latter two conditions are necessary, as resilience generalizes determinacy and computing optimally resilient strategies generalizes solving games. We discuss these assumptions in Sect. 4.
The algorithm uses solvers for the underlying game without disturbances as a subroutine, which it invokes a linear number of times on various subgames. For many winning conditions, the time complexity of our algorithm thus falls into the same complexity class as solving the original game without disturbances, e.g., we obtain a quasipolynomial algorithm for parity games with disturbances, which matches the currently best known upper bound for classical parity games. Stated differently, if the three assumptions above are satisfied by a winning condition, then computing the resilience and optimally resilient strategies is not harder than determining winning regions and winning strategies (ignoring a polynomial overhead).
Our algorithm requires the winning condition of the game to be prefix-independent. We also show how to overcome this restriction by generalizing the classical notion of game reductions to the setting of games with disturbances. As a consequence, via reductions, our algorithm can be applied to prefix-dependent winning conditions. We discuss details in Sect. 4.
Altogether, we have generalized the original result of Dallal, Neider, and Tabuada from safety games to all games which are algorithmically solvable, in particular all ω-regular games.
Finally, in Sect. 5, we discuss further phenomena that arise in the presence of disturbances. Amongst others, we illustrate how the additional goal of avoiding disturbances whenever possible affects the memory requirements of strategies. Similarly, we exhibit a tradeoff between resilience and the (semantic) quality of strategies in quantitative games. Moreover, we raise the question of how benevolent disturbances can be leveraged to recover from losing a play. However, an in-depth investigation of these phenomena is outside the scope of this paper and left for future work.

Infinite games with disturbances
the set of vertices V 0 of Player 0 (denoted by circles) and the set of vertices of Player 1 (denoted by squares), and a set D ⊆ V 0 × V of disturbance edges (denoted by dashed arrows). Note that only vertices of Player 0 have outgoing disturbance edges. We require that every vertex v ∈ V has a successor v with Hence, the additional bits b j for j > 0 denote whether a standard or a disturbance edge has been taken to move from v j−1 to v j , while b 0 is always zero. We say ρ which is either some k ∈ ω (if there are finitely many disturbances, namely k) or it is equal to ω (if there are infinitely many). A play ρ is disturbance-free, if # d (ρ) = 0.
A game (with unmodeled disturbances), denoted by G = (A, Win), consists of an Win, otherwise it is winning for Player 1. Hence, winning is oblivious to occurrences of disturbances. A winning condition Win is prefix-independent if for all ρ ∈ V ω and all w ∈ V * we have ρ ∈ Win if and only if wρ ∈ Win. If Win is not prefix-independent, then it is called prefix-dependent.
In examples, we often use the parity condition, the canonical ω-regular winning condition. Let : V → ω be a coloring of a set V of vertices. The (max-) parity condition requires the maximal color occurring infinitely often during a play to be even. A game (A, Win) is a parity game, if Win = Parity( ) for some coloring of the vertices of A. In figures, we label a vertex v with color c by v/c.
In our proofs we make use of the safety condition for a given set U ⊆ V of unsafe vertices. It requires Player 0 to only visit safe vertices, i.e., Player 1 wins a play if it visits at least one unsafe vertex. Note that due to notational convenience, we specify a safety condition by giving the unsafe vertices instead of the safe ones, i.e., V \U , which is more common. A strategy for Player i ∈ {0, 1} is a function σ : if v j+1 = σ (v 0 · · · v j ) for every j with v j ∈ V i and b j+1 = 0, i.e., if the next vertex is the one prescribed by the strategy unless a disturbance edge is used.
Remark 1 A strategy σ does not have access to the bits indicating whether a disturbance occurred or not. However, this is not a restriction for Player 0: let be a play with b j = 1 for some j > 0. We say that this disturbance is consequential (w. traversed by the play did not lead to the vertex the strategy prescribed. Such consequential disturbances can be detected by comparing the actual vertex v j to σ 's output σ (v 0 · · · v j−1 ). Hence, the bits b j denoting consequential disturbances (w.r.t. σ ) can be reconstructed by observing the sequence of vertices and by having access to the strategy σ .
On the other hand, inconsequential disturbances can just be ignored. In particular, the number of consequential disturbances is always at most the number of disturbances during each play. The update function can be extended to finite play prefixes: Upd + (v) = Init(v) and Upd + (wv) = Upd(Upd + (w), v) for w ∈ V + and v ∈ V . A next-move function Nxt : V i × M → V for Player i has to satisfy (v, Nxt(v, m)) ∈ E for all v ∈ V i and all m ∈ M. It induces a strategy σ for Player i with memory M via σ (v 0 · · · v j ) = Nxt(v j , Upd + (v 0 · · · v j )).

Fix a game (A, Win) with
We say that a strategy σ is implementable by a memory structure M, if there is a nextmove function Nxt such that M and Nxt induce σ . If σ is implementable by some memory structure, then we call σ finite-state.

Infinite games without disturbances
We can characterize the classical notion of infinite games, i.e., those without disturbances, (see, e.g., [18]) as a special case of games with disturbances. Let G be a game with vertex set V . A strategy σ for Player i in G is a winning strategy for her from v ∈ V , if every disturbance-free play that starts in v and that is consistent with σ is winning for Player i.
The winning region W i (G) of Player i in G contains those vertices v ∈ V from which Player i has a winning strategy. Thus, the winning regions of G are independent of the disturbance edges, i.e., we obtain the classical notion of infinite games. We say that Player i . Solving a game amounts to determining its winning regions. Note that every game has disjoint winning regions. In contrast, a game is determined, if every vertex is in either winning region.

Resilient strategies
Let G be a game with vertex set V and let α ∈ ω + 2. A strategy σ for Player 0 in G is α-resilient from v ∈ V if every play ρ that starts in v, that is consistent with σ , and with # d (ρ) < α, is winning for Player 0. Thus, a k-resilient strategy with k ∈ ω is winning even under at most k −1 disturbances, an ω-resilient strategy is winning even under any finite number of disturbances, and an (ω + 1)-resilient strategy is winning even under infinitely many disturbances.

Remark 2
Let v be a vertex.
1. Let α, α ∈ ω +2 with α > α . If a strategy is α-resilient from v, then it is also α -resilient from v. 2. Every strategy is 0-resilient from v. 3. A strategy is 1-resilient from v if and only if it is winning for Player 0 from v.
We define the resilience of a vertex v of G as r G (v) = sup{α ∈ ω + 2 | Player 0 has an α-resilient strategy for G from v}.
Note that the definition is not antagonistic, i.e., it is not defined via strategies of Player 1. Nevertheless, due to the remarks above, resilient strategies generalize winning strategies.

Lemma 1
Let G be a game and v a vertex of G.
Proof (1) The resilience of v is greater than zero if and only if Player 0 has a 1-resilient strategy from v due to Item 2 of Remark 2. The latter condition is equivalent to Player 0 having a winning strategy for G from v, i.e., to v ∈ W 0 (G), due to Item 3 of Remark 2.
(2) Due to Items 1 and 3 of Remark 2, the resilience of v is zero if and only if Player 0 has no winning strategy for G from v, i.e., v / ∈ W 0 (G). Due to determinacy, this is equivalent to v ∈ W 1 (G).
Note that determinacy is a necessary condition for Item 2. In an undetermined game, the vertices that are in neither winning region have resilience zero, due to Item 1, but are in particular not in W 1 (G).
A strategy σ is optimally resilient, if it is r G (v)-resilient from every vertex v. Every such strategy is a uniform winning strategy for Player 0, i.e., a strategy that is winning from every vertex in her winning region. Hence, positional optimally resilient strategies can only exist in games which have uniform positional winning strategies for Player 0.
Our goal is to determine the mapping r G and to compute an optimally resilient strategy.

Computing optimally resilient strategies
To compute optimally resilient strategies, we first characterize the vertices of finite resilience in Sect. 3.1. All other vertices either have resilience ω or ω + 1. To distinguish between these possibilities, we show how to determine the vertices with resilience ω + 1 in Sect. 3.2. In Sect. 3.3, we show how to compute optimally resilient strategies using the results of the first two sections. We only consider prefix-independent winning conditions in Sects. 3.1 and 3.3. In Sect. 4, we show how to overcome this restriction.

Characterizing vertices of finite resilience
Our goal in this section is to characterize vertices with finite resilience in a game with prefix-independent winning condition, i.e., those vertices from which Player 0 can win even under k − 1 disturbances, but not under k disturbances, for some k ∈ ω.
To illustrate our approach, consider the parity game in Fig. 1, which is determined and has a prefix-independent winning condition. The winning region of Player 1 only contains the vertex v 1 . Thus, by Lemma 1, v 1 is the only vertex with resilience zero, every other vertex has a larger resilience. Now, consider the vertex v 2 , which has a disturbance edge leading into the winning region of Player 1. Due to this edge, v 2 has resilience at most one. This implies, as argued above, that v 2 has resilience precisely one. The unique disturbance-free play starting in v 1 is consistent with every strategy for Player 0 and violates the winning condition. Due to prefixindependence, prepending the disturbance edge does not change the winner and consistency with every strategy for Player 0. Hence, this play witnesses that v 2 has resilience at most one, while v 2 being in Player 0's winning region yields the matching lower bound. However, v 2 is the only vertex to which this reasoning applies. Now, consider v 3 : from here, Player 1 can force a play to visit v 2 using a standard edge. Thus, v 3 has resilience one as well. Again, this is the only vertex to which this reasoning is applicable.
In particular, from v 4 , Player 0 can avoid reaching the vertices for which we have already determined the resilience by using the self loop. However, this comes at a steep price for her: doing so results in a losing play, as the color of v 4 is odd. Thus, if she wants to have a chance at winning, she has to take a risk by moving to v 2 , from which she has a 1-resilient strategy, i.e., one that is winning if no more disturbances occur. For this reason, v 4 has resilience one as well. The same reasoning applies to v 6 : Player 1 can force the play to v 4 and from there Player 0 has to take a risk by moving to v 2 .
The vertices v 3 , v 4 , and v 6 share the property that Player 1 can either enforce a play violating the winning condition or reach a vertex with already determined finite resilience. These three vertices are the only ones currently satisfying this property. They all have resilience one since Player 1 can enforce to reach a vertex of resilience one, but he cannot enforce reaching a vertex of resilience zero. Now, we can also determine the resilience of v 5 : the disturbance edge from v 5 to v 3 witnesses it being two.
Afterwards, these two arguments no longer apply to new vertices: no disturbance edge leads from a vertex v ∈ {v 7 , . . . , v 10 } to some vertex whose resilience is already determined and Player 0 has a winning strategy from each such v that additionally avoids vertices whose resilience is already determined. Thus, our reasoning cannot determine their resilience. This is consistent with our goal, as all four vertices have non-finite resilience: v 7 and v 8 have resilience ω and v 9 and v 10 have resilience ω + 1. Our reasoning here cannot distinguish these two values. We solve this problem in Sect. 3.2.
We now formalize the reasoning sketched above: starting from the vertices in Player 1's winning region having resilience zero, we use a so-called disturbance update and a risk update to determine all vertices of finite resilience. A disturbance update computes the resilience of vertices having a disturbance edge to a vertex whose resilience is already known (such as vertices v 2 and v 5 in the example of Fig. 1). A risk update, on the other hand, determines the resilience of vertices from which either Player 1 can force a visit to a vertex with known resilience (such as vertices v 3 and v 6 ) or Player 0 needs to move to such a vertex in order to avoid losing (e.g., vertex v 4 ). To simplify our proofs, we describe both as monotone operators updating partial rankings mapping vertices to ω, which might update already defined values.
We show that applying these updates in alternation eventually yields a stable ranking that indeed characterizes the vertices of finite resilience.
Throughout this section, we fix a game G = (A, Win) with A = (V , V 0 , V 1 , E, D) and prefix-independent Win ⊆ V ω satisfying the following condition: the game (A, Win ∩ Safety(U )) is determined for every U ⊆ V . We discuss this requirement in Sect. 4.
A ranking for G is a partial mapping r : V ω. The domain of r is denoted by dom(r ), its image by im(r ). Let r and r be two rankings. We say that r refines Let r be a ranking for G. We define the ranking r as , and min ∅ is undefined (causing r (v) to be undefined). We call r the disturbance update of r .

Lemma 2
The disturbance update r of a sound ranking r is sound and refines r .
. This immediately implies refinement. From this inequality, we also obtain r (v) = 0 for every v ∈ W 1 (G), due to soundness of r . Finally, consider some v ∈ W 0 (G). Then, r (v) > 0 by soundness of r . Thus, r (v) > 0 as well, as both r (v) and each r (v ) + 1 are greater than zero. Altogether, r is sound as well.
Again, let r be a ranking for G. For every k ∈ im(r ) let be the winning region of Player 1 in the game where he either wins by reaching a vertex v with r (v) ≤ k or by violating the winning condition of G. Now, define where min ∅ is again undefined. We call r the risk update of r .

Lemma 3 The risk update r of a sound ranking r is sound and refines r .
Proof We show r (v) ≤ r (v) for every v ∈ dom(r ), which implies both refinement and r (v) = 0 for every v ∈ W 1 (G), as argued in the proof of Lemma 2. Thus To complete the proof of soundness of r , we just have to show r (v) > 0 for every v ∈ W 0 (G). Towards a contradiction, assume r (v) = 0, i.e., v ∈ A 0 . Thus, Player 1 has a strategy τ from v that ensures that either the winning condition is violated or that a vertex v with r (v ) = 0 is reached, i.e., v ∈ W 1 (G) by soundness of r . Hence, Player 1 has a winning strategy τ v for G from every such v . This implies that he also has a winning strategy from v: play according to τ until a vertex v with r (v ) = 0 is reached. From there, mimic τ v when starting from v . Every resulting disturbance-free play has a suffix that violates the winning condition Win. Thus, by prefix-independence, the whole play violates Win as well, i.e., it is winning for Player 1. Thus, v ∈ W 1 (G), which yields the desired contradiction, as winning regions are always disjoint.
Let r 0 be the unique sound ranking with domain W 1 (G), i.e., r 0 maps exactly the vertices in Player 1's winning region to zero, all others are undefined. Starting with r 0 , we inductively define a sequence of rankings (r j ) j∈ω such that r j for an odd (even) j > 0 is the disturbance (risk) update of r j−1 , i.e., we alternate between disturbance and risk updates.
Due to refinement, the r j eventually stabilize, i.e., there is some j 0 such that r j = r j 0 for all j ≥ j 0 . Define r * = r j 0 . Due to r 0 being sound and by Lemmas 2 and 3, each r j , and r * in particular, is sound.
Proof We show the following stronger result for every v ∈ dom(r * ): The disturbance update increases the maximal rank by at most one and the risk update does not increase the maximal rank at all. Furthermore, due to refinement, the rank of v is set and then it cannot increase. Hence, we obtain r j (v) ≤ j v +1 2 and r j (v) ≤ j v 2 for odd and even j v , respectively. In the remainder of the proof, we show a matching lower bound. We Note that as part of the proof, we have to show that the second case never occurs. Now, we show the following by induction over j, which implies the matching lower bound.
-If j is odd, then no v is updated in r j to some k < j+1 2 . -If j is even, then no v is updated in r j to some k < j 2 . For j = 0, we have j 2 = 0, and clearly, no vertex is assigned a negative rank by r 0 . For j = 1 and j = 2, we obtain j+1 2 = j 2 = 1. As r 0 , r 1 , and r 2 are sound, neither r 1 nor r 2 update some v to zero. Now, let j > 2 and first consider the case where j is odd. Towards a contradiction, assume that v ∈ V is updated in r j to some value less than j+1 2 . Since j is odd, r j is the disturbance update of r j−1 . Further, as v is updated in r j , there exists some disturbance i.e., the rank of v is stable during the last two updates.
First assume towards a contradiction r j−2 (v ) = r j−1 (v ). Then, v is updated in r j−1 to some rank of at most j−3 2 , which is in turn smaller than j−1 2 , violating the induction hypothesis for j − 1. Hence, r j−2 (v ) = r j−1 (v ). The same reasoning yields a contradiction to the assumption . The latter equality contradicts our initial assumption, namely v being updated in r j to r j (v).
Now, consider the case where j is even. Again, assume towards a contradiction that v ∈ V is updated in r j to some value less than j 2 . Since j is even, r j is the risk update of r j−1 . Further, as v is updated in r j , Player 1 wins the game (A, Hence, he has a strategy τ such that every play starting in v and consistent with τ either violates Win or eventually visits some vertex v with Towards a contradiction, assume Thus, v is updated in r j−1 to some value strictly less than j 2 , which contradicts the induction hypothesis for j − 1. Hence, we indeed obtain Thus, there are two types of vertices v in U : those for which r j−3 (v ) is defined, which implies r j−3 (v ) = r j−1 (v ) due to the induction hypothesis and refinement, and those where r j−3 (v ) is undefined, which implies r j−2 (v ) = r j−1 (v ) due to the claim above.
We claim that Player 1 wins . This contradicts v being updated in r j , our initial assumption.
To this end, we construct a strategy τ from v that either violates Win or reaches a vertex v with Thus, every play that starts in v and is consistent with τ either violates Win (as it has a suffix that does) or reaches a vertex v with r j−3 (v ) ≤ r j (v), which proves our claim.
Lemma 4 implies that an algorithm computing the r j does not need to implement the definition of the two updates as presented above, but can be optimized by taking into account that a rank is never updated once set. However, for the proofs below, the definition presented above is more expedient, as it gives stronger preconditions to rely on, e.g., Lemmas 2 and 3 only hold for the definition presented above.
Also, from the proof of Lemma 4, we obtain an upper bound on the maximal rank of r * . This in turn implies that the r j stabilize quickly, as r j = r j+1 = r j+2 implies r j = r * .
The main result of this section shows that r * characterizes the resilience of vertices of finite resilience.
Lemma 5 Let r * be defined for G as above, and let v ∈ V .
we just have to show that Player 0 has no (r * (v) + 1)-resilient strategy from v. By definition, for every strategy σ for Player 0, we have to show that there is a play ρ starting in v and consistent with σ that has at most r * (v) disturbances and is winning for Player 1. So, fix an arbitrary strategy σ .
We define a play with the desired properties by constructing longer and longer finite prefixes before finally appending an infinite suffix. During the construction, we ensure that each such prefix ends in dom(r * ) in order to be able to proceed with our construction.
The first prefix just contains the starting vertex (v, 0), i.e., the prefix does indeed end in dom(r * ). Now, assume we have produced a prefix w(v , b ) ending in some vertex v ∈ dom(r * ), which implies that j v is defined. We consider three cases: by definition of r 0 , i.e., Player 1 has a winning strategy τ from v. Thus, we extend w(v , b ) by the unique disturbance-free play that starts in v and is consistent with σ and τ , without its first vertex. In that case, the construction of the infinite play is complete.
Second, if j v > 0 is odd, then v received its rank r * (v ) during a disturbance update.
In this case, we extend w (v , b ) by such a vertex v to obtain the new prefix w(v , b )(v , 1), which satisfies the invariant, as v is in dom(r * ). Further, we have j v < j v as the rank of v had to be defined in order to be considered during the disturbance update assigning a rank to v .
Finally, if j v > 0 is even, then v received its rank r * (v ) during a risk update. We claim that Player 1 has a strategy τ v that guarantees one of the following outcomes from v : either the resulting play violates Win or it encounters a vertex v that satisfies r * (v ) ≤ r * (v ) and In that case, consider the unique disturbance-free play ρ that starts in v and is consistent with σ and the strategy τ v as above. If ρ violates Win, then we extend w(v , b ) by ρ without its first vertex. In that case, the construction of the infinite play is complete.
If ρ does not violate Win, then we extend w(v , b ) by the prefix of ρ without its first vertex and up to (and including) the first occurrence of a vertex v in ρ satisfying the properties described above. Note that this again satisfies the invariant.
It remains to argue our claim: v was assigned its rank r * (v ) = r j v (v ) because it is in Player 1's winning region in the game with winning condition Win ∩ Safety(U ), for Hence, from v , Player 1 has a strategy to either violate the winning condition or to reach U .
. Finally, we have j v < j v , as the rank of v is assigned due to vertices in U already having ranks.
Note that only in two cases, we extend the prefix to an infinite play. In the other two cases, we just extend the prefix to a longer finite one. Thus, we first show that this construction always results in an infinite play. To this end, let w 0 (v 0 , b 0 ) and w 1 (v 1 , b 1 ) be two of the prefixes constructed above such that w 1 (v 1 , b 1 ) is an extension of w 0 (v 0 , b 0 ). A simple induction proves j v 1 < j v 0 . Hence, as the value can only decrease finitely often, at some point an infinite suffix is added. Thus, we indeed construct an infinite play.
Finally, we have to show that the resulting play has the desired properties: by construction, the play starts in v and is consistent with σ . Furthermore, by construction, it has a disturbancefree suffix that violates Win. Thus, by prefix-independence, the whole play also violates Win. It remains to show that it has at most r * (v) disturbances. To this end, let w 0 (v 0 , b 0 ) and The only other possibility is the extension consisting of a finite play prefix that is consistent with the strategy τ v 0 . Then, by construction, we obtain r * (v 1 ) ≤ r * (v 0 ). So, there are at most r * (v) many disturbances in the play, as the current rank decreases with every disturbance edge and does not increase with the other type of extension, but is always non-negative.
"r G (v) ≥ r * (v)": Here, we construct a strategy σ f for Player 0 that is r * (v)-resilient from every v ∈ dom(r * ), i.e., from v, σ f has to be winning even under r * (v) − 1 disturbances. As every strategy is 0-resilient, we only have to consider those v with r * (v) > 0.
The proof is based on the fact that r * is both stable under the disturbance and under the risk update, i.e., the disturbance update and the risk update of r * are r * , which yields the following properties. Let (v, v ) ∈ D be a disturbance edge such that r * (v) > 0. Then, we have r * (v ) ≥ r * (v)−1. Also, for every v ∈ dom(r * ) with r * (v) > 0, Player 0 has a winning strategy σ v from v for the game G v = (A, Win ∩ Safety({v ∈ dom(r * ) | r * (v ) < r * (v)})) (note the strict inequality). Here, we apply determinacy of G v , as the risk update is formulated in terms of Player 1's winning region. Now, we define σ f to always mimic a strategy σ v cur for some v cur ∈ dom(r * ), which is initialized by the starting vertex. The strategy σ v cur is mimicked until a consequential (w.r.t. σ v cur ) disturbance edge is taken, say by reaching v . In that case, the strategy σ f discards the history of the play constructed so far, updates v cur to v , and begins mimicking σ v . This is repeated ad infinitum. Now, consider a play that starts in dom(r * ), is consistent with σ f , and has less than r * (v) disturbances. The part up to the first consequential disturbance edge (if it exists at all) is consistent with σ v . Now, let (v 0 , v 0 ) be the corresponding disturbance edge. Then, we have r * (v 0 ) ≥ r * (v), as σ v being a winning strategy for the safety condition never visits vertices with a rank smaller than r * (v). Thus, we conclude r Similarly, the part between the first and the second consequential disturbance edge (if it exists at all) is consistent with Continuing this reasoning shows that less than r * (v) (consequential) disturbance edges lead to a vertex v with r * (v ) > 0, as the rank is decreased by at most one for every disturbance edge. The suffix starting in this vertex is disturbance-free and consistent with σ v . Hence, the suffix satisfies Win, i.e., by prefixindependence, the whole play satisfies Win as well. Thus, σ f is indeed r * (v)-resilient from every v ∈ dom(r * ).
(2) Let X = V \dom(r * ). The disturbance update of r * being r * implies that every disturbance edge starting in X leads back to X . Similarly, the risk update of r * being r * implies X = W 0 (G X ) for G X = (A, Win ∩ Safety(V \X )). Thus, from every v ∈ X , Player 0 has a strategy σ v such that every disturbance-free play that starts in v and is consistent with σ v satisfies the winning condition Win and never leaves X . Using these properties, we construct a strategy σ ω that is ω-resilient from each v ∈ X . Thus, r G (v) ∈ {ω, ω + 1}.
The definition of the strategy σ ω here is similar to the one above yielding the lower bound on the resilience. Again, σ ω always mimics a strategy σ v cur for some v cur ∈ X , which is initialized by the starting vertex. The strategy σ v cur is mimicked until a consequential (w.r.t. σ v cur ) disturbance edge is taken, say by reaching the vertex v . In that case, the strategy σ ω discards the history of the play constructed so far, updates v cur to v , and begins mimicking σ v . This is repeated ad infinitum.
Due to the properties of the disturbance edges and the strategies σ v , such a play never leaves X , even if disturbances occur. Furthermore, if only finitely many disturbances occur, then the resulting play has a disturbance-free suffix that starts in some v ∈ X and is consistent with σ v . As σ v is winning from v in G X , this suffix satisfies Win. Hence, by prefix-independence of Win, the whole play also satisfies Win. Thus, σ ω is indeed an ω-resilient strategy from every v ∈ X .
Combining Corollary 1 and Lemma 5, we obtain an upper bound on the resilience of vertices with finite resilience.

Characterizing vertices of resilience ! + 1
Our goal in this section is to determine the vertices of resilience ω + 1, i.e., those from which Player 0 can win even under an infinite number of disturbances. Intuitively, in this setting, we give Player 1 control over the disturbance edges, as he cannot execute more than infinitely many disturbances during a play.
In the following, we prove this intuition to be correct. To this end, we transform the arena of the game so that at a vertex of Player 0, first Player 1 gets to chose whether he wants to take one of the disturbance edges and, if not, gives control to Player 0, who is then able to use a standard edge.
Given  Figure 2 illustrates the construction of a rigged game for the example game of Fig. 1 (note that the rigged game is also a parity game in this example). And indeed, the winning region of Player 0 corresponds to the vertices of resilience ω + 1 in the game of Fig. 1.
The following lemma formalizes the observation that W 0 (G rig ) characterizes the vertices of resilience ω + 1 in G. Note that we have no assumptions on G here.

Lemma 6 Let v be a vertex of the game
Proof The proof consists of constructing mappings between play prefixes and plays in both games, which are then used to transfer strategies between the games. This is conceptually straightforward, but technical due to the presence of the bits indicating whether a disturbance occurred or not. These have to be reconstructed to obtain proper mappings.
"⇒": Let Player 0 win G rig from v, say with winning strategy σ . We inductively translate play prefixes w in G into play prefixes t (w) in G rig that satisfy the following invariant: t ((v 0 , b 0 ) · · · (v j , b j )) starts in v 0 and ends in v j .
For the induction start, we define t (v 0 , b 0 ) = (v 0 , 0); to define we consider several cases: i.e., the play traverses the disturbance edge (v j , v j+1 ). This move is mimicked by defining and v j ∈ V 0 , then the play did not traverse a disturbance edge and instead allowed Player 0 to pick a standard edge (v j , v j+1 ) to traverse. This move is mimicked by defining and v j ∈ V 1 , then the play traversed the standard edge (v j , v j+1 ). This move is mimicked by defining Note that our invariant is satisfied in any case. Also, we lift t to infinite plays by taking limits as usual.
"⇐": Now, let r G (v) = ω + 1, i.e., Player 0 has an (ω + 1)-resilient strategy σ from v in G. This time, we inductively define a translation t of play prefixes in G rig into play prefixes in G. Here, it suffices to consider those prefixes that start and end in V 1 . For these, we satisfy the following invariant: if w starts in v 0 and ends in v j , then t(w) starts in v 0 and ends in v j as well. Note that G rig has no disturbance edges. Hence, the bits indicating whether such an edge has been traversed are always zero in plays of G rig . Thus, we define t(v 0 , 0) = (v 0 , 0) and consider several cases for the inductive step: -First, assume we have a prefix of the form i.e., Player 1's move simulates the disturbance edge (v j , v j+1 ) ∈ D. Then, we define -Next, assume we have a prefix of the form (v 0 , 0) · · · (v j , 0)(v j+1 , 0) for some v j ∈ V 1 , i.e., Player 1's move simulates the standard edge (v j , v j+1 ) ∈ E. Then, we define -Finally, the last case is a prefix of the form (v 0 , 0) · · · (v j , 0)(v j , 0)(v j+1 , 0) for some v j ∈ V 0 , i.e., Player 0's move simulates the standard edge (v j , v j+1 ) ∈ E. Then, we define The invariant is satisfied in any case. Also, we can again lift t to infinite plays via limits. Now, let d be the homomorphism induced by mapping (v, b) ∈ V ×{0, 1} to v ∈ V , i.e., d again deletes the bits indicating the occurrence of disturbances. Then, we define a strategy σ for Player 0 in G rig via 0)))).
A straightforward induction shows that for every play ρ that is consistent with σ , the play t(ρ) is consistent with σ . Hence, if ρ starts in v, then t(ρ) satisfies the winning condition, as σ Thus, σ is a winning strategy for Player 0 from v.
With an adaption of the rigged game, one can also directly characterize the vertices with resilience ω. However, since our algorithm and the rigged game already provide an indirect characterization, we do not present this construction here.
Furthermore, the proof of Lemma 6 also yields the preservation of positional and finitestate strategies. To this end, consider the first implication proved above. If σ is positional (finite-state), then σ is positional (finite-state) as well. Thus, applying both implications yields the following corollary.

Corollary 3 Let G and G rig be defined as above and v a vertex of G.
1. Assume Player 0 has a positional winning strategy for G rig from v. Then, Player 0 has an (ω + 1)-resilient positional strategy for G from v. 2. Assume Player 0 has a finite-state winning strategy for G rig from v. Then, Player 0 has an (ω + 1)-resilient finite-state strategy (of the same size) for G from v.

Computing optimally resilient strategies
This section is concerned with computing the resilience and optimally resilient strategies.
Here, we focus on positional and finite-state strategies, which are sufficient for the majority of winning conditions in the literature. Nevertheless, it is easy to see that our framework is also applicable to infinite-state strategies.
In the proof of Lemma 5, we construct strategies σ f and σ ω such that σ f is r G (v)-resilient from every v with r G (v) ∈ ω and such that σ ω is ω-resilient from every v with r G (v) ≥ ω. Both strategies are obtained by combining winning strategies for some game (A, Win∩Safety(U )). However, even if these winning strategies are positional, the strategies σ f and σ ω are in general not positional. Nonetheless, we show in the proof of Theorem 1 that such positional winning strategies and a positional one for G rig can be combined into a single positional optimally resilient strategy.
Recall the requirements from Sect. 3.1 for a game (A, Win): Win is prefix-independent and the game G U is determined for every U ⊆ V , where we write G U for the game (A, Win ∩ Safety(U )) for some U ⊆ V . To prove the results of this section, we need to impose some additional effectiveness requirements: we require that each game G U and the rigged game G rig can be effectively solved. Also, we first assume that Player 0 has positional winning strategies for each of these games, which have to be effectively computable as well. We discuss the severity of these requirements in Sect. 4.

Theorem 1 Let G satisfy all the above requirements. Then, the resilience of G's vertices and a positional optimally resilient strategy can be effectively computed.
To prove this result, we refine the following standard technique that combines positional winning strategies for games with prefix-independent winning conditions. Assume we have a positional strategy σ v for every vertex v in some set W ⊆ V such that σ v is winning from v. Furthermore, let R v be the set of vertices visited by plays that start in v and are consistent with σ v . Also, let m(v) = min ≺ {v ∈ V | v ∈ R v } for some strict total ordering ≺ of W . Then, the positional strategy σ defined by σ (v) = σ m(v) (v) is winning from each v ∈ W , as along every play that starts in some v ∈ W and is consistent with σ , the value of the function m cannot increase. Thus, after it has stabilized, the remaining suffix is consistent with some strategy σ v . Hence, the suffix is winning for Player 0 and prefix-independence implies that the whole play is winning for her as well.
Here, we have to adapt this reasoning to respect the resilience of the vertices and to handle disturbance edges. Also, we have to pay attention to vertices of resilience ω + 1, as plays starting in such vertices have to be winning under infinitely many disturbances.

Proof of Theorem 1
The effective computability of the resilience follows from the effectiveness requirements on G: to compute the ranking r * , it suffices to compute the disturbance and risk updates. The former are trivially effective while the effectiveness of the latter ones follows from our assumption. Lemma 5 shows that r * correctly determines the resilience of all vertices with finite resilience. Finally, by solving the rigged game, we also determine the resilience of the remaining vertices (Lemma 6). Again, this game can be solved due to our assumption.
Thus, it remains to show how to compute a positional optimally resilient strategy. To this end, we compute a positional strategy σ v for every v satisfying the following: We have shown the existence of such a strategy in the proof of Item 1 of Lemma 5. -For every v ∈ V with r G (v) = ω, the strategy σ v is winning for Player 0 from v for the game (A, Win ∩ Safety({v ∈ V | r G (v ) ∈ ω})). We have shown the existence of such a strategy in the proof of Item 2 of Lemma 5. -For every v ∈ V with r G (v) = ω + 1, the strategy σ v is (ω + 1)-resilient from v. The existence of such a strategy follows from Item 1 of Corollary 3, as we assume Player 0 to win G rig with positional strategies. -For every v ∈ V with r G (v) = 0, we fix an arbitrary positional strategy σ v for Player 0.

Furthermore, we fix a strict linear order
i.e., we order the vertices by ascending resilience. For v ∈ V with r G (v) = ω+1, let R v be the set of vertices reachable via disturbance-free plays that start in v and are consistent with σ v . On the other hand, for v ∈ V with r G (v) = ω + 1, let R v be the set of vertices reachable via plays with arbitrarily many disturbances that start in v and are consistent with σ v . We For v with r G (v) = ω + 1 this follows immediately from the choice of σ v . Thus, let v with r G (v) = ω + 1. Assume σ v reaches a vertex v of resilience r G (v ) = ω + 1. Then, there exists a play ρ starting in v that is consistent with σ v , has less than ω + 1 many disturbances and is losing for Player 0. Thus, the play obtained by first taking the play prefix to v and then appending ρ without its first vertex yields a play starting in v, consistent with σ v , but losing for Player 0. This play witnesses that σ v is not (ω + 1)-resilient from v, which contradicts our assumption and thus concludes the proof of the claim for the case r G (v) = ω + 1.
Let m : V → V be given as m(v) = min ≺ {v ∈ V | v ∈ R v } and define the positional strategy σ as σ (v) = σ m(v) (v). By our assumptions, σ can be effectively computed. It remains to show that it is optimally resilient.
To this end, we apply the following two properties of edges (v, v ) that may appear during a play that is consistent with σ , i.e., we either have v ∈ V 0 and σ The first property follows from minimality of m(v) and ( * ) while the second follows from the definition of R v . 2. If (v, v ) ∈ D, then we distinguish several subcases, which all follow immediately from the definition of resilience: here, the second property follows from the definition of R v for v with r G (v) = ω + 1, which takes disturbance edges into account).

Now, consider a play
If r G (v 0 ) = 0 then we have nothing to show, as every strategy is 0-resilient from v. Now, assume r G (v 0 ) ∈ ω\{0}. We have to show that if ρ has less than r G (v 0 ) disturbances, then it is winning for Player 0. An inductive application of the above properties shows that in that case the last disturbance edge leads to a vertex of non-zero resilience. Furthermore, as the values m(v j ) are only decreasing afterwards, they have to stabilize at some later point. Hence, there is some suffix of ρ that starts in some v with non-zero resilience and that is consistent with the strategy σ v . Thus, the suffix is winning for Player 0 by the choice of σ v and prefix-independence implies that ρ is winning for her as well.
Next, assume r G (v 0 ) = ω. We have to show that if ρ has a finite number of disturbances, then it is winning for Player 0. Again, an inductive application of the above properties shows that in that case the last disturbance edge leads to a vertex of resilience ω or ω+1. Afterwards, the values m(v j ) stabilize again. Hence, there is some suffix of ρ that starts in some v with non-zero resilience and that is consistent with the strategy σ v . Thus, the suffix is winning for Player 0 by the choice of σ v and prefix-independence implies that ρ is winning for her as well.
Finally, assume r G (v 0 ) = ω + 1. Then, the above properties imply that ρ only visits vertices with resilience ω + 1 and that the values m(v j ) eventually stabilize. Hence, there is a suffix of ρ that is consistent with some (ω + 1)-resilient strategy σ v , where v is the first vertex of the suffix. Hence, the suffix is winning for Player 0, no matter how many disturbances occur. This again implies that ρ is winning for her as well.
The algorithm determining the vertices' resilience and a positional optimally resilient strategy first computes r * and the winner of the rigged game. This yields the resilience of G's vertices. Furthermore, the strategy is obtained by combining winning strategies for the games G U and for the rigged game as explained above.
Next, we analyze the complexity of the algorithm sketched above in some more detail. The inductive definition of the r j can be turned into an algorithm computing r * (using the results of Lemma 4 to optimize the naive implementation), which has to solve O(|V |) many games (and compute winning strategies for some of them) with winning condition Win ∩Safety(U ). Furthermore, the rigged game, which is of size O(|V |), has to be solved and winning strategies have to be determined. Thus, the overall complexity is in general dominated by the complexity of solving these tasks.
We explicitly state one complexity result for the important case of parity games, using the fact that each of these games is then a parity game as well. Also, we use a quasipolynomial time algorithm for solving parity games [8,15,20,22] to solve the games G U and G rig .

Theorem 2 Optimally resilient strategies in parity games are positional and can be computed in quasipolynomial time.
Using similar arguments, one can also analyze games where positional strategies do not suffice. As above, assume G satisfies the same assumptions on determinacy and effectiveness, but only require that Player 0 has finite-state winning strategies for each game with winning condition (A, Win∩Safety(U )) and for the rigged game G rig . Then, one can show that she has a finite-state optimally resilient strategy. In fact, by reusing memory states, one can construct an optimally resilient strategy that it is not larger than any constituent strategy.

Discussion
In this section, we discuss the assumptions required to be able to compute positional (finitestate) optimally resilient strategies with the algorithm presented in Sect. 3. Here, we only consider the case of positional strategies. The case of finite-state strategies is analogous.
To this end, fix a game G = (A, Win) with vertex set V and recall that G rig is the corresponding rigged game and that we defined G U = (A, Win ∩ Safety(U )) for U ⊆ V . Now, the assumptions on G that need to be satisfied for Theorem 1 to hold are as follows: 1. The game G U is determined for every U ⊆ V . 2. Player 0 has a positional winning strategy from every vertex in her winning regions in the G U and in the game G rig . 3. Each G U and the game G rig can be effectively solved and positional winning strategies can be effectively computed for each such game. 4. Win is prefix-independent.
First, consider the determinacy assumption. For W ⊆ V let A\W denote the arena obtained from A by removing all vertices from W , as well as all edges from or to vertices in W . It is easy to show that A\W has no terminal vertices, if W is the winning region of Player 1 in a safety game played in A. Now, it is straightforward to show Safety(U )). Thus, one can remove the winning region of Player 1 in the safety game and then consider the subgame of G played in Player 0's winning region of the safety game. Thus, all subgames of G being determined suffices for the determinacy requirement being satisfied. The winning conditions one typically studies, e.g., parity and in fact all Borel ones [24], satisfy this property.
The next requirement concerns the existence of positional winning strategies for the games G U and G rig . For the G U , this requirement is satisfied if Player 0 has positional winning strategies for all subgames of G, as argued above. As every positional optimally resilient strategy is also a winning strategy in a certain subgame, this condition is necessary. Now, consider G rig , whose winning condition can be written as h −1 (Win) for the homomorphism h  Fig. 4 Counterexample for requiring only the implication from left to right from the definition of prefixindependence for the computation of resilience from Sect. 3.2. The winning conditions one typically studies, e.g., the Borel ones, are closed w.r.t. such supersequences. If G is from a class of winning conditions that allows for positional winning strategies for Player 0, then this class typically also contains G rig . Also, the assumption on the effective solvability and computability of positional strategies is obviously necessary, as we solve a more general problem when determining optimally resilient strategies.
Finally, let us consider prefix-independence. If the winning condition is not prefixindependent, then the algorithm presented in Sect. 3 does not compute the resilience of vertices correctly. In fact, recall that a winning condition Win is prefix-independent if, for all plays ρ and all play prefixes w, we have ρ ∈ Win if and only if wρ ∈ Win. We show that neither implication of this equivalence suffices on its own for the algorithm from Sect. 3 to compute the correct resilience of vertices.
First, consider the family G k = (A, Win k ) of games shown in Fig. 3. In G k , it is the goal of Player 0 to avoid more than k visits to v. Hence, for all plays ρ and all play prefixes w we have that wρ ∈ Win implies ρ ∈ Win.
In each of the G k , a visit to v only occurs via a disturbance or if the initial vertex is v. Hence, we have r G k (v) = k and r G k (v ) = k + 1. If we apply the algorithm from Sect. 3, however, the initial ranking function r 0 has an empty domain, since we have W 1 (G k ) = ∅. Thus, the computation of the r j immediately stabilizes, yielding r * with empty domain. Hence, that algorithm does, in general, not compute the correct resilience if only the implication from right to left from the definition of prefix-independence is satisfied.
Conversely, consider the game G shown in Fig. 4. The winning condition of this game satisfies that, for all play prefixes w and all plays ρ, we have that ρ ∈ Win implies wρ ∈ Win. If we apply the algorithm from Sect. 3, however, the initial ranking r 0 has the domain {v } with r 0 (v ) = 0, due to W 1 (G) = {v }. The disturbance update of r 0 then yields the ranking r 1 with r 1 (v) = 1 due to the single disturbance edge of G and with r 1 (v ) = 0. At this point, the rankings stabilize and we obtain r * = r 1 .
While we indeed have r G (v ) = 0 = r * (v ), we furthermore have r G (v) = ω + 1 = r * (v), as every play starting in vertex v satisfies the winning condition. Hence, this example showcases that the implication from left to right from the definition of prefix-independence also does not suffice for the algorithm from Sect. 3 to correctly compute the resilience. Thus, we indeed require full prefix-independence of the winning condition as a precondition for the correctness of that algorithm.
In the following section, we show that one can still leverage our algorithm from Sect. 3 in order to compute the resilience of a wide range of games with prefix-dependent winning conditions. To this end, we extend the framework of game reductions to games with dis-turbances, in such a way that the existence of α-resilient strategies is preserved. Using this framework shows that Player 0 has a finite-state optimally resilient strategy in every game with ω-regular winning condition.

Prefix-dependent winning conditions
We begin by introducing some notation regarding game reductions. An arena A = (V , V 0 , V 1 , E, D) and a memory structure M = (M, Init, Upd) for A induce the expanded , m), (v , m )) ∈ E if and only if (v, v ) ∈ E and Upd(m, v ) = m . The disturbance edges D are defined analogously, i.e., ((v, m), Play prefixes are translated analogously.

Remark 3 Let ρ be a play in
and every play ρ in G is won by the same player that wins ext(ρ) in G .
Proof We show that Player 0 has an α-resilient strategy σ for G from (v, Init(v)) if and only if she has an α-resilient strategy σ for G from v, which implies our claim. The translation of the strategies is the same as in the disturbance-free setting (see, e.g., [21]), but here we have to argue about resilience instead of just winning. "⇐": Given a strategy σ for G, we define σ for G via which is consistent with σ . Hence, ρ and ρ have the same winner and the same number of disturbances. Hence, if σ is α-resilient from a vertex v, then σ is α-resilient from (v, Init(v)).
"⇒": Given a strategy σ for G , we define σ A straightforward induction shows that a play in G is consistent with σ if and only if its extended play in G is consistent with σ . Thus, these plays have the same winner and the same number of disturbances. Thus, again, if σ is α-resilient from a vertex (v, Init(v)) then σ is α-resilient from v.
As usual for game reductions, we obtain a finite-state strategy for G when starting with a positional strategy in G . To this end, consider the proof of the second implication above. If σ is positional, then the strategy σ is implemented by M and the next-move function Nxt given by Nxt(v, m) A similar construction works in case σ is finite-state, say implemented by M . Then, σ is implemented by the product of M and M , which is defined as expected (we refer to, e.g., [21] for a formal definition). Altogether, we obtain the following result.

Corollary 4 Let
1. If Player 0 has an α-resilient positional strategy from (v, Init(v)) in G , then she has an α-resilient finite-state strategy from v in G, which is implemented by M. 2. If Player 0 has an α-resilient finite-state strategy from (v, Init(v)) in G , say implemented by M , then she has an α-resilient finite-state strategy from v in G, which is implemented by the product of M and M . Now, we can formulate the main theorem of this section, which shows that prefixdependence is not a restriction, as long as the game is reducible to a prefix-independent one. Note that this is in particular true for every ω-regular winning condition (see, e.g., [18]): every such condition is recognized by a deterministic parity automaton, which can be turned into a memory structure which allows to reduce the original game to a parity game.
Theorem 3 Let G ≤ M G so that G has a prefix-independent winning condition, can be effectively computed from G, and satisfies the assumptions from Sect. 3

.3 (with finite-state strategies).
Then, the resilience of G's vertices and a finite-state optimally resilient strategy can be effectively computed.
Proof This is a direct consequence of Lemma 7 and Theorem 1. To obtain an optimally resilient strategy, we apply Corollary 4 for finite-state strategies.
Recall the family of games shown in Fig. 3 in which Player 0 aims to prevent more than k visits to the vertex v 1 for some parameter k ∈ ω. Such a game can be reduced to a parity game using a memory structure implementing a counter up to k +1. Such a memory structure has k + 1 memory states, and a straightforward pumping argument shows that there is no smaller memory structure.
Thus, we obtain an optimally resilient strategy for Player 0 that is implemented by a memory structure with k + 1 states. While this strategy is indeed optimally resilient, it is not of minimal size: in fact, the unique strategy for Player 0 in G k is positional and optimally resilient. Thus, the approach of computing optimally resilient strategies for games with prefixdependent winning conditions via reductions to prefix-independent winning conditions is not optimal in that sense, as it may yield unnecessarily large optimally resilient strategies. In current research, we study how to synthesize minimal optimally resilient strategies for games with prefix-dependent winning conditions. Moreover, in the case of prefix-dependent winning conditions, the question arises whether or not optimally resilient strategies may be necessarily larger than winning ones. It is easy to construct a game in which Player 0 has a positional winning strategy, but an optimally resilient one requires an infinite amount of memory. One example is a game with a dedicated vertex v with a self-loop, such that using the self-loop ad infinitum is winning for Player 0. Furthermore, there is a disturbance edge leading from v into a disturbance-free subgame in which Player 0 needs an infinite amount of memory to win.
However, this example is not very useful, as Player 0 needs infinite memory to win the game from some vertex of her winning region. A more interesting question for further research is whether a result similar to Theorem 1 holds true for prefix-dependent games with positional winning strategies, e.g., weak parity games [9] or bounded parity games [11]. However, for both of these conditions, monotonicity arguments allow to transform finite-state optimally resilient strategies into positional ones (similar to the construction in [16,Section 5]). However, these arguments rely on monotonicity properties of the parity condition and are Fig. 5 Intuitively, moving from v 1 to v 2 is preferable for Player 0, as it allows her to possibly "recover" from a first disturbance with the "help" of a second one therefore unlikely to be generalizable. On the other hand, we are not aware of an example of a class of winning conditions that always allow for positional winning strategies for Player 0, but require memory to implement optimally resilient strategies. In future work, we investigate whether the blowup introduced by the reduction can be avoided.

Outlook
We have developed a fine-grained view on the quality of strategies: instead of evaluating whether or not a strategy is winning, we compute its resilience against intermittent disturbances. While this measure of quality allows constructing "better" strategies than the distinction between winning and losing strategies, there remain aspects of optimality that are not captured in our notion of resilience. In this section we discuss these aspects and give examples of games in which there are crucial differences between optimally resilient strategies. In further research, we aim to synthesize optimal strategies with respect to these criteria. As a first example, consider the parity game shown in Fig. 5. Vertices v 0 and v 3 have resilience 1 and ω + 1, respectively, while vertices v 1 , v 2 , and v 2 have resilience 0. Player 0's only choice consists of moving to v 2 or to v 2 from v 1 . Let σ and σ be strategies for Player 0 that always move to v 2 and v 2 from v 1 , respectively. Both strategies are optimally resilient. Hence, the algorithm from Sect. 3 may yield either one, depending on the underlying parity game solver used. Intuitively, however, σ is preferable for Player 0, as a play prefix ending in v 2 may proceed to her winning region if a single disturbance occurs. All plays encountering v 2 at some point, however, are losing for her. Hence, another interesting avenue for further research is to study how to recover from losing, i.e., how to construct strategies that leverage disturbances in order to leave Player 1's winning region. For safety games, this has been addressed by Dallal et al. [12].
The previous example shows that Player 0 can still make "meaningful" choices even if the play has moved outside her winning region. The game G shown in Fig. 6 demonstrates that she can do so as well when remaining in vertices of resilience ω. Every vertex in G has resilience ω, since every play with finitely many disturbances eventually remains in vertices of color 0. Moreover, the only choice to be made by Player 0 is whether to move to vertex v 1 or to vertex v 1 from vertex v 0 . Let σ and σ be positional strategies that implement the former and the latter choice, respectively.
First consider a scenario in which visiting an odd color models the occurrence of some undesirable event, e.g., that a request has not been answered. In this case, Player 0 should aim to prevent visits to v 3 in G, the only vertex of odd color. Hence, the strategy σ should be more desirable for her, as it requires two disturbances in direct succession in order to visit v 3 . When playing consistently with σ , however, a single disturbance suffices to visit v 3 . Fig. 6 Moving from v 0 to v 1 allows Player 0 to minimize visits to odd colors, while moving to v 1 allows her to minimize the occurrence of disturbances Fig. 7 Additional memory allows Player 0 to remain in v 1 longer and longer, thus decreasing the potential for disturbances On the other hand, consider a setting in which Player 0's goal is to avoid the occurrence of disturbances. In that case, σ is preferable over σ , as it allows for fewer situations in which disturbances may occur, since no disturbances are possible from vertices v 2 and v 3 .
Note that the goals of minimizing visits to vertices of odd color and minimizing the occurrence of disturbances are not contradictory: if both events are undesirable, it may be optimal for Player 0 to combine the strategies σ and σ . In general, it is interesting to study how to how to best brace for a finite number of disturbances.
Recall that, due to Theorem 2, optimally resilient strategies for parity games do not require memory. In contrast, the game shown in Fig. 7 demonstrates that additional memory can serve to further improve such strategies. Any strategy for Player 0 that does not stay in v 1 from some point onwards is optimally resilient. However, every visit to v 2 risks a disturbance occurring, which would lead the play into a losing sink for Player 0. Hence, it is in her best interest to remain in vertex v 1 for as often as possible, thus minimizing the possibility for disturbances to occur. This behavior does, however, require memory to implement, as Player 0 needs to count the visits to v 1 in order to not remain in that state ad infinitum. Even worse, for each optimally resilient strategy σ with finite memory there exists another optimally resilient strategy that uses more memory, but visits v 2 more rarely than σ , reducing the possibilities for disturbances to occur. Hence, it is interesting to study how to balance avoiding disturbances with satisfying the winning condition. This is particularly interesting if there is some cost assigned to disturbances.
In quantitative games, there is a further tradeoff between resilience and the semantic quality of strategies. As a simple example, consider the parity game in Fig. 8 and assume, for the sake of argument, that Player 0 aims to maximize the maximal color seen infinitely often. Thus, when it comes to semantic quality of strategies, Player 0 prefers moving to v 2 over moving to v 0 , when starting in v 1 . However, v 2 has resilience one while v 0 has resilience two. Hence, in this aspect, Player 0 prefers moving to v 0 over moving to v 2 . In general, it is an interesting question to determine the tradeoff between resilience and semantic quality and to compute strategies that optimize both aspects, if possible.
Finally, another important and interesting aspect, which falls outside the scope of this paper, is to provide general guidelines and best practices on how to model synthesis problems by games with disturbances. We will address these problems in future research.

Related work
The notion of unmodeled intermittent disturbances in infinite games has recently been formulated by Dallal, Neider, and Tabuada [12]. In that work, the authors also present an algorithm for computing optimally resilient strategies for safety games with disturbances, which is an extension of the classical attractor computation [18]. Due to the relatively simple nature of such games, however, this algorithm cannot easily be extended to handle more expressive winning conditions, and the approach presented in this work relies on fundamentally different ideas.
Resilience is not a novel concept in the context of reactive systems synthesis. It appears, for instance, in the work by Topcu et al. [28] as well as Ehlers and Topcu [14]. A notion of resilience that is very similar to the one considered here has been proposed by Huang et al. [19], where the game graph is augmented with so-called "error edges". However, this setting differs from the one studied in this work in various aspects. Firstly, Huang et al. work in the framework of concurrent games and model errors as being under the control of Player 1. This contrasts to the setting considered here, in which the players play in alternation and disturbances are seen as rare events rather than antagonistic to Player 0. Secondly, Huang et al. restrict themselves to safety games, whereas we consider a much broader class of infinite games. Finally, Huang et al. compute resilient strategies with respect to a fixed parameter k, thus requiring to repeat the computation for various values of k to find optimally resilient strategies. In contrast, our approach computes an optimal strategy in a single run. Hence, they consider a more general model of interaction, but only a simple winning condition, while the notion of disturbances considered here is incomparable to theirs.
For instance, Brihaye et al. [7] consider quantitative games under failures, which are a generalization of sabotage games [29]. The main difference to our setting is that Brihaye et al. consider failures-embodied by a saboteur player-as antagonistic, whereas we consider disturbances as non-antagonistic events. Moreover, solving a parity game while maintaining a cost associated with the sabotage semantics below a given threshold is Exp-Time-complete, whereas our approach computes optimally resilient controllers for parity conditions in quasipolynomial time.
Besides fault tolerance, robustness in the area of reactive controller synthesis has also attracted considerable interest in the recent years, typically in settings with specifications of the form ϕ ⇒ ψ stating that the controller needs to fulfill the guarantee ψ if the environment satisfies the assumption ϕ. A prominent example of such work is that of Bloem et al. [3], in which the authors understand robustness as the property that "if assumptions are violated temporarily, the system is required to recover to normal operation with as few errors as possible" and consider the synthesis of robust controllers for the GR(1) fragment of Linear Temporal Logic [6]. Other examples include quantitative synthesis [4], where robustness is defined in terms of payoffs, and the synthesis of robust controllers for cyber-physical systems [23,26]. For a more in-depth discussion of related notions of resilience and robustness in reactive synthesis, we refer the interested reader to Dallal, Neider, and Tabuada's section on related work [12, Section I]. Moreover, a survey of a large body of work dealing with robustness in reactive synthesis has been presented by Bloem et al. [5].
Finally, note that for the special case of parity games, we can also characterize vertices of finite resilience (cf. Sect. 3.1) by a reduction to finding optimal strategies in energy parity games [10], which yields the same complexity as our algorithm (though such a reduction would not distinguish between vertices with resilience ω and vertices with resilience ω + 1. Also, it is unclear if and how this reduction can be extended to other winning conditions and if custom-made solutions would be required for each new class of game. By contrast, our refinement-based approach works for any class of infinite games that satisfies the mild assumptions discussed in Sect. 4.

Conclusion
We presented an algorithm for computing optimally resilient strategies in games with disturbances that is applicable to any game that satisfies some mild (and necessary) assumptions. Thereby, we have vastly generalized the work of Dallal, Neider, and Tabuada, who only considered safety games. Furthermore, we showed that optimally resilient strategies are typically of the same size as classical winning strategies. Finally, we have illustrated numerous novel phenomena that appear in the setting with disturbances but not in the classical one. Studying these phenomena is a very promising direction of future work.
As part of future work, we are currently implementing our proposed method on top of the parity game solver Oink [30] and SCOTS [25], a tool for the synthesis of controllers in the context of dynamic and cyber-physical systems. Besides developing an end-to-end synthesis tool for controllers of dynamic and cyber-physical systems, a major part of this effort is to evaluate the impact of the polynomial overhead as compared to classical parity game solvers. Preliminary experiments with this prototype implementation suggest that the additional overhead does not impact the overall performance much.