Symbolic Algorithms for Graphs and Markov Decision Processes with Fairness Objectives

Given a model and a specification, the fundamental model-checking problem asks for algorithmic verification of whether the model satisfies the specification. We consider graphs and Markov decision processes (MDPs), which are fundamental models for reactive systems. One of the very basic specifications that arise in verification of reactive systems is the strong fairness (aka Streett) objective. Given different types of requests and corresponding grants, the objective requires that for each type, if the request event happens infinitely often, then the corresponding grant event must also happen infinitely often. All $\omega$-regular objectives can be expressed as Streett objectives and hence they are canonical in verification. To handle the state-space explosion, symbolic algorithms are required that operate on a succinct implicit representation of the system rather than explicitly accessing the system. While explicit algorithms for graphs and MDPs with Streett objectives have been widely studied, there has been no improvement of the basic symbolic algorithms. The worst-case numbers of symbolic steps required for the basic symbolic algorithms are as follows: quadratic for graphs and cubic for MDPs. In this work we present the first sub-quadratic symbolic algorithm for graphs with Streett objectives, and our algorithm is sub-quadratic even for MDPs. Based on our algorithmic insights we present an implementation of the new symbolic approach and show that it improves the existing approach on several academic benchmark examples.


Introduction
In this work we present faster symbolic algorithms for graphs and Markov decision processes (MDPs) with strong fairness objectives. For the fundamental model-checking problem, the input consists of a model and a specification, and the algorithmic verification problem is to check whether the model satisfies the specification. We first describe the specific model-checking problem we consider and then our contributions. Models: Graphs and MDPs. Two standard models for reactive systems are graphs and Markov decision processes (MDPs). Vertices of a graph represent states of a reactive system, edges represent transitions of the system, and infinite paths of the graph represent non-terminating trajectories of the reactive system. MDPs extend graphs with probabilistic transitions that represent reactive systems with uncertainty. Thus graphs and MDPs are the de-facto model of reactive systems with nondeterminism, and nondeterminism with stochastic aspects, respectively [18,3].
Specification: Strong Fairness (aka Streett) Objectives. A basic and fundamental property in the analysis of reactive systems is the strong fairness condition, which informally requires that if events are enabled infinitely often, then they must be executed infinitely often. More precisely, the strong fairness conditions (aka Streett objectives) consist of k types of requests and corresponding grants, and the objective requires that for each type if the request happens infinitely often, then the corresponding grant must also happen infinitely often. After safety, reachability, and liveness, the strong fairness condition is one of the most standard properties that arise in the analysis of reactive systems, and chapters of standard textbooks in verification are devoted to it (e.g., [18,Chapter 3.3], [31,Chapter 3], [2,Chapters 8,10]). Moreover, all ω-regular objectives can be described by Streett objectives, e.g., LTL formulas and non-deterministic ω-automata can be translated to deterministic Streett automata [33] and efficient translation has been an active research area [15,22,27]. Thus Streett objectives are a canonical class of objectives that arise in verification. Satisfaction. The basic notions of satisfaction for graphs and MDPs are as follows: For graphs the notion of satisfaction requires that there is a trajectory (infinite path) that belongs to the set of paths described by the Streett objective. For MDPs the satisfaction requires that there is a policy to resolve the nondeterminism such that the Streett objective is ensured almost-surely (with probability 1). Thus the algorithmic model-checking problem of graphs and MDPs with Streett objectives is a core problem in verification. Explicit vs Symbolic Algorithms. The traditional algorithmic studies consider explicit algorithms that operate on the explicit representation of the system. In contrast, implicit or symbolic algorithms only use a set of predefined operations and do not explicitly access the system [19]. The significance of symbolic algorithms in verification is as follows: to combat the state-space explosion, large systems must be succinctly represented implicitly and then symbolic algorithms are scalable, whereas explicit algorithms do not scale as it is computationally too expensive to even explicitly construct the system. Relevance. In this work we study symbolic algorithms for graphs and MDPs with Streett objectives. Symbolic algorithms for the analysis of graphs and MDPs are at the heart of many state-of-the-art tools such as SPIN, NuSMV for graphs [26,17] and PRISM, LiQuor, Storm for MDPs [28,16,21]. Our contributions are related to the algorithmic complexity of graphs and MDPs with Streett objectives for symbolic algorithms. We first present previous results and then our contributions. Previous Results. The most basic algorithm for the problem for graphs is based on repeated SCC (strongly connected component) computation, and informally can be described as follows: for a given SCC, (a) if for every request type that is present in the SCC the corresponding grant type is also present in the SCC, then the SCC is identified as "good", (b) else vertices of each request type that has no corresponding grant type in the SCC are removed, and the algorithm recursively proceeds on the remaining graph. Finally, reachability to good SCCs is computed. The current bestknown symbolic algorithm for SCC computation requires O(n) symbolic steps, for graphs with n vertices [24], and moreover, the algorithm is optimal [14]. For MDPs, the SCC computation has to be replaced by MEC (maximal end-component) computation, and the current best-known symbolic algorithm for MEC computation requires O(n 2 ) symbolic steps. While there have been several explicit algorithms for graphs with Streett objectives [25,12], MEC computation [8,9,10], and MDPs with Streett objectives [7], as well as symbolic algorithms for MDPs with Büchi objectives [11], the current bestknown bounds for symbolic algorithms with Streett objectives are obtained from the basic algorithms, which are O(n · min(n, k)) for graphs and O(n 2 · min(n, k)) for MDPs, where k is the number of types of request-grant pairs.
Our Contributions. In this work our main contributions are as follows: -We present a symbolic algorithm that requires O(n · √ m log n) symbolic steps, both for graphs and MDPs, where m is the number of edges. In the case k = O(n), the previous worst-case bounds are quadratic (O(n 2 )) for graphs and cubic (O(n 3 )) for MDPs. In contrast, we present the first sub-quadratic symbolic algorithm both for graphs as well as MDPs. Moreover, in practice, since most graphs are sparse (with m = O(n)), the worst-case bounds of our symbolic algorithm in these cases are O(n · √ n log n). Another interesting contribution of our work is that we also present an O(n · √ m) symbolic steps algorithm for MEC decomposition, which is relevant for our results as well as of independent interest, as MEC decomposition is used in many other algorithmic problems related to MDPs. Our results are summarized in Table 1.
-While our main contribution is theoretical, based on the algorithmic insights we also present a new symbolic algorithm implementation for graphs and MDPs with Streett objectives. We show that the new algorithm improves (by around 30%) the basic algorithm on several academic benchmark examples from the VLTS benchmark suite [20].  Technical Contributions. The two key technical contributions of our work are as follows: -Symbolic Lock Step Search: We search for newly emerged SCCs by a local graph exploration around vertices that lost adjacent edges. In order to find small new SCCs first, all searches are conducted "in parallel", i.e., in lock-step, and the searches stop as soon as the first one finishes successfully. This approach has successfully been used to improve explicit algorithms [25,13,9,7]. Our contribution is a non-trivial symbolic variant (Section 3) which lies at the core of the theoretical improvements. -Symbolic Interleaved MEC Computation: For MDPs the identification of vertices that have to be removed can be interleaved with the computation of MECs such that in each iteration the computation of SCCs instead of MECs is sufficient to make progress [7]. We present a symbolic variant of this interleaved computation. This interleaved MEC computation is the basis for applying the lock-step search to MDPs.

Basic Problem Definitions
Markov Decision Processes (MDPs) and Graphs.
consists of a finite directed graph G = (V, E) with a set of n vertices V and a set of m edges E, a partition of the vertices into player 1 vertices V 1 and random vertices V R , and a probabilistic transition function δ. We call an edge (u, v) with u ∈ V 1 a player 1 edge and an edge we denote by Ω the set of all plays. A player 1 strategy σ : V * · V 1 → V is a function that assigns to every finite prefix ω ∈ V * · V 1 of a play that ends in a player 1 vertex v a successor vertex σ(ω) ∈ V such that (v, σ(ω)) ∈ E; we denote by Σ the set of all player 1 strategies. A strategy is memoryless if we have σ(ω) = σ(ω ) for any ω, ω ∈ V * · V 1 that end in the same vertex v ∈ V 1 .
Objectives. An objective φ is a subset of Ω said to be winning for player 1. We say that a play ω ∈ Ω satisfies the objective if ω ∈ φ. For a vertex set T ⊆ V the reachability objective is the set of infinite paths that contain a vertex of T , i.e., for ω ∈ Ω denote the set of vertices that occur infinitely often in ω. Given a set TP of k pairs the Streett objective is the set of infinite paths for which it holds for each 1 ≤ i ≤ k that whenever a vertex of L i occurs infinitely often, then a vertex of U i occurs infinitely often, i.e., Almost-Sure Winning Sets. For any measurable set of plays A ⊆ Ω we denote by Pr σ v (A) the probability that a play starting at v ∈ V belongs to A when player 1 plays strategy σ. A strategy σ is almost-sure (a.s.) winning from a vertex v ∈ V for an objective φ if Pr σ v (φ) = 1. The almost-sure winning set 1 as (P, φ) of player 1 is the set of vertices for which player 1 has an almost-sure winning strategy. In graphs the existence of an almost-sure winning strategy corresponds to the existence of a play in the objective, and the set of vertices for which player 1 has an (almost-sure) winning strategy is called the winning set 1 (P, φ) of player 1. Symbolic Encoding of MDPs. Symbolic algorithms operate on sets of vertices, which are usually described by Binary Decision Diagrams (BDDs) [29,1]. In particular Ordered Binary Decision Diagrams [6] (OBDDs) provide a canonical symbolic representation of Boolean functions. For the computation of almost-sure winning sets of MDPs it is sufficient to encode MDPs with OBDDs and one additional bit that denotes whether a vertex is in V 1 or V R . Symbolic Steps. One symbolic step corresponds to one primitive operation as supported by standard symbolic packages like CUDD [34]. In this paper we only allow the same basic set-based symbolic operations as in [32,23,5,11], namely set operations and the following one-step symbolic operations for a set of vertices Z: (a) the one-step predecessor operator i.e., the CPre R operator computes all vertices such that the successor belongs to Z with positive probability. This operator can be defined using the Pre operator and basic set operations as follows: CPre R (Z) = Pre(Z) \ (V 1 ∩ Pre(V \ Z)) . We additionally allow cardinality computation and picking an arbitrary vertex from a set as in [11]. Symbolic Model. Informally, a symbolic algorithm does not operate on explicit representation of the transition function of a graph, but instead accesses it through Pre and Post operations. For explicit algorithms, a Pre/Post operation on a set of vertices (resp., a single vertex) requires O(m) (resp., the order of indegree/outdegree of the vertex) time. In contrast, for symbolic algorithms Pre/Post operations are considered unit-cost. Thus an interesting algorithmic question is whether better algorithmic bounds can be obtained considering Pre/Post as unit operations. Moreover, the basic set operations are computationally less expensive (as they encode the relationship between the state variables) compared to the Pre/Post symbolic operations (as they encode the transitions and thus the relationship between the present and the next-state variables). In all presented algorithms, the number of set operations is asymptotically at most the number of Pre/Post operations. Hence in the sequel we focus on the number of Pre/Post operations of algorithms. Algorithmic Problem. Given an MDP P (resp. a graph G) and a set of Streett pairs TP, the problem we consider asks for a symbolic algorithm to compute the almost-sure winning set 1 as (P, Streett(TP)) (resp. the winning set 1 (G, Streett(TP))), which is also called the qualitative analysis of MDPs (resp. graphs).

Basic Concepts related to Algorithmic Solution
Reachability. For a graph G = (V, E) and a set of vertices S ⊆ V the set GRAPHREACH(G, S) is the set of vertices of V that can reach a vertex of S within G, and it can be identified with at most |GRAPHREACH(G, S) \ S| + 1 many Pre operations. Strongly Connected Components. For a set of vertices S ⊆ V we denote by G[S] = (S, E ∩ (S × S)) the subgraph of the graph G induced by the vertices of S. An induced subgraph G[S] is strongly connected if there exists a path in G[S] between every pair of vertices of S. A strongly connected component (SCC) of G is a set of vertices C ⊆ V such that the induced subgraph G[C] is strongly connected and C is a maximal set in V with this property. We call an SCC trivial if it only contains a single vertex and no edges; and non-trivial otherwise. The SCCs of G partition its vertices and can be found in O(n) symbolic steps [24]. A bottom SCC C in a directed graph G is an SCC with no edges from vertices of C to vertices of V \ C, i.e., an SCC without outgoing edges. Analogously, a top SCC C is an SCC with no incoming edges from V \ C. For more intuition for bottom and top SCCs, consider the graph in which each SCC is contracted into a single vertex (ignoring edges within an SCC). In the resulting directed acyclic graph the sinks represent the bottom SCCs and the sources represent the top SCCs. Note that every graph has at least one bottom and at least one top SCC. If the graph is not strongly connected, then there exist at least one top and at least one bottom SCC that are disjoint and thus one of them contains at most half of the vertices of G. Random Attractors. In an MDP P the random attractor Attr R (P, W ) of a set of vertices W is defined as Attr R (P, W ) = j≥0 Z j where Z 0 = W and Z j+1 = Z j ∪ CPre R (Z j ) for all j > 0. The attractor can be computed with at most |Attr R (P, W ) \ W | + 1 many CPre R operations. Maximal End-Components. Let X be a vertex set without outgoing random edges, i.e., with Out(v) ⊆ X for all v ∈ X ∩ V R . A sub-MDP of an MDP P induced by a vertex set X ⊆ V without outgoing random edges is defined as P [X] = ((X, E ∩ (X × X), (V 1 ∩ X, V R ∩ X), δ). Note that the requirement that X has no outgoing random edges is necessary in order to use the same probabilistic transition function δ. Good End-Components. All algorithms for MDPs with Streett objectives are based on finding good end-components, defined below. Given the union of all good endcomponents, the almost-sure winning set is obtained by computing the almost-sure winning set for the reachability objective with the union of all good end-components as the target set. The correctness of this approach is shown in [7,30] (see also [3,Chap. 10.6.3]). For Streett objectives a good end-component is defined as follows. In the special case of graphs they are called good components.
Definition 1 (Good end-component). Given an MDP P and a set TP = {(L j , U j ) | 1 ≤ j ≤ k} of target pairs, a good end-component is an end-component X of P such that for each 1 ≤ j ≤ k either L j ∩ X = ∅ or U j ∩ X = ∅. A maximal good end-component is a good end-component that is maximal with respect to set inclusion.
Lemma 1 (Correctness of Computing Good End-Components [30, Corollary 2.6.5, Proposition 2.6.9]). For an MDP P and a set TP of target pairs, let X be the set of all maximal good end-components. Then 1 as P, Reach( X∈X X) is equal to 1 as (P, Streett(TP)).
Iterative Vertex Removal. All the algorithms for Streett objectives maintain vertex sets that are candidates for good end-components. For such a vertex set S we (a) refine the maintained sets according to the SCC decomposition of P [S] and (b) for a set of vertices W for which we know that it cannot be contained in a good end-component, we remove its random attractor from S. The following lemma shows the correctness of these operations.  Let X be a good end-component. Then X is an end-component and for each index j, X ∩ U j = ∅ implies X ∩ L j = ∅ . Hence we obtain the following corollary.
For an index j with S ∩ U j = ∅ we call the vertices of S ∩ L j bad vertices. The set of all bad vertices BAD(S) = 1≤i≤k {v ∈ L i ∩ S | U i ∩ S = ∅} can be computed with 2k set operations.

Symbolic Divide-and-Conquer with Lock-Step Search
In this section we present a symbolic version of the lock-step search for strongly connected subgraphs [25]. This symbolic version is used in all subsequent results, i.e., the sub-quadratic symbolic algorithms for graphs and MDPs with Streett objectives, and for MEC decomposition. Divide-and-Conquer. The common property of the algorithmic problems we consider in this work is that the goal is to identify subgraphs of the input graph G = (V, E) that are strongly connected and satisfy some additional properties. The difference between the problems lies in the required additional properties. We describe and analyze the Procedure LOCK-STEP-SEARCH that we use in all our improved algorithms to efficiently implement a divide-and-conquer approach based on the requirement of strong connectivity, that is, we divide a subgraph G[S], induced by a set of vertices S, into two parts that are not strongly connected within G[S] or detect that G[S] is strongly connected. Start Vertices of Searches. The input to Procedure LOCK-STEP-SEARCH is a set of vertices S ⊆ V and two subsets of S denoted by H S and T S . In the algorithms that call the procedure as a subroutine, vertices contained in H S have lost incoming edges (i.e., they were a "head" of a lost edge) and vertices contained in T S have lost outgoing edges (i.e., they were a "tail" of a lost edge) since the last time a superset of S was identified as being strongly connected. For each vertex h of H S the procedure conducts a backward search (i.e., a sequence of Pre operations) within G[S] to find the vertices of S that can reach h; and analogously a forward search (i.e., a sequence of Post operations) from each vertex t of T S is conducted. Intuition for the Choice of Start Vertices. If the subgraph G[S] is not strongly connected, then it contains at least one top SCC and at least one bottom SCC that are disjoint. Further, if for a superset S ⊃ S the subgraph G[S ] was strongly connected, then each top SCC of G[S] contains a vertex that had an additional incoming edge in G[S ] compared to G[S], and analogously each bottom SCC of G[S] contains a vertex that had an additional outgoing edge. Thus by keeping track of the vertices that lost incoming or outgoing edges, the following invariant will be maintained by all our improved algorithms.
The searches from the vertices of H S ∪ T S are performed in lock-step, that is, (a) one step is performed in each of the searches before the next step of any search is done and (b) all searches stop as soon as the first of the searches finishes. This is implemented in Procedure LOCK-STEP-SEARCH as follows. A step in the search from a vertex t ∈ T S (and analogously for h ∈ H S ) corresponds to the execution of the iteration of the for-each loop for t ∈ T S . In an iteration of a for-each loop we might discover that we do not need to consider this search further (see the paragraph on ensuring strong connectivity below) and update the set T S (via T S ) for future iterations accordingly.
Otherwise the set C t is either strictly increasing in this step of the search or the search for t terminates and we return the set of vertices in G[S] that are reachable from t. So the two for-each loops over the vertices of T S and H S that are executed in an iteration of the while-loop perform one step of each of the searches and the while-loop stops as soon as a search stops, i.e., a return statement is executed and hence this implements properties (a) and (b) of lock-step search. Note that the while-loop terminates, i.e., a return statement is executed eventually because for all t ∈ T S (and resp. for all h ∈ H S ) the sets C t are monotonically increasing over the iterations of the while-loop, we have C t ⊆ S, and if some set C t does not increase in an iteration, then it is either removed from T S and thus not considered further or a return statement is executed. Note that when a search from a vertex t ∈ T S stops, it has discovered a maximal set of vertices C that can be reached from t; and analogously for h ∈ H S . Figure 1 shows a small intuitive example of a call to the procedure.
Comparison to Explicit Algorithm. In the explicit version of the algorithm [25,7] the search from vertex t ∈ T S performs a depth-first search that terminates exactly when every edge reachable from t is explored. Since any search that starts outside of a bottom SCC but reaches the bottom SCC has to explore more edges than the search started inside of the bottom SCC, the first search from a vertex of T S that terminates has exactly In symbolically represented graphs it can happen (1) that a search started outside of a bottom (resp. top) SCC terminates earlier than the search started within the bottom (resp. top) SCC and (2) that a search started in a larger (in the number of vertices) top or bottom SCC terminates before one in a smaller top or bottom SCC. We discuss next how we address these two challenges.
Ensuring Strong Connectivity. First, we would like the set returned by Procedure LOCK-STEP-SEARCH to indeed be a top or bottom SCC of G[S]. For this we use the following observation for bottom SCCs that can be applied to top SCCs analogously. If a search starting from a vertex of t 1 ∈ T S encounters another vertex t 2 ∈ T S , t 1 = t 2 , there are two possibilities: either (1) both vertices are in the same SSC or (2) t 1 can reach t 2 but not vice versa. In Case (1) the searches from both vertices can explore all vertices in the SCC and thus it is sufficient to only search from one of them. In Case (2) the SCC of t 1 has an outgoing edge and thus cannot be a bottom SCC. Hence in both cases we can remove the vertex t 1 from the set T S while still maintaining Invariant 1. By Invariant 1 we further have that each search from a vertex of T S that is not in a bottom SCC encounters another vertex of T S in its search and therefore is removed from the set T S during Procedure LOCK-STEP-SEARCH (if no top or bottom SCC is found earlier). This ensures that the returned set is either a top or a bottom SCC. 1 Bound on Symbolic Steps. Second, observe that we can still bound the number of symbolic steps needed for the search that terminates first by the number of vertices in the smallest top or bottom SCC of G[S], since this is an upper bound on the symbolic steps needed for the search started in this SCC. Thus provided Invariant 1, we can bound the number of symbolic steps in Procedure LOCK-STEP-SEARCH to identify a vertex set C S such that C and S \ C are not strongly connected in G[S] by O((|H S | + |T S |) · min(|C|, |S \ C|)). In the algorithms that call Procedure LOCK-STEP-SEARCH we charge the number of symbolic steps in the procedure to the vertices in the smaller set of C and S \ C; this ensures that each vertex is charged at most O(log n) times over the whole algorithm. We obtain the following result (proof in Appendix A).

Graphs with Streett Objectives
Basic Symbolic Algorithm. Recall that for a given graph (with n vertices) and a Streett objective (with k target pairs) each non-trivial strongly connected subgraph without bad vertices is a good component. The basic symbolic algorithm for graphs with Streett objectives repeatedly removes bad vertices from each SCC and then recomputes the SCCs until all good components are found. The winning set then consists of the vertices that can reach a good component. We refer to this algorithm as STREETTGRAPHBASIC.
For the pseudocode and more details see Appendix B.
Proposition 1. Algorithm STREETTGRAPHBASIC correctly computes the winning set in graphs with Streett objectives and requires O(n · min(n, k)) symbolic steps.
Improved Symbolic Algorithm. In our improved symbolic algorithm we replace the recomputation of all SCCs with the search for a new top or bottom SCC with Procedure LOCK-STEP-SEARCH from vertices that have lost adjacent edges whenever there are not too many such vertices. We present the improved symbolic algorithm for graphs with Streett objectives in more detail as it also conveys important intuition for the MDP case. The pseudocode is given in Algorithm STREETTGRAPHIMPR. Iterative Refinement of Candidate Sets. The improved algorithm maintains a set goodC of already identified good components that is initially empty and a set X of candidates for good components that is initialized with the SCCs of the input graph G. The difference to the basic algorithm lies in the properties of the vertex sets maintained in X and the way we identify sets that can be separated from each other without destroying a good component. In each iteration one vertex set S is removed from X and, after the removal of bad vertices from the set, either identified as a good component or split into several candidate sets. By Lemma 2 and Corollary 1 the following invariant is maintained throughout the algorithm for the sets in goodC and X .
Invariant 2 (Maintained Sets). The sets in X ∪ goodC are pairwise disjoint and for every good component C of G there exists a set Y ⊇ C such that either Y ∈ X or Y ∈ goodC.
Lost Adjacent Edges. In contrast to the basic algorithm, the subgraph induced by a set S contained in X is not necessarily strongly connected. Instead, we remember vertices of S that have lost adjacent edges since the last time a superset of S was determined to induce a strongly connected subgraph; vertices that lost incoming edges are contained in H S and vertices that lost outgoing edges are contained in T S . In this way we maintain Invariant 1 throughout the algorithm, which enables us to use Procedure LOCK-STEP-SEARCH with the running time guarantee provided by Theorem 1.
Algorithm STREETTGRAPHIMPR: Improved Alg. for Graphs with Streett Obj. Identifying SCCs. Let S be the vertex set removed from X in a fixed iteration of Algorithm STREETTGRAPHIMPR after the removal of bad vertices in the inner whileloop. First note that if S is strongly connected and contains at least one edge, then it is a good component. If the set S was already identified as strongly connected in a previous iteration, i.e., H S and T S are empty, then S is identified as a good component in line 12. If many vertices of S have lost adjacent edges since the last time a super-set of S was identified as a strongly connected subgraph, then the SCCs of G[S] are determined as in the basic algorithm. To achieve the optimal asymptotic upper bound, we say that many vertices of S have lost adjacent edges when we have |H S | + |T S | ≥ m/ log n, while lower thresholds are used in our experimental results. Otherwise, if not too many vertices of S lost adjacent edges, then we start a symbolic lock-step search for top SCCs from the vertices of H S and for bottom SCCs from the vertices of T S using Procedure LOCK-STEP-SEARCH. The set returned by the procedure is either a top or a bottom SCC C of G[S] (Theorem 1). Therefore we can from now on consider C and S \ C separately, maintaining Invariants 1 and 2.
Algorithm STREETTGRAPHIMPR. A succinct description of the pseudocode is as follows: Lines 1-2 initialize the set of candidates for good components with the SCCs of the input graph. In each iteration of the main while-loop one candidate is considered and the following operations are performed: (a) lines 5-10 iteratively remove all bad vertices; if afterwards the candidate is still strongly connected (and contains at least one edge), it is identified as a good component in the next step; otherwise it is partitioned into new candidates in one of the following ways: (b) if many vertices lost adjacent edges, lines 13-19 partition the candidate into its SCCs (this corresponds to an iteration of the basic algorithm); (c) otherwise, lines 20-28 use symbolic lock-step search to partition the candidate into one of its SCCs and the remaining vertices. The while-loop terminates when no candidates are left. Finally, vertices that can reach some good component are returned. We have the following result (proof in Appendix B).
Theorem 2 (Improved Algorithm for Graphs). Algorithm STREETTGRAPHIMPR correctly computes the winning set in graphs with Streett objectives and requires O(n · √ m log n) symbolic steps.

Symbolic MEC Decomposition
In this section we present a succinct description of the basic symbolic algorithm for MEC decomposition and then present the main ideas for the improved algorithm. Basic symbolic algorithm for MEC decomposition. The basic symbolic algorithm for MEC decomposition maintains a set of identified MECs and a set of candidates for MECs, initialized with the SCCs of the MDP. Whenever a candidate is considered, either (a) it is identified as a MEC or (b) it contains vertices with outgoing random edges, which are then removed together with their random attractor from the candidate, and the SCCs of the remaining sub-MDP are added to the set of candidates. We refer to the algorithm as MECBASIC.
Proposition 2. Algorithm MECBASIC correctly computes the MEC decomposition of MDPs and requires O(n 2 ) symbolic steps.
Improved symbolic algorithm for MEC decomposition. The improved symbolic algorithm for MEC decomposition uses the ideas of symbolic lock-step search presented in Section 3. Informally, when considering a candidate that lost a few edges from the remaining graph, we use the symbolic lock-step search to identify some bottom SCC. We refer to the algorithm as MECIMPR. Since all the important conceptual ideas regarding the symbolic lock-step search are described in Section 3, we relegate the technical details to Appendix C. We summarize the main result (proof in Appendix C).

MDPs with Streett Objectives
Basic Symbolic Algorithm. We refer to the basic symbolic algorithm for MDPs with Streett objectives as STREETTMDPBASIC, which is similar to the algorithm for graphs, with SCC computation replaced by MEC computation. The pseudocode of Algorithm STREETTMDPBASIC together with its detailed description is presented in Appendix D.
Proposition 3. Algorithm STREETTMDPBASIC correctly computes the almost-sure winning set in MDPs with Streett objectives and requires O(n 2 · min(n, k)) symbolic steps.
Remark. The above bound uses the basic symbolic MEC decomposition algorithm. Using our improved symbolic MEC decomposition algorithm, the above bound could be improved to O(n · √ m · min(n, k)).
Improved Symbolic Algorithm. We refer to the improved symbolic algorithm for MDPs with Streett objectives as STREETTMDPIMPR. First we present the main ideas for the improved symbolic algorithm. Then we explain the key differences compared to the improved symbolic algorithm for graphs. A thorough description with the technical details and proofs is presented in Appendix D.
-First, we improve the algorithm by interleaving the symbolic MEC computation with the detection of bad vertices [7,30]. This allows to replace the computation of MECs in each iteration of the while-loop with the computation of SCCs and an additional random attractor computation.
• Intuition of interleaved computation. Consider a candidate for a good endcomponent S after a random attractor to some bad vertices is removed from it. After the removal of the random attractor, the set S does not have random vertices with outgoing edges. Consider that further BAD(S) = ∅ holds. If S is strongly connected and contains an edge, then it is a good end-component. If S is not strongly connected, then P [S] contains at least two SCCs and some of them might have random vertices with outgoing edges. Since end-components are strongly connected and do not have random vertices with outgoing edges, we have that (1) every good end-component is completely contained in one of the SCCs of P [S] and (2) the random vertices of an SCC with outgoing edges and their random attractor do not intersect with any good end-component (see Lemma 2). • Modification from basic to improved algorithm. We use these observations to modify the basic algorithm as follows: First, for the sets that are candidates for good end-components, we do not maintain the property that they are endcomponents, but only that they do not have random vertices with outgoing edges (it still holds that every maximal good end-component is either already identified or contained in one of the candidate sets). Second, for a candidate set S, we repeat the removal of bad vertices until BAD(S) = ∅ holds before we continue with the next step of the algorithm. This allows us to make progress after the removal of bad vertices by computing all SCCs (instead of MECs) of the remaining sub-MDP. If there is only one SCC, then this is a good endcomponent (if it contains at least one edge). Otherwise (a) we remove from each SCC the set of random vertices with outgoing edges and their random attractor and (b) add the remaining vertices of each SCC as a new candidate set. -Second, as for the improved symbolic algorithm for graphs, we use the symbolic lock-step search to quickly identify a top or bottom SCC every time a candidate has lost a small number of edges since the last time its superset was identified as being strongly connected. The symbolic lock-step search is described in detail in Section 3.
Using interleaved MEC computation and lock-step search leads to a similar algorithmic structure for Algorithm STREETTMDPIMPR as for our improved symbolic algorithm for graphs (Algorithm STREETTGRAPHIMPR). The key differences are as follows: First, the set of candidates for good end-components is initialized with the MECs of the input graph instead of the SCCs. Second, whenever bad vertices are removed from a candidate, also their random attractor is removed. Further, whenever a candidate is partitioned into its SCCs, for each SCC, the random attractor of the vertices with outgoing random edges is removed. Finally, whenever a candidate S is separated into C and S \ C via symbolic lock-step search, the random attractor of the vertices with outgoing random edges is removed from C, and the random attractor of C is removed from S.

Experiments
We present a basic prototype implementation of our algorithm and compare against the basic symbolic algorithm for graphs and MDPs with Streett objectives. Models. We consider the academic benchmarks from the VLTS benchmark suite [20], which gives representative examples of systems with nondeterminism, and has been used in previous experimental evaluation (such as [4,11]). Specifications. We consider random LTL formulae and use the tool Rabinizer [27] to obtain deterministic Rabin automata. Then the negations of the formulae give us Streett automata, which we consider as the specifications.
Graphs. For the models of the academic benchmarks, we first compute SCCs, as all algorithms for Streett objectives compute SCCs as a preprocessing step. For SCCs of the model benchmarks we consider products with the specification Streett automata, to obtain graphs with Streett objectives, which are the benchmark examples for our experimental evaluation. The number of transitions in the benchmarks ranges from 300K to 5Million.
MDPs. For MDPs, we consider the graphs obtained as above and consider a fraction of the vertices of the graph as random vertices, which is chosen uniformly at random. We consider 10%, 20%, and 50% of the vertices as random vertices for different experimental evaluation. Experimental evaluation. In the experimental evaluation we compare the number of symbolic steps (i.e., the number of Pre/Post operations 2 ) executed by the algorithms, the comparison of running time yields similar results and is provided in Appendix E.
As the initial preprocessing step is the same for all the algorithms (computing all SCCs for graphs and all MECs for MDPs), the comparison presents the number of symbolic steps executed after the preprocessing. The experimental results for graphs are shown in Figure 2 and the experimental results for MDPs are shown in Figure 3 (in each figure the two lines represent equality and an order-of-magnitude improvement, respectively).
Discussion. Note that the lock-step search is the key reason for theoretical improvement, however, the improvement relies on a large number of Streett pairs. In the experimental evaluation, the LTL formulae generate Streett automata with small number of pairs, which after the product with the model accounts for an even smaller fraction of pairs as compared to the size of the state space. This has two effects: -In the experiments the lock-step search is performed for a much smaller parameter value (O(log n) instead of the theoretically optimal bound of m/ log n), and leads to a small improvement. -For large graphs, since the number of pairs is small as compared to the number of states, the improvement over the basic algorithm is minimal. In contrast to graphs, in MDPs even with small number of pairs as compared to the statespace, the interleaved MEC computation has a notable effect on practical performance, and we observe performance improvement even in large MDPs.

Conclusion
In this work we consider symbolic algorithms for graphs and MDPs with Streett objectives, as well as for MEC decomposition. Our algorithmic bounds match for both graphs and MDPs. In contrast, while SCCs can be computed in linearly many symbolic steps no such algorithm is known for MEC decomposition. An interesting direction of future work would be to explore further improved symbolic algorithms for MEC decomposition. Moreover, further improved symbolic algorithms for graphs and MDPs with Streett objectives is also an interesting direction of future work.

thatC is a bottom SCC. By Invariant 1 there is a search, conducted by
Post operations, that starts from a vertex t ∈ T S withinC and that is not canceled, and therefore this search terminates after at most |C| many Post operations. Other searches may terminate earlier but this gives an upper bound of O((|H S | + |T S |) · |C|) on the number of symbolic steps until the lock-step search terminates. Finally, consider the returned set C ← LOCK-STEP-SEARCH(G, S, H S , T S ). There are two possible cases: either (i) S = C, which implies C =C so the number of symbolic steps can be bounded by O((|H S | + |T S |) · |C|), or (ii) S = C. In the second case, sinceC is (some) smallest SCC, C is an SCC, and S \ C contains at least one SCC, we have |C| ≤ |C| and |C| ≤ |S \ C|, and hence we can bound the number of symbolic steps in this case by O((|H S | + |T S |) · min(|C|, |S \ C|)).

B.1 Basic Symbolic Algorithm for Graphs with Streett Objectives
The pseudocode of the basic symbolic algorithm for graphs with Streett objectives is given in Algorithm STREETTGRAPHBASIC.
The basic symbolic algorithm for Streett objectives on graphs finds good components as follows. The algorithm maintains two sets of vertex sets: goodC contains identified good components and is initially empty; X contains candidates for good components and is initialized with the SCCs of the input graph G. The sets in X are strongly connected subgraphs of G throughout the algorithm. In each iteration of the while-loop one of Algorithm STREETTGRAPHBASIC: Basic Algorithm for Graphs with Streett Obj. the candidate sets S maintained in X is considered. If the set S does not contain bad vertices and contains at least one edge, then it is a good component and added to goodC. Otherwise, the set of bad vertices B in S is removed from S; the subgraph induced by S = S \ B might not be strongly connected but every good component contained in S must still be strongly connected, therefore the maximal strongly connected subgraphs of G[S ] are added to X as new candidates for good components. By Lemma 2 and Corollary 1 this procedure maintains the property that every good component of G is completely contained in one of the vertex sets of goodC or X . Further in each iteration either (a) vertices are removed or separated into different vertex sets or (b) a new good component is identified. Thus after at most O(n) iterations the set X is empty and all good components of G are contained in goodC. Furthermore, whenever bad vertices are removed from a given candidate set, the number of target pairs this candidate set intersects is reduced by one. Thus each vertex is considered in at most O(k) iterations of the main while-loop. Finally, the set of vertices that can reach a good component is determined (by O(n) Pre operations) and output as the winning set. Since computing SCCs can be done in O(n) symbolic steps, the total number of symbolic steps of the basic algorithm is bounded by O(n · min(n, k)).

B.2 Improved Symbolic Algorithm for Graphs with Streett Objectives
Lemma 3 (Invariants of Improved Algorithm for Graphs). Invariant 1 and Invariant 2 are preserved throughout Algorithm STREETTGRAPHIMPR, i.e., they hold before the first iteration, after each iteration, and after termination of the main while-loop. Further, Invariant 1 is preserved during each iteration of the main while-loop.
Proof. Invariant 1. Whenever a new candidate S is added as a result from ALLSCCS, it is strongly connected, and we set H S = T S = ∅; this in particular implies that the invariant is satisfied after the initialization of the algorithm.
By induction and Theorem 1, the invariant is satisfied whenever Procedure LOCK-STEP-SEARCH returns a candidate C and we set H C = T C = ∅. Now consider an update of a candidate S where some subset B is deleted from it and assume the invariant holds before the update. In these cases we update H S and T S by setting H S ← (H S ∪ Post(B)) ∩ S and T S ← (T S ∪ Pre(B)) ∩ S. This adds the vertices that remain in S and have an edge from a vertex of B to H S and those with an edge to B to T S . Suppose a new top (resp. bottom) SCCS ⊆ S emerges in S by the removal of B from S. Then some vertex ofS had an outgoing edge to B (resp. an incoming edge from B) and thus is contained in the updated set T S (resp. H S ), maintaining the invariant. This happens whenever we remove BAD(S) from S, and whenever we subtract a result from Procedure LOCK-STEP-SEARCH C from S. Invariant 2 -Disjointness. The sets in X ∪ goodC are pairwise disjoint at the initialization since goodC is initialized as ∅. Furthermore, whenever a set S is added to goodC in an iteration of the main while-loop, a supersetS ⊇ S is removed from X in the same iteration of the while-loop. Therefore by induction the disjointness of the sets in X ∪ goodC is preserved. Invariant 2 -Containment of good components. At initialization, X contains all SCCs of the input graph G. Each good component C of G is strongly connected, so there exists an SCC Y ⊇ C such that Y ∈ X for each good component C.
Consider a set S ∈ X that is removed from X at the beginning of an iteration of the main while-loop. Consider further a good component C of G such that C ⊆ S. We require that a set Y ⊇ C is added to either X or goodC in this iteration of the main while-loop.
First, whenever we remove BAD(S) from S, by Corollary 1 we maintain the fact that C ⊆ S. Second, G[S] contains an edge since C ⊆ S. Finally, one of the three cases happens: Case (1): If |H S | + |T S | = 0, then the set S ⊇ C is added to goodC. Since C ⊆ S is strongly connected, it is completely contained in some SCC Y of G[S], and Y is added either to X or to goodC.
Case (3): If 0 < |H S | + |T S | < m/ log n, then the algorithm either adds S ⊇ C to goodC, or partitions S intoS and S \S. Suppose the latter case happens, then by Theorem 1 we have thatS is an SCC of G[S]. Further, since C ⊆ S is strongly connected, it is completely contained in some SCC of G[S]. Therefore either C ⊆S or C ⊆ (S \S), and bothS and S \S are added to X .
By the above case analysis we have that a set Y ⊇ C is added to either X or goodC in the iteration of the main while-loop, and thus the invariant is preserved throughout the algorithm.

Proof (of Theorem 2).
Correctness. Whenever a candidate set S is added to goodC, it contains an edge by the check at line 11, and BAD(S) = ∅ by the check at line 6. Furthermore, (a) at line 12, S is strongly connected by Invariant 1, (b) at line 16, S is strongly connected by the result of ALLSCCS, and (c) at line 22, S is strongly connected by Theorem 1. Therefore we have that whenever a candidate set is added to goodC, it is indeed a good component (soundness).
Finally, by soundness, Invariant 2, the termination of the algorithm (shown below), and the fact that X = ∅ at the termination of the algorithm, we have that goodC contains all good components of G (completeness). Symbolic steps analysis. By [24], the initialization with the SCCs of the input graph takes O(n) symbolic steps. Furthermore, the reachability computation in the last step takes O(n) Pre operations.
In each iteration of the outer while-loop, a set S is removed from X and either (a) a set S ⊆ S is added to goodC and no set is added to X or (b) at least two sets that are Note that every vertex in each of H S and T S can be attributed to at least one unique implicit edge deletion since we only add vertices to H S resp. T S that are successors resp. predecessors of vertices that were separated from S (or deleted from the maintained graph). Whenever the case |H S | + |T S | ≥ m/ log n occurs, for all subsets C ⊆ S that are then added to X , we initialize H C = T C = ∅. Therefore the case |H S | + |T S | ≥ m/ log n can happen at most O( √ m log n) times throughout the algorithm since there are at most m edges that can be deleted, and hence in total takes O(n · √ m log n) symbolic steps.
It remains to bound the number of symbolic steps in Procedure LOCK-STEP-SEARCH. Let C be the set returned by the procedure; we charge the symbolic steps in this call of the procedure to the vertices of the smaller set of C and S \ C. By Theorem 1 we have either (a) C = S, the number of symbolic steps in this call is bounded by O( m/ log n · |C|), and the set S is added to goodC or (b) min(|C|, |S \ C|) ≤ |S|/2 and the number of symbolic steps in this call is bounded by O( m/ log n · min(|C|, |S \ C|)). Case (a) can happen at most once for the vertices of C, and for case (b) note that the size of a set containing a specific vertex can be halved at most O(log n) times; thus we charge each vertex at most O(log n) times. Hence we can bound the total number of symbolic steps in all calls to the procedure by O(n · √ m log n).

C.1 Basic Symbolic Algorithm for MEC decomposition
Recall that an end-component is a set of vertices that (a) has no random edges to vertices not in the set and its induced sub-MDP is (b) strongly connected and (c) contains at least one edge. Algorithm MECBASIC computes all maximal end-components of a given MDP and is formulated as to highlight the similarities to the algorithms for graphs and MDPs with Streett objectives. The algorithm maintains two sets, the set goodC of identified

Algorithm MECBASIC: Basic Algorithm for Maximal End-Components
Input : an MDP P = (G = (V, E), (V1, VR)) Output : the set of maximal end-components of P maximal end-components that is initially empty and the set X of candidates for maximal end-components that is initialized with the SCCs of the MDP. In each iteration of the while-loop one set S is removed from X and either (1a) identified as a maximal end-component and added to goodC or (1b) removed because the induced sub-MDP does not contain an edge or (2) it contains vertices with outgoing random edges. In the latter case these vertices rout are identified and their random attractor is removed from S. After this step the sub-MDP induced by the remaining vertices of S might not be strongly connected any more. Therefore the SCCs of this sub-MDP are determined and added to X as new candidates for maximal end-components. Note that this maintains the invariants that (i) each set in X induces a strongly connected subgraph and (ii) each end-component is a subset of one set in either goodC or X . By (i) a set in X is an end-component if it does not have outgoing random edges and the induced sub-MDP contains an edge, i.e., in particular this holds for the sets added to goodC (soundness). By (ii) and X = ∅ at termination of the while-loop the algorithm identifies all maximal end-components of the MDP (completeness). Since both (1) and (2) can happen at most O(n) times, there are O(n) iterations of the while-loop. In each iteration the most expensive operations are the computation of a random attractor and of SCCs, which can both be done in O(n) symbolic steps. Thus Algorithm MECBASIC correctly computes all maximal end-components of an MDP and takes O(n 2 ) symbolic steps.

C.2 Improved Symbolic Algorithm for MEC decomposition
Informal description. We show how to determine all maximal end-components (MECs) of an MDP in O(n √ m) symbolic operations. The difference to the basic algorithm lies in the way strongly connected parts of the MDP are identified after the deletion of vertices that cannot be contained in a MEC. For this the symbolic lock-step search from Section 3 is used whenever not too many edges have been deleted since the last re-computation of SCCs.
Let P be the given MDP and G = (V, E) its underlying graph. The algorithm maintains two sets of vertex sets: the set goodC of already identified MECs that is initialized with the empty set and the set X that is initialized with the SCCs of G and contains vertex sets that are candidates for MECs. The algorithm preserves the following invariant for the goodC and X over the iterations of the while-loop and returns the set goodC when the set X is empty after an iteration of the while-loop.
Invariant 4 (Maintained Sets). The sets in X ∪ goodC are pairwise disjoint and for every maximal end-component X of G there exists a set Y ⊇ X such that either Y ∈ X or Y ∈ goodC.
For each vertex set S in X additionally a subset T S of S is maintained that contains vertices that have lost outgoing edges since the last time a superset of S was identified as strongly connected. We use the following restrictions of Invariant 1 and Theorem 1 (presented in Section 3) to bottom SCCs only. Proof. The proof of Theorem 5 is a straightforward simplification of the proof of Theorem 1 located in Appendix A.
Initially the sets T S are empty. The algorithm maintains Invariant 5 for all S ∈ X . This will ensure the correctness and the number of symbolic steps of Procedure LOCK-STEP-SEARCH (Section 3) as called by the algorithm.
In each iteration of the while-loop one vertex set S is removed from X and processed. First the random vertices of S with edges to vertices of V \ S are identified and their random attractor is removed from S. After this step, there are no random vertices with edges from S to V \ S. The predecessors of the removed vertices that are contained in S are added to T S and additionally T S is updated to only include vertices that are still in S. This preserves Invariant 5 (see also [30,Lemma 4.5.2]). The number of symbolic steps for the attractor computation can be charged to the removed vertices and is therefore bounded by O(n) in total.
If afterwards G[S] does not contain an edge anymore, then S is not considered further and the algorithm continues with the next iteration. Otherwise one of three cases happens. Case (1): If T S is empty, then by Invariant 5 G[S] is strongly connected, contains at least one edge and does not contain a random vertex with edges to V \ S, i.e., S is an end-component, and by Invariant 4 it is a MEC. In this case the algorithm adds the set S to goodC, which preserves both invariants and can happen at most O(n) times. Case (2): If there are at least √ m vertices in T S , then the set T S is deleted and as in the basic algorithm all SCCs of G[S] are computed and add to X as new candidates for MECs. For each of the SCCs C a set T C is initialized with the empty set. As a vertex is Algorithm MECIMPR: Improved Algorithm for Maximal End-Components Input : an MDP P = (G = (V, E), (V1, VR)) Output : the set of maximal end-components of P Case (3): If T S contains less than √ m vertices, then Procedure LOCK-STEP-SEARCH(G, S, ∅, T S ) is called. By Invariant 5 and Theorem 5 the procedure returns a bottom SCC C of G[S] in O(|T S | · |C|) many symbolic steps. Since there are no random edges between S and V \ S in P and C has no outgoing edges in G[S], we have that C is an endcomponent if it contains at least one edge. By Invariant 4 it is also a MEC and is correctly added to goodC. As the sets in goodC are not considered further by the algorithm, we can charge the symbolic steps of Procedure LOCK-STEP-SEARCH to the vertices of C. Thus this part takes at most O(n √ m) symbolic steps over the whole algorithm. The vertices of S \ C are added back to X , which preserves Invariant 4. The predecessors of C in S \ C are added to T S\C and vertices of C are removed from T S\C , which preserves Invariant 5.
By the above case analysis we have that each vertex set that is added to goodC is indeed a MEC (soundness). By Invariant 4 and X = ∅ at termination of the algorithm we further have completeness. In each iteration either S does not contain an edge and is not considered further, a set is added to goodC (and not contained in X after that) or case (2)  Invariant 4 -Disjointness. The sets in X ∪ goodC are pairwise disjoint at the initialization since goodC is initialized as ∅. Furthermore, whenever a set S is added to goodC in an iteration of the main while-loop, a supersetS ⊇ S is removed from X in the same iteration of the while-loop. Therefore by induction the disjointness of the sets in X ∪ goodC is preserved.
Consider a set S ∈ X that is removed from X at the beginning of an iteration of the main while-loop. Consider further a maximal end-component X of P such that X ⊆ S. We require that a set Y ⊇ X is added to either X or goodC in this iteration of the main while-loop.
First, after we remove Attr R (G, S ∩ V R ∩ Pre(V \ S)) from S, we maintain the fact that X ⊆ S by Lemma 2. Second, G[S] contains an edge since X ⊆ S. Finally, one of the three cases happens: Case (1): If |T S | = 0, then the set S ⊇ X is added to goodC. Case (2): If |T S | ≥ √ m, then the algorithm computes the SCCs of G[S]. Since X ⊆ S is strongly connected, it is completely contained in some SCC Y of G[S], and Y is added to X . Case (3): If 0 < |T S | < √ m, then the algorithm partitions S into C and S \ C. By Theorem 5 we have that C is a (bottom) SCC of G[S]. Since X ⊆ S is strongly connected, it is completely contained in some SCC of G[S]. Therefore either X ⊆ C or X ⊆ (S \ C). The set S \ C is added to X . If X ⊆ C, then in particular G[C] contains an edge, and C is added to goodC. By the above case analysis we have that a set Y ⊇ X is added to either X or goodC in the iteration of the main while-loop.

Proof (of Theorem 3).
Correctness. A candidate set can be added to goodC in three cases. When S is added to goodC at line 12 (resp. at line 17), then it contains an edge by the check at line 10, it is strongly connected by |T S | = 0 and Invariant 5 (resp. by the result of ALLSCCS), and it has no random vertices with edges to V \ S by the random attractor removal at lines 6-9. When C is added at line 25, then it contains an edge by the check at line 24, it is strongly connected by Theorem 5, it contains no random vertices with edges to V \ S by the random attractor removal at lines 6-9, and it contains no random vertices with edges to S \ C by the fact that C is a bottom SCC of G[S] (see Theorem 5). Therefore we have that whenever a candidate set is added to goodC, it is an end-component, and by induction and Invariant 4 we have that it is a maximal end-component (soundness).
Finally, by soundness, Invariant 4, the termination of the algorithm (shown below), and the fact that X = ∅ at the termination of the algorithm, we have that goodC contains all the maximal end-components of P (completeness). Symbolic steps analysis. By [24], the initialization with the SCCs of a given MDP takes O(n) symbolic steps.
In each iteration of the outer while-loop, a set S is removed from X and (a) S is added to goodC, or (b) at least two sets that are (subsets of) a partition of S are added to X , or (c) S is partitioned into two sets, one of them may be added to goodC and the other is added to X . All three cases can happen at most O(n) times, so there can be at most O(n) iterations of the outer while-loop. The Pre and Post operations at lines 6, 9, 10, 24, and 27 can be charged to the iterations of the outer while-loop.
Each CPre R operation executed as a part of the random attractor computation at line 7 adds at least one vertex to A, and the vertices of A are then not considered any further in the algorithm. Therefore there can, in total, be at most O(n) CPre R operations over all attractor computations at line 7.
Note that every vertex in each of T S can be attributed to at least one unique implicit edge deletion since we only add vertices to T S that are predecessors of the vertices that were separated from S (or deleted from the maintained graph). Whenever the case |T S | ≥ √ m occurs, for all subsets C ⊆ S that are then added to X , we initialize T C = ∅. Therefore, the case |T S | ≥ √ m can happen at most O( √ m) times throughout the algorithm since there are at most m edges that can be deleted. By [24] we have a bound O(n) for one iteration, so we can bound the total number of symbolic steps in all iterations of this case by O(n · √ m). It remains to bound the number of symbolic steps in Procedure LOCK-STEP-SEARCH. Let C be the set returned by LOCK-STEP-SEARCH(G, S, ∅, T S ). By Theorem 5 and the fact that |T S | < √ m, the number of symbolic steps in this call is bounded by O( √ m · |C|), and the set C is not considered further in the algorithm after this call. Hence we can bound the total number of symbolic steps in all calls of the procedure by O(n · √ m). The pseudocode of the basic symbolic algorithm for MDPs with Streett objectives is given in Algorithm STREETTMDPBASIC. The key differences compared to Algorithm STREETTGRAPHBASIC are as follows: (a) SCC computation is replaced by MEC computation; (b) along with the removal of bad vertices, their random attractor is also removed; and (c) removing the attractor ensures that the check required for trivial SCCs for graphs (line 9) is not required any further.
To compute the almost-sure winning set for MDPs with Streett objectives, we first find all (maximal) good end-components and then solve almost-sure reachability with the union of the good end-components as target set as the last step of the algorithm. This is correct by Lemma 1. Towards finding all good end-components, the algorithm maintains two sets, the set goodEC of identified good end-components that is initially empty and the set X of end-components that are candidates for good end-components that is initialized with the MECs of the MDP. In each iteration of the while-loop one set S is removed from the set of candidates X and the set of bad vertices BAD(S) of S is determined. If BAD(S) is empty, then S is a good end-component and added to goodEC. Otherwise the random attractor of BAD(S) in P [S] is removed from S, which by Corollary 1 does not remove any vertices that are in a good end-component. The remaining vertices of S have no outgoing random edges and thus still induce a sub-MDP but the sub-MDP might not be strongly connected any more. Then the MECs of this sub-MDP are added to X . These operations maintain the invariants that (i) each set in X is an end-component and (ii) each good end-component is a subset of one set in either goodEC or X . By (i) a set in X is a (maximal) good end-component if it does not contain any bad vertices, i.e., in particular this holds for the sets added to goodEC (soundness). By (ii) and X = ∅ at termination of the while-loop the algorithm identifies all (maximal) good end-components of the MDP (completeness). Since in each iteration of the while-loop either (1) a set is removed from X and added to goodEC or (2) bad vertices are removed from a set and not considered further by the algorithm, there can be at most O(n) iterations of the while-loop. Furthermore, whenever bad vertices are removed, then the number of target pairs a given candidate set intersects is reduced by one. Thus each vertex is considered in at most O(k) iterations of the while-loop. The most expensive operation in the while-loop is the computation of the MECs. Denoting the number of symbolic steps for the MEC computation with O(MEC), the number of symbolic steps of Algorithm STREETTMDPBASIC is O(min(n, k) · MEC) (assuming that the number of symbolic steps for the almost-sure reachability computation is lower than that).

D.2 Improved Symbolic Algorithm for MDPs with Streett Objectives
We present the technical details regarding the improved symbolic algorithm for MDPs with Streett objectives. The main ideas of the algorithm are presented in Section 6. The pseudocode is given in Algorithm STREETTMDPIMPR.
The following invariant is maintained throughout Algorithm STREETTMDPIMPR for the sets in goodEC and X .
Invariant 7 (Maintained Sets). The sets in X ∪ goodEC are pairwise disjoint and for every good end-component C of G there exists a set Y ⊇ C such that either Y ∈ X or Y ∈ goodEC.
Furthermore, the algorithm maintains the invariant that each candidate for a good end-component S ∈ X contains no random edges to vertices not in S.

Invariant 8 (No Random Outgoing Edges).
Given an MDP P and its underlying graph G = (V, E), for each set S ∈ X there are no random vertices in S with edges to vertices in V \ S.
Finally, for each candidate set S ∈ X the algorithm remembers sets H S and T S of vertices that have lost incoming resp. outgoing edges since the last time a superset of S was identified as being strongly connected. The algorithm maintains Invariant 1 and therefore it can use Procedure LOCK-STEP-SEARCH together with its correctness guarantee and bound on symbolic steps provided by Theorem 1.
Lemma 9 (Invariants of Improved Algorithm for MDPs). Invariant 1, Invariant 7, and Invariant 8 are preserved throughout Algorithm STREETTMDPIMPR, i.e., they hold before the first iteration, after each iteration, and after termination of the main while-loop. Further, Invariant 1 is preserved during each iteration of the main while-loop.
Input : MDP P = ((V, E), (V1, VR), δ) and pairs TP = {(Li, Ui) | 1 ≤ i ≤ k} Output : 1 as (P, Streett(TP)) Proof. Invariant 1. The proof is a minor extension of the maintenance proof for Algorithm STREETTGRAPHIMPR that is given in Appendix B. In terms of strong connectivity of a candidate S and the maintenance of the sets H S and T S , the only difference to the graph case is that after an SCC C is computed by ALLSCCS or Procedure LOCK-STEP-SEARCH, another subset of vertices A (vertices with outgoing random edges and their random attractor) is removed from C. In this case the invariant is maintained by initializing H C resp. T C with the vertices of C \ A with edges from resp. to vertices of A, i.e., H C ← Post(A) ∩ C and T C ← Pre(A) ∩ C. Invariant 7 -Disjointness. The sets in X ∪ goodEC are pairwise disjoint at the initialization since goodEC is initialized as ∅. Furthermore, whenever a set S is added to goodEC in an iteration of the main while-loop, a supersetS ⊇ S is removed from X in the same iteration of the while-loop. Therefore by induction the disjointness of the sets in X ∪ goodEC is preserved. Invariant 7 -Containment of good end-components. At initialization, X contains all MECs of the input MDP P = (G = (V, E), (V 1 , V R ), δ). Each good end-component C of P is an end-component, so there exists a MEC Y ⊇ C such that Y ∈ X for each good end-component C.
Consider a set S ∈ X that is removed from X at the beginning of an iteration of the main while-loop. Consider further a good end-component C of P such that C ⊆ S. We require that a set Y ⊇ C is added to either X or goodEC in this iteration of the main while-loop.
First, whenever we remove Attr R (P [S], BAD(S)) from S, by Corollary 1, we maintain the fact that C ⊆ S. Second, P [S] contains an edge since C ⊆ S. Finally, one of the three cases happens: Case (1): If |H S | + |T S | = 0, then the set S ⊇ C is added to goodEC. Case (2): If |H S | + |T S | ≥ m/ log n, then the algorithm computes the SCCs of P [S]. If S itself is the (sole) SCC of P [S], then it is added to goodEC. Otherwise, since C ⊆ S is strongly connected, it is completely contained in some SCC Y of P [S]. Furthermore, since C has no outgoing random edges, by Lemma 2 it is contained in Y even after we remove Attr R (P [Y ], Y ∩ V R ∩ Pre(S \ Y )) from it. Finally, Y is added to X . Case (3): If 0 < |H S | + |T S | < m/ log n, then the algorithm either adds S ⊇ C to goodEC, or partitions S intoS and S \S. Suppose the latter case happens, then by Theorem 1 we have thatS is an SCC of P [S]. Further, since C ⊆ S is strongly connected, it is completely contained in some SCC of P [S]. Therefore either C ⊆S or C ⊆ (S \S). If C ⊆S, then by Lemma 2 after the removal of Attr R (P [S],S ∩ V R ∩ Pre(S \S)) fromS we maintain that C ⊆S. If C ⊆ (S \S), then by Lemma 2 after the removal of Attr R (P [S],S) from (S \S) we maintain that C ⊆ (S \S). Finally, bothS and S \S are added to X . By the above case analysis we have that a set Y ⊇ C is added to either X or goodEC in the iteration of the main while-loop. Invariant 8. Given an MDP, the set X is initialized with the MECs of the MDP, and by definition they have no random outgoing edges. Therefore the invariant holds before the first iteration of the main while-loop.
Consider a candidate set S ∈ X in a given iteration of the main while-loop. By the induction hypothesis, S has no random vertices with edges to V \ S. First, some bad vertices can be iteratively removed from S. At each such removal, the random attractor to these vertices is removed from S as well. After the removal, by the definition of a random attractor, S has no random outgoing edges to the attractor, and therefore by induction has no random outgoing edges to V \ S. Second, S may be partitioned into at least two proper subsets. Then for each such subset C, the random attractor to random vertices in C with edges to S \ C is removed from C. By induction and the definition of a random attractor, after the removal C contains no random outgoing edges to V \ C and adding it to X preserves the invariant.
Proof (of Theorem 4). Correctness. Whenever a candidate set S is added to goodEC, it contains an edge by the check at line 12, BAD(S) = ∅ by the check at line 6, and it has no outgoing random edges by Invariant 8 and the random attractor removal at line 8. Furthermore, (a) at line 13, S is strongly connected by Invariant 1, (b) at line 17, S is strongly connected by the result of ALLSCCS, and (c) at line 28, S is strongly connected by Theorem 1. Therefore we have that whenever a candidate set is added to goodEC, it is indeed a good end-component (soundness).
Finally, by soundness, Invariant 7, the termination of the algorithm (shown below), and the fact that X = ∅ at the termination of the algorithm, we have that goodEC contains all good end-components of G (completeness). Symbolic steps analysis. When using our improved symbolic algorithm for MEC decomposition, the initialization takes O(n · √ m) symbolic steps by Theorem 3. In each iteration of the outer while-loop, a set S is removed from X and either (a) a set S ⊆ S is added to goodEC and no set is added to X or (b) at least two sets that are (subsets of) a partition of S are added to X . Similarly, each CPre R operation executed as a part of a random attractor computation adds at least one vertex to the attractor, and the vertices of the attractor are then not considered any further in the algorithm. Therefore there can, in total, be at most O(n) CPre R operations over all attractor computations at lines 7, 21, 30, and 30.
Note that every vertex in each of H S and T S can be attributed to at least one unique implicit edge deletion since we only add vertices to H S resp. T S that are successors resp. predecessors of vertices that were separated from S (or deleted from the maintained graph). Whenever the case |H S | + |T S | ≥ m/ log n occurs, for all subsets C ⊆ S that are then added to X , we initialize H C = T C = ∅. Therefore, the case |H S | + |T S | ≥ m/ log n can happen at most O( √ m log n) times throughout the algorithm since there are at most m edges that can be deleted. In one iteration of this case, the number of symbolic steps executed by ALLSCCS together with symbolic steps executed at lines 20, 23, and 24, is bounded by O(n) [24].
It remains to bound the number of symbolic steps in Procedure LOCK-STEP-SEARCH. Let C be the set returned by the procedure; we charge the symbolic steps in this call of the procedure to the vertices of the smaller set of C and S \ C. By Theorem 1 we have either (a) C = S, the number of symbolic steps in this call is bounded by O( m/ log n · |C|), and the set S is added to goodEC or (b) min(|C|, |S \ C|) ≤ |S|/2 and the number of symbolic steps in this call is bounded by O( m/ log n · min(|C|, |S \ C|)). Case (a) can happen at most once for the vertices of C, and for case (b) note that the size of a set containing a specific vertex can be halved at most O(log n) times; thus we charge each vertex at most O(log n) times. Hence we can bound the total number of symbolic steps in all calls to the procedure by O(n · √ m log n).

E Details of Section 7: Experiments
We present the results of the experimental evaluation when comparing based on the time. In all the figures, both axes plot the amount of seconds spent on the execution. Similar to the case of symbolic steps, we begin the measurement after the initial preprocessing step (computing all SCCs for graphs and all MECs for MDPs) is finished. The results for graphs are shown in Figure 4 and the results for MDPs are shown in Figure 5.