Formal Methods in System Design

, Volume 43, Issue 2, pp 268–284 | Cite as

A survey of partial-observation stochastic parity games

  • Krishnendu Chatterjee
  • Laurent Doyen
  • Thomas A. Henzinger


We consider two-player zero-sum stochastic games on graphs with ω-regular winning conditions specified as parity objectives. These games have applications in the design and control of reactive systems. We survey the complexity results for the problem of deciding the winner in such games, and in classes of interest obtained as special cases, based on the information and the power of randomization available to the players, on the class of objectives and on the winning mode.

On the basis of information, these games can be classified as follows: (a) partial-observation (both players have partial view of the game); (b) one-sided partial-observation (one player has partial-observation and the other player has complete-observation); and (c) complete-observation (both players have complete view of the game). The one-sided partial-observation games have two important subclasses: the one-player games, known as partial-observation Markov decision processes (POMDPs), and the blind one-player games, known as probabilistic automata.

On the basis of randomization, (a) the players may not be allowed to use randomization (pure strategies), or (b) they may choose a probability distribution over actions but the actual random choice is external and not visible to the player (actions invisible), or (c) they may use full randomization.

Finally, various classes of games are obtained by restricting the parity objective to a reachability, safety, Büchi, or coBüchi condition. We also consider several winning modes, such as sure-winning (i.e., all outcomes of a strategy have to satisfy the winning condition), almost-sure winning (i.e., winning with probability 1), limit-sure winning (i.e., winning with probability arbitrarily close to 1), and value-threshold winning (i.e., winning with probability at least ν, where ν is a given rational).


Partial-observation games Stochastic games ω-regular conditions and parity objectives 



The research was supported by Austrian Science Fund (FWF) Grant No. P 23499-N23 on Modern Graph Algorithmic Techniques in Formal Verification, FWF NFN Grant No. S11407-N23 (RiSE), ERC Start grant (279307: Graph Games), Microsoft faculty fellows award, ERC Advanced grant QUAREM, and FWF Grant No. S11403-N23 (RiSE).


  1. 1.
    Alur R, Henzinger TA, Kupferman O (2002) Alternating-time temporal logic. J ACM 49:672–713 MathSciNetCrossRefGoogle Scholar
  2. 2.
    Baier C, Bertrand N, Größer M (2008) On decision problems for probabilistic Büchi automata. In: FoSSaCS. LNCS, vol 4962. Springer, Berlin, pp 287–301 Google Scholar
  3. 3.
    Baier C, Größer M (2005) Recognizing omega-regular languages with probabilistic automata. In: LICS, pp 137–146 Google Scholar
  4. 4.
    Bertrand N, Genest B, Gimbert H (2009) Qualitative determinacy and decidability of stochastic games with signals. In: LICS. IEEE Comput Soc, Los Alamitos, pp 319–328 Google Scholar
  5. 5.
    Berwanger D, Doyen L (2008) On the power of imperfect information. In: FSTTCS, Dagstuhl seminar proceedings 08004. Internationales Begegnungs- und Forschungszentrum fuer Informatik (IBFI) Google Scholar
  6. 6.
    Billingsley P (ed) (1995) Probability and measure. Wiley-Interscience, New York MATHGoogle Scholar
  7. 7.
    Büchi JR, Landweber LH (1969) Solving sequential conditions by finite-state strategies. Trans Am Math Soc 138:295–311 MATHGoogle Scholar
  8. 8.
    Cerný P, Chatterjee K, Henzinger TA, Radhakrishna A, Singh R (2011) Quantitative synthesis for concurrent programs. In: Proc of CAV. LNCS, vol 6806. Springer, Berlin, pp 243–259 Google Scholar
  9. 9.
    Chadha R, Sistla AP, Viswanathan M (2009) On the expressiveness and complexity of randomization in finite state monitors. J ACM 56(5) Google Scholar
  10. 10.
    Chadha R, Sistla AP, Viswanathan M (2009) Power of randomization in automata on infinite strings. In: Proc of CONCUR. LNCS, vol 5710. Springer, Berlin, pp 229–243 Google Scholar
  11. 11.
    Chadha R, Sistla AP, Viswanathan M (2011) Probabilistic Büchi automata with non-extremal acceptance thresholds. In: Proc of VMCAI, pp 103–117 Google Scholar
  12. 12.
    Chatterjee K (2007) Stochastic ω-regular games. PhD thesis, UC Berkeley Google Scholar
  13. 13.
    Chatterjee K, Doyen L (2010) The complexity of partial-observation parity games. In: Proc of LPAR (Yogyakarta), pp 1–14 Google Scholar
  14. 14.
    Chatterjee K, Doyen L (2012) Partial-observation stochastic games: how to win when belief fails. In: LICS’12, IEEE Press, New York. CoRR 1107.2141, July 2011 Google Scholar
  15. 15.
    Chatterjee K, Doyen L, Gimbert H, Henzinger TA (2010) Randomness for free. In: MFCS. Springer, Berlin Google Scholar
  16. 16.
    Chatterjee K, Doyen L, Henzinger TA (2010) Qualitative analysis of partially-observable Markov decision processes. In: MFCS, pp 258–269 Google Scholar
  17. 17.
    Chatterjee K, Doyen L, Henzinger TA, Raskin J-F (2007) Algorithms for omega-regular games of incomplete information. Log Methods Comput Sci 3(3:4) Google Scholar
  18. 18.
    Chatterjee K, Jurdziński M, Henzinger TA (2003) Simple stochastic parity games. In: CSL’03. LNCS, vol 2803. Springer, Berlin, pp 100–113 Google Scholar
  19. 19.
    Chatterjee K, Jurdziński M, Henzinger TA (2004) Quantitative stochastic parity games. In: SODA’04. SIAM, Philadelphia, pp 121–130 Google Scholar
  20. 20.
    Condon A (1992) The complexity of stochastic games. Inf Comput 96(2):203–224 MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Cristau J, David C, Horn F (2010) How do we remember the past in randomised strategies? In: GANDALF, pp 30–39 Google Scholar
  22. 22.
    de Alfaro L, Henzinger TA (2001) Interface theories for component-based design. In: EMSOFT. LNCS, vol 2211. Springer, Berlin, pp 148–165 Google Scholar
  23. 23.
    Emerson EA, Jutla C (1991) Tree automata, mu-calculus and determinacy. In: FOCS. IEEE Press, New York, pp 368–377 Google Scholar
  24. 24.
    Gimbert H, Oualhadj Y (2010) Probabilistic automata on finite words: decidable and undecidable problems. In: Proc of ICALP. LNCS, vol 6199. Springer, Berlin, pp 527–538 Google Scholar
  25. 25.
    Gripon V, Serre O (2009) Qualitative concurrent stochastic games with imperfect information. In: ICALP(2). LNCS, vol 5556. Springer, Berlin, pp 200–211 Google Scholar
  26. 26.
    Henzinger TA, Kupferman O, Rajamani S (2002) Fair simulation. Inf Comput 173:64–81 MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Immerman N (1981) Number of quantifiers is better than number of tape cells. J Comput Syst Sci 22:384–406 MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Kechris A (1995) Classical descriptive set theory. Springer, Berlin CrossRefMATHGoogle Scholar
  29. 29.
    Kress-Gazit H, Fainekos GE, Pappas GJ (2009) Temporal-logic-based reactive mission and motion planning. IEEE Trans Robot 25(6):1370–1381 CrossRefGoogle Scholar
  30. 30.
    Madani O, Hanks S, Condon A (2003) On the undecidability of probabilistic planning and related stochastic optimization problems. Artif Intell 147(1–2):5–34 MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Martin DA (1998) The determinacy of Blackwell games. J Symb Log 63(4):1565–1581 CrossRefMATHGoogle Scholar
  32. 32.
    McNaughton R (1993) Infinite games played on finite graphs. Ann Pure Appl Log 65:149–184 MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Paz A (1971) Introduction to probabilistic automata. Computer science and applied mathematics. Academic Press, San Diego MATHGoogle Scholar
  34. 34.
    Pnueli A, Rosner R (1989) On the synthesis of a reactive module. In: POPL. ACM, New York, pp 179–190 Google Scholar
  35. 35.
    Rabin MO (1963) Probabilistic automata. Inf Control 6:230–245 CrossRefMATHGoogle Scholar
  36. 36.
    Ramadge PJ, Wonham WM (1987) Supervisory control of a class of discrete-event processes. SIAM J Control Optim 25(1):206–230 MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Reif JH (1979) Universal games of incomplete information. In: STOC. ACM, New York, pp 288–308 Google Scholar
  38. 38.
    Thomas W (1997) Languages, automata, and logic. In: Beyond words. Handbook of formal languages, vol 3. Springer, Berlin, pp 389–455. Chap 7 Google Scholar
  39. 39.
    Vardi MY (1985) Automatic verification of probabilistic concurrent finite-state systems. In: FOCS. IEEE Comput Soc, Los Alamitos, pp 327–338 Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Krishnendu Chatterjee
    • 1
  • Laurent Doyen
    • 2
  • Thomas A. Henzinger
    • 1
  1. 1.IST Austria (Institute of Science and Technology Austria)KlosterneuburgAustria
  2. 2.LSVENS Cachan & CNRSCachanFrance

Personalised recommendations