Recursive Concurrent Stochastic Games

  • Kousha Etessami
  • Mihalis Yannakakis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4052)


We study Recursive Concurrent Stochastic Games (RCSGs), extending our recent analysis of recursive simple stochastic games [14, 15] to a concurrent setting where the two players choose moves simultaneously and independently at each state. For multi-exit games, our earlier work already showed undecidability for basic questions like termination, thus we focus on the important case of single-exit RCSGs (1-RCSGs).

We first characterize the value of a 1-RCSG termination game as the least fixed point solution of a system of nonlinear minimax functional equations, and use it to show PSPACE decidability for the quantitative termination problem. We then give a strategy improvement technique, which we use to show that player 1 (maximizer) has ε-optimal randomized Stackless & Memoryless (r-SM) strategies, while player 2 (minimizer) has optimal r-SM strategies. Thus, such games are r-SM-determined. These results mirror and generalize in a strong sense the randomized memoryless determinacy results for finite stochastic games, and extend the classic Hoffman-Karp [19] strategy improvement approach from the finite to an infinite state setting. The proofs in our infinite-state setting are very different however.

We show that our upper bounds, even for qualitative termination, can not be improved without a major breakthrough, by giving two reductions: first a P-time reduction from the long-standing square-root sum problem to the quantitative termination decision problem for finite concurrent stochastic games, and then a P-time reduction from the latter problem to the qualitative termination problem for 1-RCSGs.


Strategy Improvement Stochastic Game Matrix Game Outgoing Transition Exit Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allender, E., Bürgisser, P., Kjeldgaard-Pedersen, J., Miltersen, P.B.: On the complexity of numerical analysis. In: 21st IEEE Computational Complexity Conference (2006)Google Scholar
  2. 2.
    Bewley, T., Kohlberg, E.: The asymptotic theory of stochastic games. Math. Oper. Res. 1(3), 197–208 (1976)MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Brázdil, T., Kučera, A., Stražovský, O.: Decidability of temporal properties of probabilistic pushdown automata. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Canny, J.: Some algebraic and geometric computations in PSPACE. In: Proc. of 20th ACM STOC, pp. 460–467 (1988)Google Scholar
  5. 5.
    Chatterjee, K.: Personal communicationGoogle Scholar
  6. 6.
    Chatterjee, K., de Alfaro, L., Henzinger, T.: The complexity of quantitative concurrent parity games. In: Proc. of SODA (2006)Google Scholar
  7. 7.
    Chatterjee, K., Majumdar, R., Jurdzinski, M.: On Nash equilibria in stochastic games. In: Marcinkowski, J., Tarlecki, A. (eds.) CSL 2004. LNCS, vol. 3210, pp. 26–40. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Condon, A.: The complexity of stochastic games. Inf. & Comp. 96(2), 203–224 (1992)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    de Alfaro, L., Henzinger, T.A., Kupferman, O.: Concurrent reachability games. In: Proc. of FOCS 1998, pp. 564–575 (1998)Google Scholar
  10. 10.
    de Alfaro, L., Majumdar, R.: Quantitative solution of omega-regular games. J. Comput. Syst. Sci. 68(2), 374–397 (2004)MATHCrossRefGoogle Scholar
  11. 11.
    Esparza, J., Kučera, A., Mayr, R.: Model checking probabilistic pushdown automata. In: Proc. of 19th IEEE LICS (2004)Google Scholar
  12. 12.
    Etessami, K., Yannakakis, M.: Recursive markov chains, stochastic grammars, and monotone systems of nonlinear equations. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Etessami, K., Yannakakis, M.: Algorithmic Verification of Recursive Probabilistic State Machines. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 253–270. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Etessami, K., Yannakakis, M.: Recursive Markov Decision Processes and Recursive Stochastic Games. In: Caires, L., Italiano, G.F., Monteiro, L., Palamidessi, C., Yung, M. (eds.) ICALP 2005. LNCS, vol. 3580, pp. 891–903. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Etessami, K., Yannakakis, M.: Efficient Qualitative Analysis of Classes of Recursive Markov Decision Processes and Simple Stochastic Games. In: Durand, B., Thomas, W. (eds.) STACS 2006. LNCS, vol. 3884, pp. 634–645. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)MATHGoogle Scholar
  17. 17.
    Garey, M.R., Graham, R.L., Johnson, D.S.: Some NP-complete geometric problems. In: 8th ACM STOC, pp. 10–22 (1976)Google Scholar
  18. 18.
    Harris, T.E.: The Theory of Branching Processes. Springer, Heidelberg (1963)MATHGoogle Scholar
  19. 19.
    Hoffman, A.J., Karp, R.M.: On nonterminating stochastic games. Management Sci 12, 359–370 (1966)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Jagers, P.: Branching Processes with Biological Applications. Wiley, Chichester (1975)MATHGoogle Scholar
  21. 21.
    Maitra, A., Sudderth, W.: Finitely additive stochastic games with Borel measurable payoffs. Internat. J. Game Theory 27(2), 257–267 (1998)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Martin, D.A.: Determinacy of Blackwell games. J. Symb. Logic 63(4), 1565–1581 (1998)MATHCrossRefGoogle Scholar
  23. 23.
    Renegar, J.: On the computational complexity and geometry of the first-order theory of the reals, parts I-III. J. Symb. Comp. 13(3), 255–352 (1992)MATHMathSciNetCrossRefGoogle Scholar
  24. 24.
    Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Sci. 39, 1095–1100 (1953)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Kousha Etessami
    • 1
  • Mihalis Yannakakis
    • 2
  1. 1.LFCS, School of InformaticsUniversity of Edinburgh 
  2. 2.Department of Computer ScienceColumbia University 

Personalised recommendations