Solving Simple Stochastic Games with Few Coin Toss Positions

  • Rasmus Ibsen-Jensen
  • Peter Bro Miltersen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7501)


Gimbert and Horn gave an algorithm for solving simple stochastic games with running time O(r! n) where n is the number of positions of the simple stochastic game and r is the number of its coin toss positions. Chatterjee et al. pointed out that a variant of strategy iteration can be implemented to solve this problem in time 4 r n O(1). In this paper, we show that an algorithm combining value iteration with retrograde analysis achieves a time bound of O(r 2 r (r logr + n)), thus improving both time bounds. We also improve the analysis of Chatterjee et al. and show that their algorithm in fact has complexity 2 r n O(1).


Strategy Iteration Stochastic Game Goal Position Positional Strategy Danish National Research Foundation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andersson, D., Hansen, K.A., Miltersen, P.B., Sørensen, T.B.: Deterministic Graphical Games Revisited. In: Beckmann, A., Dimitracopoulos, C., Löwe, B. (eds.) CiE 2008. LNCS, vol. 5028, pp. 1–10. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Andersson, D., Miltersen, P.B.: The Complexity of Solving Stochastic Games on Graphs. In: Dong, Y., Du, D.-Z., Ibarra, O. (eds.) ISAAC 2009. LNCS, vol. 5878, pp. 112–121. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Bellman, R.E.: On the application of dynamic programming to the determination of optimal play in chess and checkers. Procedings of the National Academy of Sciences of the United States of America 53, 244–246 (1965)zbMATHCrossRefGoogle Scholar
  4. 4.
    Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Termination criteria for solving concurrent safety and reachability games. In: Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2009, pp. 197–206 (2009)Google Scholar
  5. 5.
    Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Strategy improvement for concurrent reachability games. In: Third International Conference on the Quantitative Evaluation of Systems. QEST 2006, pp. 291–300. IEEE Computer Society (2006)Google Scholar
  6. 6.
    Condon, A.: The complexity of stochastic games. Information and Computation 96, 203–224 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Condon, A.: On algorithms for simple stochastic games. In: Advances in Computational Complexity Theory. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 13, pp. 51–73. American Mathematical Society (1993)Google Scholar
  8. 8.
    Dai, D., Ge, R.: New Results on Simple Stochastic Games. In: Dong, Y., Du, D.-Z., Ibarra, O. (eds.) ISAAC 2009. LNCS, vol. 5878, pp. 1014–1023. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Gillette, D.: Stochastic games with zero stop probabilities. In: Dresher, M., Tucker, A.W., Wolfe, P. (eds.) Contributions to the Theory of Games III. Annals of Mathematics Studies, vol. 39, pp. 179–187. Princeton University Press (1957)Google Scholar
  10. 10.
    Gimbert, H., Horn, F.: Simple Stochastic Games with Few Random Vertices Are Easy to Solve. In: Amadio, R.M. (ed.) FoSSaCS 2008. LNCS, vol. 4962, pp. 5–19. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Ibsen-Jensen, R., Miltersen, P.B.: Solving simple stochastic games with few coin toss positions,
  12. 12.
    Kwek, S., Mehlhorn, K.: Optimal search for rationals. Inf. Process. Lett. 86(1), 23–26 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Liggett, T.M., Lippman, S.A.: Stochastic games with perfect information and time average payoff. SIAM Review 11(4), 604–607 (1969)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Mertens, J.F., Neyman, A.: Stochastic games. International Journal of Game Theory 10, 53–66 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Moon, J., Moser, L.: On cliques in graphs. Israel Journal of Mathematics 3, 23–28 (1965), doi:10.1007/BF02760024MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Science 39, 1095–1100 (1953)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Rasmus Ibsen-Jensen
    • 1
  • Peter Bro Miltersen
    • 1
  1. 1.Department of Computer ScienceAarhus UniversityDenmark

Personalised recommendations