Theory of Computing Systems

, Volume 55, Issue 2, pp 380–403 | Cite as

The Complexity of Solving Reachability Games Using Value and Strategy Iteration

  • Kristoffer Arnsfelt Hansen
  • Rasmus Ibsen-Jensen
  • Peter Bro Miltersen
Article
First Online:

Abstract

Two standard algorithms for approximately solving two-player zero-sum concurrent reachability games are value iteration and strategy iteration. We prove upper and lower bounds of \(2^{m^{\varTheta(N)}}\) on the worst case number of iterations needed by both of these algorithms for providing non-trivial approximations to the value of a game with N non-terminal positions and m actions for each player in each position. In particular, both algorithms have doubly-exponential complexity. Even when the game given as input has only one non-terminal position, we prove an exponential lower bound on the worst case number of iterations needed to provide non-trivial approximations.

Keywords

Concurrent reachability games Value iteration Strategy iteration Analysis of algorithms 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

First and foremost, we would like to thank Uri Zwick for extremely helpful discussions and Kousha Etessami for being instrumental for starting this research. We would also like to thank Vladimir V. Podolskii for helpful discussions. A preliminary version of this paper [10] appeared in the proceeings of CSR’11.

References

  1. 1.
    de Alfaro, L., Henzinger, T.A., Kupferman, O.: Concurrent reachability games. Theor. Comput. Sci. 386(3), 188–217 (2007). doi: 10.1016/j.tcs.2007.07.008 CrossRefMATHGoogle Scholar
  2. 2.
    Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Strategy improvement for concurrent reachability games. In: Third International Conference on the Quantitative Evaluation of Systems. QEST’06, pp. 291–300. IEEE Computer Society, New York (2006) Google Scholar
  3. 3.
    Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Termination criteria for solving concurrent safety and reachability games. In: Proceedings of the Twenteeth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA’09) (2009) Google Scholar
  4. 4.
    Chatterjee, K., Majumdar, R., Jurdziński, M.: On Nash equilibria in stochastic games. In: Marcinkowski, J., Tarlecki, A. (eds.) CSL 2004. LNCS, vol. 3210, pp. 26–40. Springer, Berlin (2004) Google Scholar
  5. 5.
    Condon, A.: On algorithms for simple stochastic games. In: Advances in Computational Complexity Theory. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 13, pp. 51–73 (1993) Google Scholar
  6. 6.
    Dai, D., Ge, R.: New results on simple stochastic games. In: Algorithms and Computation 20th International Symposium, ISAAC Proceedings 2009, Honolulu, Hawaii, USA, December 16–18, 2009. Lecture Notes in Computer Science, vol. 5878, pp. 1014–1023. Springer, Berlin (2009) CrossRefGoogle Scholar
  7. 7.
    Etessami, K., Yannakakis, M.: Recursive concurrent stochastic games. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP (2). Lecture Notes in Computer Science, vol. 4052, pp. 324–335. Springer, Berlin (2006) Google Scholar
  8. 8.
    Everett, H.: Recursive games. In: Kuhn, H.W., Tucker, A.W. (eds.) Contributions to the Theory of Games Vol. III. Annals of Mathematical Studies, vol. 39. Princeton University Press, Princeton (1957) Google Scholar
  9. 9.
    Friedmann, O.: An exponential lower bound for the parity game strategy improvement algorithm as we know it. In: Proceedings of the 24th Annual IEEE Symposium on Logic in Computer Science, LICS 2009, 11–14 August 2009 Los Angeles, CA, USA, pp. 145–156 (2009) CrossRefGoogle Scholar
  10. 10.
    Hansen, K.A., Ibsen-Jensen, R., Miltersen, P.B.: The complexity of solving reachability games using value and strategy iteration. In: Kulikov, A.S., Vereshchagin, N.K. (eds.) Computer Science—Theory and Applications, Proceedings CSR 2011, St. Petersburg, Russia, June 14–18, 2011. Lecture Notes in Computer Science, vol. 6651, pp. 77–90. Springer, Berlin (2011) Google Scholar
  11. 11.
    Hansen, K.A., Koucký, M., Lauritzen, N., Miltersen, P.B., Tsigaridas, E.P.: Exact algorithms for solving discounted stochastic games and recursive games. In: STOC’11, pp. 205–214. (2011) Google Scholar
  12. 12.
    Hansen, K.A., Koucky, M., Miltersen, P.B.: Winning concurrent reachability games requires doubly exponential patience. In: 24th Annual IEEE Symposium on Logic in Computer Science (LICS’09), pp. 332–341. IEEE, New York (2009) CrossRefGoogle Scholar
  13. 13.
    Himmelberg, C.J., Parthasarathy, T., Raghavan, T.E.S., Vleck, F.S.V.: Existence of p-equilibrium and optimal stationary strategies in stochastic games. Proc. Am. Math. Soc. 60, 245–251 (1976) Google Scholar
  14. 14.
    Hoffman, A., Karp, R.: On nonterminating stochastic games. Manag. Sci. 359–370 (1966) Google Scholar
  15. 15.
    Howard, R.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960) MATHGoogle Scholar
  16. 16.
    Mertens, J.F., Neyman, A.: Stochastic games. Int. J. Game Theory 10, 53–66 (1981) CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Parthasarathy, T.: Discounted and positive stochastic games. Bull. Am. Math. Soc. 77, 134–136 (1971) CrossRefMATHGoogle Scholar
  18. 18.
    Rao, S., Chandrasekaran, R., Nair, K.: Algorithms for discounted games. J. Optim. Theory Appl. 627–637 (1973) Google Scholar
  19. 19.
    Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. USA 39, 1095–1100 (1953) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Kristoffer Arnsfelt Hansen
    • 1
  • Rasmus Ibsen-Jensen
    • 1
  • Peter Bro Miltersen
    • 1
  1. 1.Department of Computer ScienceAarhus UniversityAarhus NDenmark

Personalised recommendations