Abstract
Two standard algorithms for approximately solving two-player zero-sum concurrent reachability games are value iteration and strategy iteration. We prove upper and lower bounds of \(2^{m^{\varTheta(N)}}\) on the worst case number of iterations needed by both of these algorithms for providing non-trivial approximations to the value of a game with N non-terminal positions and m actions for each player in each position. In particular, both algorithms have doubly-exponential complexity. Even when the game given as input has only one non-terminal position, we prove an exponential lower bound on the worst case number of iterations needed to provide non-trivial approximations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Including the TRAP position in the setup is actually not strictly needed, as one could replace it with any non-terminal position from which no escape is possible, but including it is quite convenient and fairly standard. In particular, including it makes “a reachability game with one non-terminal position” mean what we think it should.
In this paper, we assume the real number model of computation and ignore the (severe) technical issues arising when implementing the algorithm using finite-precision arithmetic.
References
de Alfaro, L., Henzinger, T.A., Kupferman, O.: Concurrent reachability games. Theor. Comput. Sci. 386(3), 188–217 (2007). doi:10.1016/j.tcs.2007.07.008
Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Strategy improvement for concurrent reachability games. In: Third International Conference on the Quantitative Evaluation of Systems. QEST’06, pp. 291–300. IEEE Computer Society, New York (2006)
Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Termination criteria for solving concurrent safety and reachability games. In: Proceedings of the Twenteeth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA’09) (2009)
Chatterjee, K., Majumdar, R., Jurdziński, M.: On Nash equilibria in stochastic games. In: Marcinkowski, J., Tarlecki, A. (eds.) CSL 2004. LNCS, vol. 3210, pp. 26–40. Springer, Berlin (2004)
Condon, A.: On algorithms for simple stochastic games. In: Advances in Computational Complexity Theory. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 13, pp. 51–73 (1993)
Dai, D., Ge, R.: New results on simple stochastic games. In: Algorithms and Computation 20th International Symposium, ISAAC Proceedings 2009, Honolulu, Hawaii, USA, December 16–18, 2009. Lecture Notes in Computer Science, vol. 5878, pp. 1014–1023. Springer, Berlin (2009)
Etessami, K., Yannakakis, M.: Recursive concurrent stochastic games. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP (2). Lecture Notes in Computer Science, vol. 4052, pp. 324–335. Springer, Berlin (2006)
Everett, H.: Recursive games. In: Kuhn, H.W., Tucker, A.W. (eds.) Contributions to the Theory of Games Vol. III. Annals of Mathematical Studies, vol. 39. Princeton University Press, Princeton (1957)
Friedmann, O.: An exponential lower bound for the parity game strategy improvement algorithm as we know it. In: Proceedings of the 24th Annual IEEE Symposium on Logic in Computer Science, LICS 2009, 11–14 August 2009 Los Angeles, CA, USA, pp. 145–156 (2009)
Hansen, K.A., Ibsen-Jensen, R., Miltersen, P.B.: The complexity of solving reachability games using value and strategy iteration. In: Kulikov, A.S., Vereshchagin, N.K. (eds.) Computer Science—Theory and Applications, Proceedings CSR 2011, St. Petersburg, Russia, June 14–18, 2011. Lecture Notes in Computer Science, vol. 6651, pp. 77–90. Springer, Berlin (2011)
Hansen, K.A., Koucký, M., Lauritzen, N., Miltersen, P.B., Tsigaridas, E.P.: Exact algorithms for solving discounted stochastic games and recursive games. In: STOC’11, pp. 205–214. (2011)
Hansen, K.A., Koucky, M., Miltersen, P.B.: Winning concurrent reachability games requires doubly exponential patience. In: 24th Annual IEEE Symposium on Logic in Computer Science (LICS’09), pp. 332–341. IEEE, New York (2009)
Himmelberg, C.J., Parthasarathy, T., Raghavan, T.E.S., Vleck, F.S.V.: Existence of p-equilibrium and optimal stationary strategies in stochastic games. Proc. Am. Math. Soc. 60, 245–251 (1976)
Hoffman, A., Karp, R.: On nonterminating stochastic games. Manag. Sci. 359–370 (1966)
Howard, R.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960)
Mertens, J.F., Neyman, A.: Stochastic games. Int. J. Game Theory 10, 53–66 (1981)
Parthasarathy, T.: Discounted and positive stochastic games. Bull. Am. Math. Soc. 77, 134–136 (1971)
Rao, S., Chandrasekaran, R., Nair, K.: Algorithms for discounted games. J. Optim. Theory Appl. 627–637 (1973)
Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. USA 39, 1095–1100 (1953)
Acknowledgements
First and foremost, we would like to thank Uri Zwick for extremely helpful discussions and Kousha Etessami for being instrumental for starting this research. We would also like to thank Vladimir V. Podolskii for helpful discussions. A preliminary version of this paper [10] appeared in the proceeings of CSR’11.
Author information
Authors and Affiliations
Corresponding author
Additional information
Work supported by Center for Algorithmic Game Theory, funded by the Carlsberg Foundation. The authors acknowledge support from The Danish National Research Foundation and The National Science Foundation of China (under the grant 61061130540) for the Sino-Danish Center for the Theory of Interactive Computation, under which part of this work was performed. A preliminary version of this paper appeared in the proceedings of CSR’11.
Rights and permissions
About this article
Cite this article
Hansen, K.A., Ibsen-Jensen, R. & Miltersen, P.B. The Complexity of Solving Reachability Games Using Value and Strategy Iteration. Theory Comput Syst 55, 380–403 (2014). https://doi.org/10.1007/s00224-013-9524-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00224-013-9524-6