Skip to main content
SpringerLink
Log in

A Pseudo-Polynomial Algorithm for Mean Payoff Stochastic Games with Perfect Information and a Few Random Positions

  • Conference paper
Automata, Languages, and Programming (ICALP 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7965))

Included in the following conference series:

Abstract

We consider two-person zero-sum stochastic mean payoff games with perfect information, or BWR-games, given by a digraph G = (V, E), with local rewards r: E → ℝ, and three types of vertices: black V B , white V W , and random V R forming a partition of V. It is a long-standing open question whether a polynomial time algorithm for BWR-games exists, or not. In fact, a pseudo-polynomial algorithm for these games would already imply their polynomial solvability. In this paper, we show that BWR-games with a constant number of random nodes can be solved in pseudo-polynomial time. That is, for any such game with a few random nodes |V R | = O(1), a saddle point in pure stationary strategies can be found in time polynomial in |V W | + |V B |, the maximum absolute local reward R, and the common denominator of the transition probabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andersson, D., Miltersen, P.B.: The complexity of solving stochastic games on graphs. In: Dong, Y., Du, D.-Z., Ibarra, O. (eds.) ISAAC 2009. LNCS, vol. 5878, pp. 112–121. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  2. Beffara, E., Vorobyov, S.: Adapting gurvich-karzanov-khachiyan’s algorithm for parity games: Implementation and experimentation. Technical Report 2001-020, Department of Information Technology, Uppsala University (2001), https://www.it.uu.se/research/reports/#2001

  3. Beffara, E., Vorobyov, S.: Is randomized gurvich-karzanov-khachiyan’s algorithm for parity games polynomial? Technical Report 2001-025, Department of Information Technology, Uppsala University (2001), https://www.it.uu.se/research/reports/#2001

  4. Björklund, H., Vorobyov, S.: Combinatorial structure and randomized sub-exponential algorithm for infinite games. Theoretical Computer Science 349(3), 347–360 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  5. Björklund, H., Vorobyov, S.: A combinatorial strongly sub-exponential strategy improvement algorithm for mean payoff games. Discrete Applied Mathematics 155(2), 210–229 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  6. Boros, E., Elbassioni, K., Fouz, M., Gurvich, V., Makino, K., Manthey, B.: Stochastic mean payoff games: Smoothed analysis and approximation schemes. In: Aceto, L., Henzinger, M., Sgall, J. (eds.) ICALP 2011, Part I. LNCS, vol. 6755, pp. 147–158. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  7. Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: A pumping algorithm for ergodic stochastic mean payoff games with perfect information. In: Eisenbrand, F., Shepherd, F.B. (eds.) IPCO 2010. LNCS, vol. 6080, pp. 341–354. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: Discounted approximations of undiscounted stochastic games and markov decision processes are already poor in the almost deterministic case. Operations Research Letters (to appear, 2013)

    Google Scholar 

  9. Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: On canocical forms for zero-sum stochastic mean payoff games. In: Dynamic Games and Applications (2013), doi:10.1007/s13235-013-0075-x; Special volume dedicated to the 60th anniversary of Shapley’s 1953 paper on stochastic games

    Google Scholar 

  10. Chatterjee, K., Henzinger, T.A.: Reduction of stochastic parity to stochastic mean-payoff games. Inf. Process. Lett. 106(1), 1–7 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Quantitative stochastic parity games. In: SODA 2004, pp. 121–130. Society for Industrial and Applied Mathematics, Philadelphia (2004)

    Google Scholar 

  12. Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Termination criteria for solving concurrent safety and reachability games. In: SODA, pp. 197–206 (2009)

    Google Scholar 

  13. Condon, A.: The complexity of stochastic games. Information and Computation 96, 203–224 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  14. Condon, A.: An algorithm for simple stochastic games. In: Advances in Computational Complexity Theory. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 13 (1993)

    Google Scholar 

  15. Dai, D., Ge, R.: Another sub-exponential algorithm for the simple stochastic game. Algorithmica 61, 1092–1104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Eherenfeucht, A., Mycielski, J.: Positional strategies for mean payoff games. International Journal of Game Theory 8, 109–113 (1979)

    Article  MathSciNet  Google Scholar 

  17. Gillette, D.: Stochastic games with zero stop probabilities. In: Tucker, A.W., Dresher, M., Wolfe, P. (eds.) Contribution to the Theory of Games III. Annals of Mathematics Studies, vol. 39, pp. 179–187. Princeton University Press (1957)

    Google Scholar 

  18. Gimbert, H., Horn, F.: Simple stochastic games with few random vertices are easy to solve. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 5–19. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  19. Gurvich, V., Karzanov, A., Khachiyan, L.: Cyclic games and an algorithm to find minimax cycle means in directed graphs. USSR Computational Mathematics and Mathematical Physics 28, 85–91 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  20. Sandberg, S., Björklund, H., Vorobyov, S.: A combinatorial strongly sub-exponential strategy improvement algorithm for mean payoff games. DIMACS Technical Report 2004-05, DIMACS, Rutgers University (2004)

    Google Scholar 

  21. Halman, N.: Simple stochastic games, parity games, mean payoff games and discounted payoff games are all LP-type problems. Algorithmica 49(1), 37–50 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Ibsen-Jensen, R., Miltersen, P.B.: Solving simple stochastic games with few coin toss positions. In: Epstein, L., Ferragina, P. (eds.) ESA 2012. LNCS, vol. 7501, pp. 636–647. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  23. Jurdziński, M.: Deciding the winner in parity games is in UP ∩ co-UP. Inf. Process. Lett. 68(3), 119–124 (1998)

    Article  Google Scholar 

  24. Jurdziński, M.: Games for Verification: Algorithmic Issues. PhD thesis, Department of Computer Science, University of Aarhus, Denmark (2000)

    Google Scholar 

  25. Jurdziński, M., Paterson, M., Zwick, U.: A deterministic subexponential algorithm for solving parity games. In: SODA 2006, pp. 117–123. ACM, New York (2006)

    Google Scholar 

  26. Karp, R.M.: A characterization of the minimum cycle mean in a digraph. Discrete Math. 23, 309–311 (1978)

    MathSciNet  MATH  Google Scholar 

  27. Karzanov, A.V., Lebedev, V.N.: Cyclical games with prohibition. Mathematical Programming 60, 277–293 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  28. Liggett, T.M., Lippman, S.A.: Stochastic games with perfect information and time-average payoff. SIAM Review 4, 604–607 (1969)

    Article  MathSciNet  Google Scholar 

  29. Littman, M.L.: Algorithm for sequential decision making, CS-96-09. PhD thesis, Dept. of Computer Science, Brown Univ., USA (1996)

    Google Scholar 

  30. Mine, H., Osaki, S.: Markovian decision process. American Elsevier Publishing Co., New York (1970)

    Google Scholar 

  31. Moulin, H.: Extension of two person zero sum games. Journal of Mathematical Analysis and Application 5(2), 490–507 (1976)

    Article  MathSciNet  Google Scholar 

  32. Pisaruk, N.N.: Mean cost cyclical games. Mathematics of Operations Research 24(4), 817–828 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  33. Raghavan, T.E.S., Filar, J.A.: Algorithms for stochastic games- a survey. Mathematical Methods of Operations Research 35(6), 437–472 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  34. Vorobyov, S.: Cyclic games and linear programming. Discrete Applied Mathematics, Special volume in Memory of Leonid Khachiyan (1952 - 2005) 156(11), 2195–2231 (2008)

    MathSciNet  MATH  Google Scholar 

  35. Zwick, U., Paterson, M.: The complexity of mean payoff games on graphs. Theoretical Computer Science 158(1-2), 343–359 (1996)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RUTCOR, Rutgers University, 640 Bartholomew Road, Piscataway, NJ, USA, 08854-8003

    Endre Boros & Vladimir Gurvich

  2. Masdar Institute of Science and Technology, Abu Dhabi, UAE

    Khaled Elbassioni

  3. Research Institute for Mathematical Sciences (RIMS), Kyoto University, Kyoto, 606-8502, Japan

    Kazuhisa Makino

Authors
  1. Endre Boros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khaled Elbassioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Gurvich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuhisa Makino
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Bergen, Postboks 7803, 5020, Bergen, Norway

    Fedor V. Fomin

  2. Faculty of Computing, University of Latvia, Raina bulv. 19, 1586, Riga, Latvia

    Rūsiņš Freivalds

  3. Department of Computer Science, University of Oxford, Wolfson Building, Parks Road, OX1 3QD, Oxford, UK

    Marta Kwiatkowska

  4. Faculty of Mathematics and Computer Science, Weizmann Institute of Science, POB 26, 76100, Rehovot, Israel

    David Peleg

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Boros, E., Elbassioni, K., Gurvich, V., Makino, K. (2013). A Pseudo-Polynomial Algorithm for Mean Payoff Stochastic Games with Perfect Information and a Few Random Positions. In: Fomin, F.V., Freivalds, R., Kwiatkowska, M., Peleg, D. (eds) Automata, Languages, and Programming. ICALP 2013. Lecture Notes in Computer Science, vol 7965. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39206-1_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39206-1_19

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39205-4

  • Online ISBN: 978-3-642-39206-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation