Skip to main content
SpringerLink
Log in

A Potential Reduction Algorithm for Two-Person Zero-Sum Mean Payoff Stochastic Games

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

We suggest a new algorithm for two-person zero-sum undiscounted stochastic games focusing on stationary strategies. Given a positive real \(\varepsilon \), let us call a stochastic game \(\varepsilon \)-ergodic, if its values from any two initial positions differ by at most \(\varepsilon \). The proposed new algorithm outputs for every \(\varepsilon >0\) in finite time either a pair of stationary strategies for the two players guaranteeing that the values from any initial positions are within an \(\varepsilon \)-range, or identifies two initial positions u and v and corresponding stationary strategies for the players proving that the game values starting from u and v are at least \(\varepsilon /24\) apart. In particular, the above result shows that if a stochastic game is \(\varepsilon \)-ergodic, then there are stationary strategies for the players proving \(24\varepsilon \)-ergodicity. This result strengthens and provides a constructive version of an existential result by Vrieze (Stochastic games with finite state and action spaces. PhD thesis, Centrum voor Wiskunde en Informatica, Amsterdam, 1980) claiming that if a stochastic game is 0-ergodic, then there are \(\varepsilon \)-optimal stationary strategies for every \(\varepsilon > 0\). The suggested algorithm is based on a potential transformation technique that changes the range of local values at all positions without changing the normal form of the game.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Andersson D, Miltersen PB (2009) The complexity of solving stochastic games on graphs. In: Proceedings of the 20th ISAAC, LNCS, vol 5878, pp 112–121

  2. Basu S, Pollack R, Roy M (1996) On the combinatorial and algebraic complexity of quantifier elimination. J ACM 43(6):1002–1045 (Preliminary version in FOCS 1994)

    Article  MathSciNet  MATH  Google Scholar 

  3. Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39(1):159–163

    Article  MathSciNet  MATH  Google Scholar 

  4. Boros E, Elbassioni K, Gurvich V, Makino K (2013) On canonical forms for zero-sum stochastic mean payoff games. Dyn Games Appl 3(2):128–161

    Article  MathSciNet  MATH  Google Scholar 

  5. Boros E, Elbassioni K, Gurvich V, Makino K (2013) On discounted approximations of undiscounted stochastic games and markov decision processes with limited randomness. Oper Res Lett 41(4):357–362

    Article  MathSciNet  MATH  Google Scholar 

  6. Boros E, Elbassioni K, Gurvich V, Makino K (2010) A pumping algorithm for ergodic stochastic mean payoff games with perfect information. In: Proceedings of the 14th IPCO, LNCS, vol 6080. Springer, pp 341–354

  7. Boros E, Elbassioni K, Gurvich V, Makino K (2013) A pseudo-polynomial algorithm for mean payoff stochastic games with perfect information and a few random positions. In: Fomin FV, Freivalds R, Kwiatkowska M, Peleg D (eds) Automata, languages, and programming, LNCS, vol 7965. Springer, Berlin Heidelberg, pp 220–231

  8. Boros E, Elbassioni K, Gurvich V, Makino K (2014) A potential reduction algorithm for ergodic two-person zero-sum limiting average payoff stochastic games. In: Zhang Z, Wu L, Xu W, Du DZ (eds) Combinatorial optimization and applications, LNCS, vol 8881. Springer, pp 694–709

  9. Chatterjee K, Majumdar R, Henzinger TA (2008) Stochastic limit-average games are in exptime. Int J Game Theory 37:219–234

    Article  MathSciNet  MATH  Google Scholar 

  10. Chatterjee K, Ibsen-Jensen R (2014) The complexity of ergodic mean-payoff games. In: Proceedings of the 41st ICALP: 2, LNCS, vol 8573, pp 122–133

  11. Federgruen A (1980) Successive approximation methods in undiscounted stochastic games. Oper Res 1:794–810

    Article  MathSciNet  MATH  Google Scholar 

  12. Gallai T (1958) Maximum–minimum Sätze über Graphen. Acta Math Acad Sci Hung 9:395–434

    Article  MATH  Google Scholar 

  13. Gillette D (1957) Stochastic games with zero stop probabilities. In: Dresher M, Tucker AW, Wolfe P (eds) Contribution to the theory of games III. Annals of mathematical studies, vol 39. Princeton University Press, Princeton, pp 179–187

    Google Scholar 

  14. Grigoriev D, Vorobjov N (1988) Solving systems of polynomial inequalities in subexponential time. J Symb Comput 5(1/2):37–64

    Article  MathSciNet  MATH  Google Scholar 

  15. Hansen KA, Koucky M, Lauritzen N, Miltersen PB, Tsigaridas EP (2011) Exact algorithms for solving stochastic games: extended abstract. In: Proceedings of the 43rd annual ACM symposium on theory of computing, STOC ’11. 2011. ACM, New York, pp 205–214

  16. Hoffman AJ, Karp RM (1966) On nonterminating stochastic games. Manag Sci A 12(5):359–370

    MathSciNet  MATH  Google Scholar 

  17. Kemeny JG, Snell JL (1963) Finite Markov chains. Springer, New York

    MATH  Google Scholar 

  18. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–66

    Article  MathSciNet  MATH  Google Scholar 

  19. Miltersen PB (2011) Discounted stochastic games poorly approximate undiscounted ones, manuscript. Technical report

  20. Mine H, Osaki S (1970) Markovian decision process. American Elsevier Publishing Co., New York

    MATH  Google Scholar 

  21. Moulin H (1976) Prolongement des jeux à deux joueurs de somme nulle. Bull Soc Math Fr Mem 45:5–111

    MATH  Google Scholar 

  22. Raghavan TES, Filar JA (1991) Algorithms for stochastic games: a survey. Math Methods Oper Res 35(6):437–472

    Article  MathSciNet  MATH  Google Scholar 

  23. Renegar J (1992) On the computational complexity and geometry of the first-order theory of the reals. J Symb Comput 13(3):255–352

    Article  MathSciNet  MATH  Google Scholar 

  24. Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  25. Vrieze OJ (1980) Stochastic games with finite state and action spaces. PhD thesis, Centrum voor Wiskunde en Informatica, Amsterdam

  26. Zwick U, Paterson M (1996) The complexity of mean payoff games on graphs. Theor Comput Sci 158(1–2):343–359

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MSIS Department of RBS and RUTCOR, Rutgers University, 100 Rockafeller Road, Piscataway, NJ, 08854-8054, USA

    Endre Boros & Vladimir Gurvich

  2. Masdar Institute of Science and Technology, B.O. Box 54224, Abu Dhabi, UAE

    Khaled Elbassioni

  3. Research Institute, Higher School of Economics (HSE), Moscow, Russia

    Vladimir Gurvich

  4. Research Institute for Mathematical Sciences (RIMS), Kyoto University, Kyoto, 606-8502, Japan

    Kazuhisa Makino

Authors
  1. Endre Boros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khaled Elbassioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Gurvich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuhisa Makino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Khaled Elbassioni.

Additional information

This research was partially supported by DIMACS, Center for Discrete Mathematics and Theoretical Computer Science, Rutgers University, and by the Scientific Grant-in-Aid from Ministry of Education, Science, Sports and Culture of Japan. Part of this research was done at the Mathematisches Forschungsinstitut Oberwolfach during stays within the Research in Pairs Program. The first author also acknowledges the partial support of NSF Grant IIS-1161476. The third author was partially funded by the Russian Academic Excellence Project ‘5-100’.

An extended abstract of this paper was published in the proceedings of the Combinatorial Optimization and Applications, 2014 [8].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boros, E., Elbassioni, K., Gurvich, V. et al. A Potential Reduction Algorithm for Two-Person Zero-Sum Mean Payoff Stochastic Games. Dyn Games Appl 8, 22–41 (2018). https://doi.org/10.1007/s13235-016-0199-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-016-0199-x

Keywords

Search

Navigation