Advertisement

Zero-Sum Stochastic Games

  • Anna Jaśkiewicz
  • Andrzej S. Nowak
Reference work entry

Abstract

In this chapter, we describe a major part of the theory of zero-sum discrete-time stochastic games. We review all basic streams of research in this area, in the context of the existence of value and uniform value, algorithms, vector payoffs, incomplete information, and imperfect state observation. Also some models related to continuous-time games, e.g., games with short-stage duration, semi-Markov games, are mentioned. Moreover, a number of applications of stochastic games are pointed out. The provided reference list reveals a tremendous progress made in the field of zero-sum stochastic games since the seminal work of Shapley (Proc Nat Acad Sci USA 39:1095–1100, 1953).

Keywords

Zero-sum game Stochastic game Borel space Unbounded payoffs Incomplete information Measurable strategy Maxmin optimization Limsup payoff Approachability Algorithms 

Notes

Acknowledgements

We thank Tamer Başar and Georges Zaccour for inviting us to write this chapter and their help. We also thank Eilon Solan, Sylvain Sorin, William Sudderth, and two reviewers for their comments on an earlier version of this survey.

References

  1. Aliprantis C, Border K (2006) Infinite dimensional analysis: a Hitchhiker’s guide. Springer, New YorkGoogle Scholar
  2. Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Meth Oper Res 62:375–386Google Scholar
  3. Altman E, Feinberg EA, Shwartz A (2000) Weighted discounted stochastic games with perfect information. Annals of the International Society of Dynamic Games, vol 5. Birkhäuser, Boston, pp 303–324Google Scholar
  4. Altman E, Gaitsgory VA (1995) A hybrid (differential-stochastic) zero-sum game with fast stochastic part. Annals of the International Society of Dynamic Games, vol 3. Birkhäuser, Boston, pp 47–59Google Scholar
  5. Altman E, Hordijk A (1995) Zero-sum Markov games and worst-case optimal control of queueing systems. Queueing Syst Theory Appl 21:415–447Google Scholar
  6. Altman E, Hordijk A, Spieksma FM (1997) Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards. Math Oper Res 22:588–618Google Scholar
  7. Arapostathis A, Borkar VS, Fernández-Gaucherand, Gosh MK, Markus SI (1993) Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J Control Optim 31:282–344Google Scholar
  8. Avrachenkov K, Cottatellucci L, Maggi L (2012) Algorithms for uniform optimal strategies in two-player zero-sum stochastic games with perfect information. Oper Res Lett 40:56–60Google Scholar
  9. Basu S (1999) New results on quantifier elimination over real-closed fields and applications to constraint databases. J ACM 46:537–555Google Scholar
  10. Basu S, Pollack R, Roy MF (2003) Algorithms in real algebraic geometry. Springer, New YorkGoogle Scholar
  11. Basu A, Stettner Ł (2015) Finite- and infinite-horizon Shapley games with non-symmetric partial observation. SIAM J Control Optim 53:3584–3619Google Scholar
  12. Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic, New YorkGoogle Scholar
  13. Berge C (1963) Topological spaces. MacMillan, New YorkGoogle Scholar
  14. Bertsekas DP, Shreve SE (1996) Stochastic Optimal Control: the Discrete-Time Case. Athena Scientic, BelmontGoogle Scholar
  15. Bewley T, Kohlberg E (1976a) The asymptotic theory of stochastic games. Math Oper Res 1:197–208Google Scholar
  16. Bewley T, Kohlberg E (1976b) The asymptotic solution of a recursion equation occurring in stochastic games. Math Oper Res 1:321–336Google Scholar
  17. Bewley T, Kohlberg E (1978) On stochastic games with stationary optimal strategies. Math Oper Res 3:104–125Google Scholar
  18. Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, CambridgeGoogle Scholar
  19. Billingsley P (1968) Convergence of probability measures. Wiley, New YorkGoogle Scholar
  20. Blackwell DA, Girshick MA (1954) Theory of games and statistical decisions. Wiley and Sons, New YorkGoogle Scholar
  21. Blackwell D (1956) An analog of the minmax theorem for vector payoffs. Pac J Math 6:1–8Google Scholar
  22. Blackwell D (1962) Discrete dynamic programming. Ann Math Statist 33:719–726Google Scholar
  23. Blackwell D (1965) Discounted dynamic programming. Ann Math Statist 36: 226–235Google Scholar
  24. Blackwell D (1969) Infinite Gδ-games with imperfect information. Zastosowania Matematyki (Appl Math) 10:99–101Google Scholar
  25. Blackwell D (1989) Operator solution of infinite Gδ-games of imperfect information. In: Anderson TW et al (eds) Probability, Statistics, and Mathematics: Papers in Honor of Samuel Karlin. Academic, New York, pp 83–87Google Scholar
  26. Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39:159–163Google Scholar
  27. Bolte J, Gaubert S, Vigeral G (2015) Definable zero-sum stochastic games. Math Oper Res 40: 80–104Google Scholar
  28. Boros E, Elbassioni K, Gurvich V, Makino K (2013) On canonical forms for zero-sum stochastic mean payoff games. Dyn Games Appl 3:128–161Google Scholar
  29. Boros E, Elbassioni K, Gurvich V, Makino K (2016) A potential reduction algorithm for two-person zero-sum mean payoff stochastic games. Dyn Games Appl doi:10.1007/s13235-016-0199-xGoogle Scholar
  30. Breton M (1991) Algorithms for stochastic games. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 45–58Google Scholar
  31. Brown LD, Purves R (1973) Measurable selections of extrema. Ann Stat 1:902–912Google Scholar
  32. Cardaliaguet P, Laraki R, Sorin S (2012) A continuous time approach for the asymptotic value in two-person zero-sum repeated games. SIAM J Control Optim 50:1573–1596Google Scholar
  33. Cardaliaguet P, Rainer C, Rosenberg D, Vieille N (2016) Markov games with frequent actions and incomplete information-The limit case. Math Oper Res 41:49–71Google Scholar
  34. Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, CambridgeGoogle Scholar
  35. Cesa-Bianchi N, Lugosi G, Stoltz G (2006) Regret minimization under partial monitoring. Math Oper Res 31:562–580Google Scholar
  36. Charnes A, Schroeder R (1967) On some tactical antisubmarine games. Naval Res Logistics Quarterly 14:291–311Google Scholar
  37. Chatterjee K, Majumdar R, Henzinger TA (2008) Stochastic limit-average games are in EXPTIME. Int J Game Theory 37:219–234Google Scholar
  38. Condon A (1992) The complexity of stochastic games. Inf Comput 96:203–224Google Scholar
  39. Coulomb JM (1992) Repeated games with absorbing states and no signals. Int J Game Theory 21:161–174Google Scholar
  40. Coulomb JM (1999) Generalized big-match. Math Oper Res 24:795–816Google Scholar
  41. Coulomb JM (2001) Absorbing games with a signalling structure. Math Oper Res 26:286–303Google Scholar
  42. Coulomb JM (2003a) Absorbing games with a signalling structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 335–355Google Scholar
  43. Coulomb JM (2003b) Games with a recursive structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 427–442Google Scholar
  44. Coulomb JM (2003c) Stochastic games with imperfect monitoring. Int J Game Theory (2003) 32:73–96Google Scholar
  45. Couwenbergh HAM (1980) Stochastic games with metric state spaces. Int J Game Theory 9:25–36Google Scholar
  46. de Alfaro L, Henzinger TA, Kupferman O (2007) Concurrent reachability games. Theoret Comp Sci 386:188–217Google Scholar
  47. Dubins LE (1957) A discrete invasion game. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of Mathematics Studies, vol 39. Princeton University Press, Princeton, pp 231–255Google Scholar
  48. Dubins LE, Maitra A, Purves R, Sudderth W (1989) Measurable, nonleavable gambling problems. Israel J Math 67:257–271Google Scholar
  49. Dubins LE, Savage LJ (2014) Inequalities for stochastic processes. Dover, New YorkGoogle Scholar
  50. Everett H (1957) Recursive games. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of Mathematics Studies, vol 39. Princeton University Press, Princeton, pp 47–78Google Scholar
  51. Fan K (1953) Minmax theorems. Proc Nat Acad Sci USA 39:42–47Google Scholar
  52. Feinberg EA (1994) Constrained semi-Markov decision processes with average rewards. Math Methods Oper Res 39:257–288Google Scholar
  53. Feinberg EA, Lewis ME (2005) Optimality of four-threshold policies in inventory systems with customer returns and borrowing/storage options. Probab Eng Inf Sci 19:45–71Google Scholar
  54. Filar JA (1981) Ordered field property for stochastic games when the player who controls transitions changes from state to state. J Optim Theory Appl 34:503–513Google Scholar
  55. Filar JA (1985). Player aggregation in the travelling inspector model. IEEE Trans Autom Control 30:723–729Google Scholar
  56. Filar JA, Schultz TA, Thuijsman F, Vrieze OJ (1991) Nonlinear programming and stationary equilibria of stochastic games. Math Program Ser A 50:227–237Google Scholar
  57. Filar JA, Tolwinski B (1991) On the algorithm of Pollatschek and Avi-Itzhak. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 59–70Google Scholar
  58. Filar JA, Vrieze, OJ (1992) Weighted reward criteria in competitive Markov decision processes. Z Oper Res 36:343–358Google Scholar
  59. Filar JA, Vrieze K (1997) Competitive Markov decision processes. Springer, New YorkGoogle Scholar
  60. Flesch J, Thuijsman F, Vrieze OJ (1999) Average-discounted equilibria in stochastic games. European J Oper Res 112:187–195Google Scholar
  61. Fristedt B, Lapic S, Sudderth WD (1995) The big match on the integers. Annals of the international society of dynamic games, vol 3. Birkhä user, Boston, pp 95–107Google Scholar
  62. Gale D, Steward EM (1953) Infinite games with perfect information. In: Kuhn H, Tucker AW (eds) Contributions to the theory of games II. Annals of mathematics studies, vol 28. Princeton University Press, Princeton, pp 241–266Google Scholar
  63. Gensbittel F (2016) Continuous-time limit of dynamic games with incomplete information and a more informed player. Int J Game Theory 45:321–352Google Scholar
  64. Gensbittel F, Oliu-Barton M, Venel X (2014) Existence of the uniform value in repeated games with a more informed controller. J Dyn Games 1:411–445Google Scholar
  65. Gensbittel F, Renault J (2015) The value of Markov chain games with lack of information on both sides. Math Oper Res 40:820–841Google Scholar
  66. Gillette D (1957) Stochastic games with zero stop probabilities.In: Dresher M et al (eds) Contributions to the theory of games III. Annals of mathematics studies, vol 39. Princeton University Press, Princeton, pp 179–187Google Scholar
  67. Ghosh MK, Bagchi A (1998) Stochastic games with average payoff criterion. Appl Math Optim 38:283–301Google Scholar
  68. Gimbert H, Renault J, Sorin S, Venel X, Zielonka W (2016) On values of repeated games with signals. Ann Appl Probab 26:402–424Google Scholar
  69. González-Trejo JI, Hernández-Lerma O, Hoyos-Reyes LF (2003) Minmax control of discrete-time stochastic systems. SIAM J Control Optim 41:1626–1659Google Scholar
  70. Guo X, Hernández-Lerma O (2003) Zero-sum games for continuous-time Markov chains with unbounded transitions and average payoff rates. J Appl Probab 40:327–345Google Scholar
  71. Guo X, Hernńdez-Lerma O (2005) Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J Appl Probab 42:303–320Google Scholar
  72. Hall P, Heyde C (1980) Martingale limit theory and its applications. Academic, New YorkGoogle Scholar
  73. Hansen LP, Sargent TJ (2008) Robustness. Princeton University Press, PrincetonGoogle Scholar
  74. Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, SingaporeGoogle Scholar
  75. Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer-Verlag, New YorkGoogle Scholar
  76. Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New YorkGoogle Scholar
  77. Himmelberg CJ (1975) Measurable relations. Fundam Math 87:53–72Google Scholar
  78. Himmelberg CJ, Van Vleck FS (1975) Multifunctions with values in a space of probability measures. J Math Anal Appl 50:108–112Google Scholar
  79. Hoffman AJ, Karp RM (1966) On non-terminating stochastic games. Management Sci 12:359–370Google Scholar
  80. Hordijk A, Kallenberg LCM (1981) Linear programming and Markov games I, II. In: Moeschlin O, Pallaschke D (eds) Game theory and mathematical economics, North-Holland, Amsterdam, pp 291–320Google Scholar
  81. Iyengar GN (2005) Robust dynamic programming. Math Oper Res 30:257–280Google Scholar
  82. Jaśkiewicz A (2002) Zero-sum semi-Markov games. SIAM J Control Optim 41:723–739Google Scholar
  83. Jaśkiewicz A (2004) On the equivalence of two expected average cost criteria for semi-Markov control processes. Math Oper Res 29:326–338Google Scholar
  84. Jaśkiewicz A (2007) Average optimality for risk-sensitive control with general state space. Ann Appl Probab 17: 654–675Google Scholar
  85. Jaśkiewicz A (2009) Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities. J Optim Theory Appl 141:321–347Google Scholar
  86. Jaśkiewicz A (2010) On a continuous solution to the Bellman-Poisson equation in stochastic games. J Optim Theory Appl 145:451–458Google Scholar
  87. Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45:773–789Google Scholar
  88. Jaśkiewicz A, Nowak AS (2011) Stochastic games with unbounded payoffs: Applications to robust control in economics. Dyn Games Appl 1: 253–279Google Scholar
  89. Jaśkiewicz A, Nowak AS (2014) Robust Markov control process. J Math Anal Appl 420:1337–1353Google Scholar
  90. Jaśkiewicz A, Nowak AS (2018) Non-zero-sum stochastic games. In: Başar T, Zaccour G (eds) Handbook of dynamic game theory. Birkhäuser, BaselGoogle Scholar
  91. Kalathil, D, Borkar VS, Jain R (2016) Approachability in Stackelberg stochastic games with vector costs. Dyn Games Appl. doi:10.1007/s13235–016-0198-yGoogle Scholar
  92. Kartashov NV(1996) Strong stable Markov chains. VSP, Utrecht, The NetherlandsGoogle Scholar
  93. Kehagias A, Mitschke D, Praat P (2013) Cops and invisible robbers: The cost of drunkenness. Theoret Comp Sci 481:100–120Google Scholar
  94. Kertz RP, Nachman D (1979) Persistently optimal plans for nonstationary dynamic programming: The topology of weak convergence case. Ann Probab 7:811–826Google Scholar
  95. Klein E, Thompson AC (1984) Theory of correspondences. Wiley, New YorkGoogle Scholar
  96. Kohlberg E (1974) Repeated games with absorbing states. Ann Statist 2:724–738Google Scholar
  97. Krass D, Filar JA, Sinha S (1992) A weighted Markov decision process. Oper Res 40:1180–1187Google Scholar
  98. Krausz A, Rieder U (1997) Markov games with incomplete information. Math Meth Oper Res 46:263–279Google Scholar
  99. Krishnamurthy N, Parthasarathy T (2011) Multistage (stochastic) games. Wiley encyclopedia of operations research and management science. Wiley online library. doi:10.1002/9780470400531.eorms0551Google Scholar
  100. Kumar PR, Shiau TH (1981) Existence of value and randomized strategies in zero-sum discrete-time stochastic dynamic games. SIAM J Control Optim 19:617–634Google Scholar
  101. Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Polish Acad Sci (Ser Math) 13:397–403Google Scholar
  102. Küenle HU (1986) Stochastische Spiele und Entscheidungsmodelle. BG Teubner, LeipzigGoogle Scholar
  103. Küenle HU (2007) On Markov games with average reward criterion and weakly continuous transition probabilities. SIAM J Control Optim 45:2156–2168Google Scholar
  104. Laraki R (2010) Explicit formulas for repeated games with absorbing states. Int J Game Theory 39:53–69Google Scholar
  105. Laraki R, Sorin S (2015) Advances in zero-sum dynamic games. In: Young HP, Zamir S (eds) Handbook of game theory with economic applications, vol 4. North Holland, pp 27–93Google Scholar
  106. Lehrer E, Solan E (2016) A general internal regret-free strategy. Dyn Games Appl 6:112–138Google Scholar
  107. Lehrer E, Sorin S (1992) A uniform Tauberian theorem in dynamic programming. Math Oper Res 17:303–307Google Scholar
  108. Levy Y (2012) Stochastic games with information lag. Games Econ Behavior 74:243–256Google Scholar
  109. Levy Y (2013) Continuous-time stochastic games of fixed duration. Dyn Games Appl 3:279–312Google Scholar
  110. Li X, Venel X (2016) Recursive games: uniform value, Tauberian theorem and the Mertens conjecture “\( Maxmin= \lim v_n=\lim v_\lambda \)”. Int J Game Theory 45:155–189Google Scholar
  111. Maccheroni F, Marinacci M, Rustichini A (2006) Dynamic variational preferences. J Econ Theory 128:4–44Google Scholar
  112. Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5:289–300Google Scholar
  113. Maitra A, Parthasarathy T (1971) On stochastic games II. J Optim Theory Appl 8:154–160Google Scholar
  114. Maitra A, Sudderth W (1992) An operator solution for stochastic games. Israel J Math 78:33–49Google Scholar
  115. Maitra A, Sudderth W (1993a) Borel stochastic games with limsup payoffs. Ann Probab 21:861–885Google Scholar
  116. Maitra A, Sudderth W (1993b) Finitely additive and measurable stochastic games. Int J Game Theory 22:201–223Google Scholar
  117. Maitra A, Sudderth W (1996) Discrete gambling and stochastic games. Springer, New YorkGoogle Scholar
  118. Maitra A, Sudderth W (1998) Finitely additive stochastic games with Borel measurable payoffs. Int J Game Theory 27:257–267Google Scholar
  119. Maitra A, Sudderth W (2003a) Stochastic games with limsup payoff. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 357–366Google Scholar
  120. Maitra A, Sudderth W (2003b) Stochastic games with Borel payoffs. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 367–373Google Scholar
  121. Martin D (1975) Borel determinacy. Ann Math 102:363–371Google Scholar
  122. Martin D (1985) A purely inductive proof of Borel determinacy. In: Nerode A, Shore RA (eds) Recursion theory. Proceedings of symposia in pure mathematics, vol 42. American Mathematical Society, Providence, pp 303–308Google Scholar
  123. Martin D (1998) The determinacy of Blackwell games. J Symb Logic 63:1565–1581Google Scholar
  124. Mertens JF (1982) Repeated games: an overview of the zero-sum case. In: Hildenbrand W (ed) Advances in economic theory. Cambridge Univ Press, Cambridge, pp 175–182Google Scholar
  125. Mertens JF (1987) Repeated games. In: Proceedings of the international congress of mathematicians, American mathematical society, Berkeley, California, pp 1528–1577Google Scholar
  126. Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. North Holland, pp 1809–1832Google Scholar
  127. Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–56MathSciNetCrossRefzbMATHGoogle Scholar
  128. Mertens JF, Neyman A (1982) Stochastic games have a value. Proc Natl Acad Sci USA 79: 2145–2146MathSciNetCrossRefzbMATHGoogle Scholar
  129. Mertens JF, Neyman A, Rosenberg D (2009) Absorbing games with compact action spaces. Math Oper Res 34:257–262MathSciNetCrossRefzbMATHGoogle Scholar
  130. Mertens JF, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, Cambridge, MACrossRefzbMATHGoogle Scholar
  131. Meyn SP, Tweedie RL (1994) Computable bounds for geometric convergence rates of Markov chains. Ann Appl Probab 4:981–1011MathSciNetCrossRefzbMATHGoogle Scholar
  132. Meyn SP, Tweedie RL (2009) Markov chains and stochastic stability. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  133. Miao J (2014) Economic dynamics in discrete time. MIT Press, CambridgezbMATHGoogle Scholar
  134. Milman E (2002) The semi-algebraic theory of stochastic games. Math Oper Res 27:401–418MathSciNetCrossRefzbMATHGoogle Scholar
  135. Milman E (2006) Approachable sets of vector payoffs in stochastic games. Games Econ Behavior 56:135–147MathSciNetCrossRefzbMATHGoogle Scholar
  136. Milnor J, Shapley LS (1957) On games of survival. In: Dresher M et al (eds) Contributions to the theory of games III. Annals of mathematics studies, vol 39. Princeton University Press, Princeton, pp 15–45zbMATHGoogle Scholar
  137. Mondal P, Sinha S, Neogy SK, Das AK (2016) On discounted ARAT semi-Markov games and its complementarity formulations. Int J Game Theory 45:567–583CrossRefzbMATHGoogle Scholar
  138. Mycielski J (1992) Games with perfect information. In: Aumann RJ, Hart S (eds) Handbook of Game Theory with Economic Applications, vol 1. North Holland, pp 41–70Google Scholar
  139. Neyman A (2003a) Real algebraic tools in stochastic games. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 57–75CrossRefzbMATHGoogle Scholar
  140. Neyman A (2003b) Stochastic games and nonexpansive maps. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 397–415CrossRefzbMATHGoogle Scholar
  141. Neyman A (2013) Stochastic games with short-stage duration. Dyn Games Appl 3:236–278MathSciNetCrossRefzbMATHGoogle Scholar
  142. Neyman A, Sorin S (eds) (2003) Stochastic games and applications. Kluwer, DordrechtzbMATHGoogle Scholar
  143. Neveu J (1965) Mathematical foundations of the calculus of probability. Holden-Day, San FranciscozbMATHGoogle Scholar
  144. Nowak AS (1985a) Universally measurable strategies in zero-sum stochastic games. Ann Probab 13: 269–287MathSciNetCrossRefzbMATHGoogle Scholar
  145. Nowak AS (1985b) Measurable selection theorems for minmax stochastic optimization problems, SIAM J Control Optim 23:466–476MathSciNetCrossRefzbMATHGoogle Scholar
  146. Nowak AS (1986) Semicontinuous nonstationary stochastic games. J Math Analysis Appl 117:84–99MathSciNetCrossRefzbMATHGoogle Scholar
  147. Nowak AS (1994) Zero-sum average payoff stochastic games wit general state space. Games Econ Behavior 7:221–232MathSciNetCrossRefzbMATHGoogle Scholar
  148. Nowak AS (2010) On measurable minmax selectors. J Math Anal Appl 366:385–388MathSciNetCrossRefzbMATHGoogle Scholar
  149. Nowak AS, Raghavan TES (1991) Positive stochastic games and a theorem of Ornstein. In: Stochastic games and related topics. Shapley honor volume. Kluwer, Dordrecht, pp 127–134CrossRefzbMATHGoogle Scholar
  150. Oliu-Barton M (2014) The asymptotic value in finite stochastic games. Math Oper Res 39:712–721MathSciNetCrossRefzbMATHGoogle Scholar
  151. Ornstein D (1969) On the existence of stationary optimal strategies. Proc Am Math Soc 20:563–569MathSciNetCrossRefzbMATHGoogle Scholar
  152. Papadimitriou CH (1994) Computational complexity. Addison-Wesley, ReadingzbMATHGoogle Scholar
  153. Parthasarathy KR (1967) Probability measures on metric spaces. Academic, New YorkCrossRefzbMATHGoogle Scholar
  154. Parthasarathy T, Raghavan TES (1981) An order field property for stochastic games when one player controls transition probabilities. J Optim Theory Appl 33:375–392MathSciNetCrossRefzbMATHGoogle Scholar
  155. Patek SD, Bertsekas DP (1999) Stochastic shortest path games. SIAM J Control Optim 37:804–824MathSciNetCrossRefzbMATHGoogle Scholar
  156. Perchet V (2011a) Approachability of convex sets in games with partial monitoring. J Optim Theory Appl 149:665–677MathSciNetCrossRefzbMATHGoogle Scholar
  157. Perchet V (2011b) Internal regret with partial monitoring calibration-based optimal algorithms. J Mach Learn Res 12:1893–1921MathSciNetzbMATHGoogle Scholar
  158. Pollatschek M, Avi-Itzhak B (1969) Algorithms for stochastic games with geometrical interpretation. Manag Sci 15:399–425MathSciNetCrossRefzbMATHGoogle Scholar
  159. Prikry K, Sudderth WD (2016) Measurability of the value of a parametrized game. Int J Game Theory 45:675–683MathSciNetCrossRefzbMATHGoogle Scholar
  160. Raghavan TES (2003) Finite-step algorithms for single-controller and perfect information stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer, Dordrecht, pp 227–251CrossRefzbMATHGoogle Scholar
  161. Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ, eds. (1991) Stochastic games and related topics: In honor of professor LS Shapley, Kluwer, DordrechtGoogle Scholar
  162. Raghavan TES, Filar JA (1991) Algorithms for stochastic games: a survey. Z Oper Res (Math Meth Oper Res) 35: 437–472MathSciNetCrossRefzbMATHGoogle Scholar
  163. Raghavan TES, Syed Z (2003) A policy improvement type algorithm for solving zero-sum two-person stochastic games of perfect information. Math Program Ser A 95:513–532MathSciNetCrossRefzbMATHGoogle Scholar
  164. Renault J (2006) The value of Markov chain games with lack of information on one side. Math Oper Res 31:490–512MathSciNetCrossRefzbMATHGoogle Scholar
  165. Renault J (2012) The value of repeated games with an uninformed controller. Math Oper Res 37:154–179MathSciNetCrossRefzbMATHGoogle Scholar
  166. Renault J (2014) General limit value in dynamic programming. J Dyn Games 1:471–484MathSciNetCrossRefzbMATHGoogle Scholar
  167. Rosenberg D (2000) Zero-sum absorbing games with incomplete information on one side: asymptotic analysis. SIAM J Control Optim 39:208–225MathSciNetCrossRefzbMATHGoogle Scholar
  168. Rosenberg D, Sorin S (2001) An operator approach to zero-sum repeated games. Israel J Math 121:221–246MathSciNetCrossRefzbMATHGoogle Scholar
  169. Rosenberg D, Solan E, Vieille N (2002) Blackwell optimality in Markov decision processes with partial observation. Ann Stat 30:1178–1193MathSciNetCrossRefzbMATHGoogle Scholar
  170. Rosenberg D, Solan E, Vieille N (2003) The maxmin value of stochastic games with imperfect monitoring. Int J Game Theory 32:133–150MathSciNetCrossRefzbMATHGoogle Scholar
  171. Rosenberg D, Solan E, Vieille N (2004) Stochastic games with a single controller and incomplete information. SIAM J Control Optim 43:86–110MathSciNetCrossRefzbMATHGoogle Scholar
  172. Rosenberg D, Vieille N (2000) The maxmin of recursive games with incomplete information on one side. Math Oper Res 25:23–35MathSciNetCrossRefzbMATHGoogle Scholar
  173. Scarf HE, Shapley LS (1957) A discrete invasion game. In: Dresher M et al (eds) Contributions to the Theory of Games III. Annals of Mathematics Studies, vol 39, Princeton University Press, Princeton, pp 213–229Google Scholar
  174. Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Z Wahrscheinlichkeitstheorie Verw Geb 32:179–196MathSciNetCrossRefzbMATHGoogle Scholar
  175. Secchi P (1997) Stationary strategies for recursive games. Math Oper Res 22:494–512MathSciNetCrossRefzbMATHGoogle Scholar
  176. Secchi P (1998) On the existence of good stationary strategies for nonleavable stochastic games. Int J Game Theory 27:61–81MathSciNetCrossRefzbMATHGoogle Scholar
  177. Selten R (1975) Re-examination of the perfectness concept for equilibrium points in extensive games. Int J Game Theory 4:25–55CrossRefzbMATHGoogle Scholar
  178. Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100MathSciNetCrossRefzbMATHGoogle Scholar
  179. Shimkin N, Shwartz A (1993) Guaranteed performance regions for Markovian systems with competing decision makers. IEEE Trans Autom Control 38:84–95MathSciNetCrossRefzbMATHGoogle Scholar
  180. Shmaya E (2011) The determinacy of infinite games with eventual perfect monitoring. Proc Am Math Soc 139:3665–3678MathSciNetCrossRefzbMATHGoogle Scholar
  181. Solan E (2009) Stochastic games. In: Meyers RA (ed) Encyclopedia of complexity and systems science. Springer, New York, pp 8698–8708CrossRefGoogle Scholar
  182. Solan E, Vieille N (2002) Uniform value in recursive games. Ann Appl Probab 12:1185–1201MathSciNetCrossRefzbMATHGoogle Scholar
  183. Solan E, Vieille N (2010) Camputing uniformly optimal strategies in two-player stochastic games. Econ Theory 42:237–253CrossRefzbMATHGoogle Scholar
  184. Solan E, Vieille N (2015) Stochastic games. Proc Nat Acad Sci USA 112:13743–13746MathSciNetCrossRefzbMATHGoogle Scholar
  185. Solan E, Ziliotto B (2016) Stochastic games with signals. Annals of the international society of dynamic game, vol 14. Birkhäuser, Boston, pp 77–94Google Scholar
  186. Sorin S (1984) Big match with lack of information on one side (Part 1). Int J Game Theory 13:201–255MathSciNetCrossRefzbMATHGoogle Scholar
  187. Sorin S (1985) Big match with lack of information on one side (Part 2). Int J Game Theory 14:173–204MathSciNetCrossRefzbMATHGoogle Scholar
  188. Sorin S (2002) A first course on zero-sum repeated games. Mathematiques et applications, vol 37. Springer, New YorkGoogle Scholar
  189. Sorin S (2003a) Stochastic games with incomplete information. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, Dordrecht, pp 375–395CrossRefzbMATHGoogle Scholar
  190. Sorin S (2003b) The operator approach to zero-sum stochastic games. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. Kluwer, DordrechtzbMATHGoogle Scholar
  191. Sorin S (2004) Asymptotic properties of monotonic nonexpansive mappings. Discrete Event Dyn Syst 14:109–122MathSciNetCrossRefzbMATHGoogle Scholar
  192. Sorin S, Vigeral G (2015a) Existence of the limit value of two-person zero-sum discounted repeated games via comparison theorems. J Optim Theory Appl 157:564–576MathSciNetCrossRefzbMATHGoogle Scholar
  193. Sorin S, Vigeral G (2015b) Reversibility and oscillations in zero-sum discounted stochastic games. J Dyn Games 2:103–115MathSciNetCrossRefzbMATHGoogle Scholar
  194. Sorin S, Vigeral G (2016) Operator approach to values of stochastic games with varying stage duration. Int J Game Theory 45:389–410MathSciNetCrossRefzbMATHGoogle Scholar
  195. Sorin S, Zamir S (1991) “Big match” with lack on information on one side (Part 3). In: Shapley LS, Raghavan TES (eds) Stochastic games and related topics. Shapley Honor Volume. Kluwer, Dordrecht, pp 101–112CrossRefGoogle Scholar
  196. Spinat X (2002) A necessary and sufficient condition for approachability. Math Oper Res 27:31–44MathSciNetCrossRefzbMATHGoogle Scholar
  197. Stokey NL, Lucas RE, Prescott E (1989) Recursive methods in economic dynamics. Harvard University Press, CambridgeGoogle Scholar
  198. Strauch R (1966) Negative dynamic programming. Ann Math Stat 37:871–890MathSciNetCrossRefzbMATHGoogle Scholar
  199. Szczechla W, Connell SA, Filar JA, Vrieze OJ (1997) On the Puiseux series expansion of the limit discount equation of stochastic games. SIAM J Control Optim 35:860–875MathSciNetCrossRefzbMATHGoogle Scholar
  200. Tarski A (1951) A decision method for elementary algebra and geometry. University of California Press, BerkeleyzbMATHGoogle Scholar
  201. Van der Wal I (1978) Discounted Markov games: Generalized policy iteration method. J Optim Theory Appl 25:125–138MathSciNetCrossRefzbMATHGoogle Scholar
  202. Vega-Amaya O (2003) Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J Control Optim 42:1876–1894MathSciNetCrossRefzbMATHGoogle Scholar
  203. Vega-Amaya O, Luque-Vásquez (2000) Sample path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times. Appl Math (Warsaw) 27:343–367Google Scholar
  204. Venel X (2015) Commutative stochastic games. Math Oper Res 40:403–428MathSciNetCrossRefzbMATHGoogle Scholar
  205. Vieille N (2002) Stochastic games: recent results. In: Aumann RJ, Hart S (eds) Handbook of Game Theory with Economic Applications, vol 3. North Holland, Amsterdam/London, pp 1833–1850Google Scholar
  206. Vigeral G (2013) A zero-sum stochastic game with compact action sets and no asymptotic value. Dyn Games Appl 3:172–186MathSciNetCrossRefzbMATHGoogle Scholar
  207. Vrieze OJ (1981) Linear programming and undiscounted stochastic games. Oper Res Spektrum 3:29–35CrossRefzbMATHGoogle Scholar
  208. Vrieze OJ (1987) Stochastic games with finite state and action spaces. Mathematisch Centrum Tract, vol 33. Centrum voor Wiskunde en Informatica, AmsterdamGoogle Scholar
  209. Vrieze OJ, Tijs SH, Raghavan TES, Filar JA (1983) A finite algorithm for switching control stochastic games. Oper Res Spektrum 5:15–24CrossRefzbMATHGoogle Scholar
  210. Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326–335MathSciNetCrossRefzbMATHGoogle Scholar
  211. Winston W (1978) A stochastic game model of a weapons development competition. SIAM J Control Optim 16:411–419MathSciNetCrossRefzbMATHGoogle Scholar
  212. Zachrisson LE (1964) Markov games. In: Dresher M, Shapley LS, Tucker AW (eds) Advances in Game Theory. Princeton University Press, Princeton, pp 211–253Google Scholar
  213. Ziliotto B (2016) General limit value in zero-sum stochastic games. Int J Game Theory 45:353–374MathSciNetCrossRefzbMATHGoogle Scholar
  214. Ziliotto B (2016) Zero-sum repeated games: counterexamples to the existence of the asymptotic value and the conjecture. Ann Probab 44:1107–1133MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Pure and Applied MathematicsWrocław University of Science and TechnologyWrocławPoland
  2. 2.Faculty of Mathematics, Computer Science and EconometricsUniversity of Zielona GóraZielona GóraPoland

Personalised recommendations