Abstract
We examine so-called product-games. These are n-player stochastic games played on a product state space S 1 × ... × S n, in which player i controls the transitions on S i. For the general n-player case, we establish the existence of 0-equilibria. In addition, for the case of two-player zero-sum games of this type, we show that both players have stationary 0-optimal strategies. In the analysis of product-games, interestingly, a central role is played by the periodic features of the transition structure. Flesch et al. (Math Oper Res 33, 403–420, 2008) showed the existence of 0-equilibria under the assumption that, for every player i, the transition structure on S i is aperiodic. In this article, we examine product-games with periodic transition structures. Even though a large part of the approach in Flesch et al. (Math Oper Res 33, 403–420, 2008) remains applicable, we encounter a number of tricky problems that we have to address. We provide illustrative examples to clarify the essence of the difference between the aperiodic and periodic cases.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Methods Oper Res 62: 375–386
Altman E, Avrachenkov K, Bonneau N, Debbah M, El-Azouzi R, Sadoc Menasche D (2007a) Constrained cost-coupled stochastic games with independent state processes (preprint)
Altman E, Avrachenkov K, Garnaev A (2007b) A jamming game in wireless networks with transmission cost. Lect Notes Comp Sci 4465: 1–12
Becker R, Zilberstein S, Lesser V, Goldman CV (2003) Transition-independent decentralized Markov decision processes. In: Proceedings of the second international joint conference on autonomous agents and multi agent systems, ACM Press, Melbourne, pp 41–48
Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39: 159–163
Dutta PK (1995) A Folk theorem for stochastic games. J Econ Theory 66: 1–32
Filar JA, Vrieze OJ (1996) Competitive Markov decision processes—theory, algorithms, and applications. Springer, New York
Fink AM (1964) Equilibrium in a stochastic n-person game. J Sci Hiroshima Univ Ser A–I 28: 89–93
Flesch J, Thuijsman F, Vrieze OJ (1997) Cyclic Markov equilibria in a cubic game. Int J Game Theory 26: 303–314
Flesch J, Thuijsman F, Vrieze OJ (2007) Stochastic games with additive transitions. Eur J Oper Res 179: 483–497
Flesch J, Schoenmakers G, Vrieze OJ (2008a) Stochastic games on a product state space. Math Oper Res 33: 403–420
Flesch J, Schoenmakers G, Vrieze OJ (2008b) Stochastic games on a product state space: the periodic case. Research memorandum RM/08/016. University of Maastricht, The Netherlands
Gillette D (1957) Stochastic games with zero stop probabilities. In: Dresher M, Tucker AW, Wolfe P (eds) Contributions to the theory of games III. Annals of Mathematical Studies 39. Princeton University Press, Princeton, pp 179–187
Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10: 53–66
Neyman A (2003) Existence of the minmax. In: Neyman A, Sorin S (eds) Stochastic games and applications, NATO Science series vol, 570. Kluwer, Dordrecht, pp 173–193
Parthasarathy T, Raghavan TES (1981) An orderfield property for stochastic games when one player controls transition probabilities. J Optim Theory Appl 33: 375–392
Ross KW, Varadarajan R (1991) Multichain Markov decision processes with a sample path constraint: a decomposition approach. Math Oper Res 16: 195–207
Simon R (2003) Value and perfection in stochastic games. Working paper, Goettingen University, Germany. Israel J Math (to appear)
Solan E (1999) Three-Player absorbing games. Math Oper Res 24: 669–698
Solan E (2003) Perturbations of Markov chains with applications to stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications, NATO science series, vol 570. Kluwer, Dordrecht, pp 265–280
Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behav 38: 362–399
Solan E, Vohra R (2002) Correlated equilibrium and public signalling in absorbing games. Int J Game Theory 31: 91–121
Sorin S (1986) Asymptotic properties of a non-zerosum game. Int J Game Theory 15: 101–107
Takahashi M (1964) Equilibrium points of stochastic noncooperative n-person games. J Sci Hiroshima Univ Ser A–I 28: 95–99
Thuijsman F, Vrieze OJ (1991) Easy initial states in stochastic games. In: Raghavan TES, Ferguson TS, Vrieze OJ, Parthasarathy T (eds) Stochastic games and related topics. Kluwer, Dordrecht, pp 85–100
Vieille N (2000a) Equilibrium in 2-person stochastic games I: a reduction. Israel J Math 119: 55–91
Vieille N (2000b) Equilibrium in 2-person stochastic games II: the case of recursive games. Israel J Math 119: 93–126
Acknowledgments
We would like to thank Rakesh V. Vohra and two anonymous referees for their suggestions, which considerably improved the presentation of the article.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Flesch, J., Schoenmakers, G. & Vrieze, K. Stochastic games on a product state space: the periodic case. Int J Game Theory 38, 263–289 (2009). https://doi.org/10.1007/s00182-009-0153-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00182-009-0153-x