Abstract
In this paper, we consider discrete-time \(N\)-person constrained stochastic games with discounted cost criteria. The state space is denumerable and the action space is a Borel set, while the cost functions are admitted to be unbounded from below and above. Under suitable conditions weaker than those in (Alvarez-Mena and Hernández-Lerma, Math Methods Oper Res 63:261–285, 2006) for bounded cost functions, we also show the existence of a Nash equilibrium for the constrained games by introducing two approximations. The first one, which is as in (Alvarez-Mena and Hernández-Lerma, Math Methods Oper Res 63:261–285, 2006), is to construct a sequence of finite games to approximate a (constrained) auxiliary game with an initial distribution that is concentrated on a finite set. However, without hypotheses of bounded costs as in (Alvarez-Mena and Hernández-Lerma, Math Methods Oper Res 63:261–285, 2006), we also establish the existence of a Nash equilibrium for the auxiliary game with unbounded costs by developing more shaper error bounds of the approximation. The second one, which is new, is to construct a sequence of the auxiliary-type games above and prove that the limit of the sequence of Nash equilibria for the auxiliary-type games is a Nash equilibrium for the original constrained games. Our results are illustrated by a controlled queueing system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman E (1999) Constrained Markov decision processes. Chapman Hall & CRC, Florida
Altman E, Shwartz A (2000) Constrained Markov games: Nash equilibria. Ann Int Soc Dynam Games, vol 5. Birkhäuser, Boston, pp 213–221
Altman E, Avrachenkov K, Bonneau N, Debbah M, EI-Azouzi R, Sadoc Menasche D (2008) Constrained cost-coupled stochastic games with independent state processes. Oper Res Lett 36:160–164
Alvarez-Mena J, Hernández-Lerma O (2002) Convergence of the optimal values of constrained Markov control processes. Math Methods Oper Res 55:461–484
Alvarez-Mena J, Hernández-Lerma O (2006) Existence of Nash equilibria for constrained stochastic games. Math Methods Oper Res 63:261–285
Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Heidelberg
Cavazos-Cadena R (1986) Finite-state approximations for denumerable state discounted Markov decision processes. Appl Math Optim 14:1–26
Durán J (2000) On dynamic programming with unbounded returns. Econom Theory 15:339–352
Federgruen A (1978) On \({N}\)-person stochastic games with denumerable state space. Adv Appl Probab 10:452–471
Filar J, Vrieze K (1997) Competitive Markov decision processes. Springer, New York
Fink AM (1964) Equilibrium in a stochastic \(n\)-person game. J Sci Hiroshima Univ Ser A-I Math 28:89–93
González-Trejo JI, Hernández-Lerma O, Hoyos-Reyes LF (2002) Minimax control of discrete-time stochastic systems. SIAM J Control Optim 41:1626–1659
Guo XP, Hernández-Lerma O (2009) Continuous-time Markov decision processes. Springer, Berlin
Guo XP, Yang J (2008) A new condition and approach for zero-sum stochastic games with average payoffs. Stoch Anal Appl 26:537–561
Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes. Springer, New York
Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York
Küenle HU (1994) On Nash equilibrium solutions in nonzero-sum stochastic games with complete information. Internat J Game Theory 23:303–324
Maitra AP, Sudderth WD (1996) Discrete gambling and stochastic games. Springer, New York
Nowak AS (1999) Sensitive equilibria for ergodic stochastic games with countable state spaces. Math Methods Oper Res 50:65–76
Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Internat J Game Theory 32:121–132
Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66:513–530
Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York
Sennott LI (1994) Nonzero-sum stochastic games with unbounded costs: discounted and average cost cases. Z Oper Res 40:145–162
Sennott LI (1999) Stochastic dynamic programming and the control of queueing systems. Wiley, New York
Sobel MJ (1971) Noncooperative stochastic games. Ann Math Statist 42:1930–1935
Acknowledgments
This research was partially supported by the NSFC and GDUPS. We also thank the anonymous referee for constructive comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang , W., Huang , Y. & Guo, X. Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs. TOP 22, 1074–1102 (2014). https://doi.org/10.1007/s11750-013-0313-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11750-013-0313-9