Abstract
In this paper, we consider the continuous-time nonzero-sum constrained stochastic games with the discounted cost criteria. The state space is denumerable and the action space of each player is a general Polish space, while the transition rates and cost functions are allowed to be unbounded from below and from above. The strategies for each player may be history-dependent and randomized. Models with these features seemingly have not been handled in the previous literature. By constructing a sequence of continuous-time finite-state game models to approximate the original denumerable-state game model, we prove the existence of constrained Nash equilibria for the constrained games with denumerable states.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
By introducing the notation \(\mathcal {G}_\infty \), we can use the notation \(\mathcal {G}_n(n\in \overline{\mathbb {N}})\) to denote the finite-state game or the original game to make the presentation concise.
To be absolutely rigorous, one should write \(\widehat{P}_{\varvec{\varphi }^{-k},\overline{\gamma }_n}^{\varphi ^k}\) as \(\widehat{P}_{\varvec{\varphi }^{-k},\overline{\gamma }_n,n}^{\varphi ^k}\) to emphasize the transition laws \(\overline{Q}^{\varvec{\varphi }^{-k}}_{n}\) and initial distribution \(\overline{\gamma }_n\) to construct \(\widehat{P}_{\varvec{\varphi }^{-k},\overline{\gamma }_n}^{\varphi ^k}\) is dependent on \(\varvec{\varphi }^{-k}\) and n. Nevertheless, we omit the subscript n for brevity. Moreover, it is obvious that \(\widehat{P}_{\varvec{\varphi }^{-k},\overline{\gamma }_n}^{\varphi ^k}\) is the marginal distribution of \(\overline{P}_{\overline{\gamma }_n,n}^{\varvec{\varphi }}\) on \(\overline{\Omega }^k_n\).
References
Altman, E., Shwartz, A.: Constrained Markov games: Nash equilibria. In: Ann Int Soc Dynam Games, vol. 5, pp. 213–221. Birkhäuser, Boston (2000)
Alvarez-Mena, J., Hernández-Lerma, O.: Existence of Nash equilibria for constrained stochastic games. Math. Methods Oper. Res. 63(2), 261–285 (2006)
Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, New York (1997)
Guo, X.P., Hernández-Lerma, O.: Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J. Appl. Probab. 42(2), 303–320 (2005)
Guo, X.P., Piunovskiy, A.B.: Discounted continuous-time Markov decision processes with constraints: unbounded transition and loss rates. Math. Oper. Res. 36(1), 105–132 (2011)
Guo, X.P., Zhang, W.Z.: Convergence of controlled models and finite-state approximation for discounted continuous-time Markov decision processes with constraints. Eur. J. Oper. Res. 238(2), 486–496 (2014)
Hernández-Lerma, O., Lasserre, J.B.: Further topics on discrete-time Markov control processes. Springer, New York (1999)
Kitaev, M.Y., Rykov, V.V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)
Kitaev, M.Y.: Semi-Markov and jump Markov controlled models: average cost criterion. Theory Probab. Appl. 30(2), 272–288 (1986)
Piunovskiy, A., Zhang, Y.: Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case. Math. Methods Oper. Res. 73(2), 159–187 (2011)
Prieto-Rumeau, T., Lorenzo, J.M.: Approximation of zero-sum continuous-time Markov games under the discounted payoff criterion. TOP 23(3), 799–836 (2015)
Wei, Q.D., Chen, X.: Constrained stochastic games with the average payoff criteria. Oper. Res. Lett. 43(1), 83–88 (2015)
Wei, Q.D., Chen, X.: Stochastic games for continuous-time jump processes under finite-horizon payoff criterion. Appl. Math. Optim. (2015). doi:10.1007/s00245-015-9314-4
Zhang, W.Z., Guo, X.P.: Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. Sci. China Math. 55(11), 2405–2416 (2012)
Zhang, W.Z., Huang, Y.H., Guo, X.P.: Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs. TOP 22(3), 1074–1102 (2014)
Acknowledgments
I am greatly indebted to the associate editor and the anonymous referees for the constructive comments. This work was partially supported by National Natural Science Foundation of China (Grant No. 11526054) and Natural Science Foundation of Fujian Province of China (Grant No. 2016J05006).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, W. Continuous-Time Constrained Stochastic Games under the Discounted Cost Criteria. Appl Math Optim 77, 275–296 (2018). https://doi.org/10.1007/s00245-016-9374-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-016-9374-0