Skip to main content
SpringerLink
Log in

Constrained Expected Average Stochastic Games for Continuous-Time Jump Processes

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

In this paper we study the nonzero-sum constrained stochastic games for continuous-time jump processes with denumerable states and possibly unbounded transition rates. The optimality criterion under consideration is the expected average payoff criterion and the payoff functions of the players are allowed to be unbounded. Under the reasonable conditions, we introduce an approximating sequence of the auxiliary game models and show the existence of stationary constrained Nash equilibria for these approximating game models via employing the average occupation measures and constructing a suitable multifunction. Moreover, we obtain that any limit point of the stationary constrained Nash equilibria for the approximating sequence of the game models is a constrained Nash equilibrium for the original game model. Furthermore, we use a controlled birth and death system to illustrate our main results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aliprantis, C., Border, K.: Infinite Dimensional Analysis. Springer, New York (2006)

    MATH  Google Scholar 

  2. Altman, E., Shwartz, A.: Constrained Markov Games: Nash Equilibria. Ann Int Soc Dynam Games, vol. 5, pp. 213–221. Birkhäuser, Boston (2000)

    MATH  Google Scholar 

  3. Alvarez-Mena, J., Hernández-Lerma, O.: Convergence of the optimal values of constrained Markov control processes. Math. Methods Oper. Res. 55, 461–484 (2002)

    Article  MathSciNet  Google Scholar 

  4. Alvarez-Mena, J., Hernández-Lerma, O.: Existence of Nash equilibria for constrained stochastic games. Math. Methods Oper. Res. 63, 261–285 (2006)

    Article  MathSciNet  Google Scholar 

  5. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific, Belmont (1996)

    MATH  Google Scholar 

  6. Föllmer, H., Schied, A.: Stochastic Finance: An Introduction in Discrete Time. Walter de Gruyter, Berlin (2004)

    Book  Google Scholar 

  7. Guo, X.P., Hernández-Lerma, O.: Continuous-Time Markov Decision Processes: Theory and Applications. Springer, Berlin (2009)

    Book  Google Scholar 

  8. Guo, X.P., Song, X.Y.: Discounted continuous-time constrained Markov decision processes in Polish spaces. Ann. Appl. Probab. 21, 2016–2049 (2011)

    Article  MathSciNet  Google Scholar 

  9. Guo, X.P., Huang, Y.H., Song, X.Y.: Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies. SIAM J. Control Optim. 50, 23–47 (2012)

    Article  MathSciNet  Google Scholar 

  10. Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)

    Book  Google Scholar 

  11. Kitaev, M.Y., Rykov, V.V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)

    MATH  Google Scholar 

  12. Piunovskiy, A., Zhang, Y.: Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach. SIAM J. Control Optim. 49, 2032–2061 (2011)

    Article  MathSciNet  Google Scholar 

  13. Prieto-Rumeau, T., Hernández-Lerma, O.: Ergodic control of continuous-time Markov chains with pathwise constraints. SIAM J. Control Optim. 47, 1888–1908 (2008)

    Article  MathSciNet  Google Scholar 

  14. Wei, Q.D., Chen, X.: Strong average optimality criterion for continuous-time Markov decision processes. Kybernetika 50, 950–977 (2014)

    MathSciNet  MATH  Google Scholar 

  15. Wei, Q.D., Chen, X.: Constrained stochastic games with the average payoff criteria. Oper. Res. Lett. 43, 83–88 (2015)

    Article  MathSciNet  Google Scholar 

  16. Wei, Q.D., Chen, X.: Stochastic games for continuous-time jump processes under finite-horizon payoff criterion. Appl. Math. Optim. 74, 273–301 (2016)

    Article  MathSciNet  Google Scholar 

  17. Zhang, W.Z.: Continuous-time constrained stochastic games under the discounted cost criteria. Appl. Math. Optim. 77, 275–296 (2018)

    Article  MathSciNet  Google Scholar 

  18. Zhang, W.Z., Guo, X.P.: Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. Sci. China Math. 55, 2405–2416 (2012)

    Article  MathSciNet  Google Scholar 

  19. Zhang, W.Z., Huang, Y.H., Guo, X.P.: Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs. TOP 22, 1074–1102 (2014)

    Article  MathSciNet  Google Scholar 

  20. Zhang, W.Z., Wang, B.F., Chen, D.W.: Continuous-time constrained stochastic games with average criteria. Oper. Res. Lett. 46, 109–115 (2018)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

I am greatly indebted to the referees for the valuable comments and suggestions which have greatly improved the presentation. The research was supported by National Natural Science Foundation of China (Grant No. 11601166).

Author information

Authors and Affiliations

  1. School of Economics and Finance, Huaqiao University, Quanzhou, 362021, People’s Republic of China

    Qingda Wei

Authors
  1. Qingda Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingda Wei.

Ethics declarations

Conflict of interest

I declare that no conflict of interest exists in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Q. Constrained Expected Average Stochastic Games for Continuous-Time Jump Processes. Appl Math Optim 83, 1277–1309 (2021). https://doi.org/10.1007/s00245-019-09588-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-019-09588-9

Keywords

Mathematics Subject Classification

Search

Navigation