Skip to main content
SpringerLink
Log in

Discounted stochastic games for continuous-time jump processes with an uncountable state space

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

In this paper we study the nonzero-sum stochastic games for continuous-time jump processes under the expected discounted payoff criterion. The state space and action spaces for players are Borel spaces and the reward and transition rates are allowed to be unbounded. Under certain reasonable conditions, we first introduce new auxiliary static games and obtain the corresponding properties of these static games. Then by an approximation approach, we show the existence of a stationary almost Markov Nash equilibrium in the class of randomized history-dependent strategy profiles. Moreover, we use two examples to illustrate our main results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

References

  • Aliprantis C, Border K (2006) Infinite dimensional analysis. Springer, New York

    MATH  Google Scholar 

  • Artstein Z (1979) A note on Fatou’s lemma in several dimensions. J Math Econ 6:277–282

    Article  MathSciNet  Google Scholar 

  • Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discouted stochastic games. J Econ Theory 151:596–604

    Article  Google Scholar 

  • Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Berlin

    Book  Google Scholar 

  • Bertsekas DP, Shreve SE (1996) Stochastic optimal control: the discrete-time case. Athena Scientific, Belmont

    MATH  Google Scholar 

  • Castaing C, Valadier M (1977) Convex analysis and measurable multifunctions. Springer, New York

    Book  Google Scholar 

  • Duggan J (2012) Noisy stochastic games. Econometrica 80:2017–2045

    Article  MathSciNet  Google Scholar 

  • Filar J, Vrieze K (1997) Competitive Markov decision processes. Springer, New York

    MATH  Google Scholar 

  • Guo XP, Hernández-Lerma O (2005) Nonzero-sum games for continous-time Markov chains with unbounded discounted payoffs. J Appl Probab 42:303–320

    Article  MathSciNet  Google Scholar 

  • Guo XP, Hernández-Lerma O (2009) Continuous-time Markov decision processes: theory and applications. Springer, Berlin

    Book  Google Scholar 

  • Guo XP, Song XY (2011) Discounted continuous-time constrained Markov decision processes in Polish spaces. Ann Appl Probab 21:2016–2049

    Article  MathSciNet  Google Scholar 

  • Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York

    Book  Google Scholar 

  • Jaśkiewicz A, Nowak AS (2015) Stochastic games of resource extraction. Automatica J IFAC 54:310–316

    Article  MathSciNet  Google Scholar 

  • Jaśkiewicz A, Nowak AS (2016) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res 41:430–441

    Article  MathSciNet  Google Scholar 

  • Jaśkiewicz A, Nowak AS (2018) Non-zero-sum stochastic games. In: Basarand T, Zaccour G (eds) Handbook of dynamic game theory. Springer, Cham

    MATH  Google Scholar 

  • Jasso-Fuentes H, López-Barrientos JD, Escobedo-Trujillo BA (2017) Infinite-horizon non-zero-sum stochastic differential games with additive structure. IMA J Math Control Inf 34:283–309

    MathSciNet  MATH  Google Scholar 

  • Kitaev MY, Rykov VV (1995) Controlled queueing systems. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Kuratowski K, Ryll-Nardzewski C (1965) A general theorem on selectors. Bull Polish Acad Sci 13:397–403

    MathSciNet  MATH  Google Scholar 

  • Levy YJ (2013) Discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 81:1973–2007

    Article  MathSciNet  Google Scholar 

  • Levy YJ, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83:1237–1252

    Article  MathSciNet  Google Scholar 

  • Mertens JF (2003) A measurable “measurable choice” theorem. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer Academic Publishers, pp 107–130

  • Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int J Game Theory 32:121–132

    Article  MathSciNet  Google Scholar 

  • Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66:513–530

    Article  MathSciNet  Google Scholar 

  • Nowak AS, Raghavan TES (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17:519–526

    Article  MathSciNet  Google Scholar 

  • Piunovskiy A, Zhang Y (2011) Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach. SIAM J Control Optim 49:2032–2061

    Article  MathSciNet  Google Scholar 

  • Piunovskiy A, Zhang Y (2014) Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach, 4OR-Q. J Oper Res 12:49–75

    Article  Google Scholar 

  • Piunovskiy A, Zhang Y (2020) Continuous-time Markov decision processes: Borel space models and general control strategies. Springer, Cham

    Book  Google Scholar 

  • Wei QD, Chen X (2016) Stochastic games for continuous-time jump processes under finite-horizon payoff criterion. Appl Math Optim 74:273–301

    Article  MathSciNet  Google Scholar 

  • Zhang WZ (2018) Continuous-time constrained stochastic games under the discounted cost criteria. Appl Math Optim 77:275–296

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We are greatly indebted to the associate editor and the reviewers for the valuable comments and suggestions which have greatly improved the presentation. The research was supported by National Natural Science Foundation of China (Grant No. 12171170), Natural Science Foundation of Fujian Province (Grant No. 2021J01308) and the Opening Project of Guangdong Province Key Laboratory of Computational Science at Sun Yat-sen University (Grant No. 2021019).

Author information

Authors and Affiliations

  1. School of Economics and Finance, Huaqiao University, Quanzhou, 362021, People’s Republic of China

    Qingda Wei

  2. School of Mathematical Sciences, Xiamen University, Xiamen, 361005, People’s Republic of China

    Xian Chen

Authors
  1. Qingda Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xian Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Q., Chen, X. Discounted stochastic games for continuous-time jump processes with an uncountable state space. Math Meth Oper Res 95, 187–218 (2022). https://doi.org/10.1007/s00186-022-00778-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-022-00778-w

Keywords

Mathematics Subject Classification

Search

Navigation