Abstract
In the paper we consider the controlled continuous-time Markov chain describing the interacting particles system with the finite number of types. The system is controlled by two players with the opposite purposes. This Markov game converges to a zero-sum differential game when the number of particles tends to infinity. Krasovskii–Subbotin extremal shift provides the optimal strategy in the limiting game. The main result of the paper is the near optimality of the Krasovskii–Subbotin extremal shift rule for the original Markov game.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Averboukh Y (2015) Universal Nash equilibrium strategies for differential games. J Dyn Control Syst 21:329–350
Bardi M, Capuzzo Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, Basel
Benaïm M, Le Boudec J-Y (2008) A class of mean field interaction models for computer and communication systems. Perform Eval 65:823–838
Darling RWR, Norris JR (2010) Differential equation approximations for Markov chains. Probab Surv 5:37–79
Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions. Springer, New York
Gast N, Gaujal B, Le Boudec J-Y (2010) Mean field for Markov decision processes: from discrete to continuous optimization. INRIA report no. 7239
Kolokoltsov VN (2010) Nonlinear Markov process and kinetic equations. Cambridge University Press, Cambridge
Kolokoltsov VN (2013) Nonlinear Markov games on a finite state space (mean-field and binary interactions). Int J Stat Probab 1:77–91
Krasovskii NN, Kotelnikova AN (2009) Unification of differential games, generalized solutions of the Hamilton–Jacobi equations, and a stochastic guide. Differ Equ 45:1653–1668
Krasovskii NN, Kotelnikova AN (2010) An approach-evasion differential game: stochastic guide. Proc Steklov Inst Math 269:S191–S213
Krasovskii NN, Kotelnikova AN (2010) On a differential interception game. Proc Steklov Inst Math 268:161–206
Krasovskii NN, Kotelnikova AN (2012) Stochastic guide for a time-delay object in a positional differential game. Proc Steklov Inst Math 277:S145–S151
Krasovskii NN, Subbotin AI (1988) Game-theoretical control problems. Springer, New York
Kriazhimskii AV (1978) On stable position control in differential games. J Appl Math Mech 42(6):1055–1060
Levy Y (2013) Continuous-time stochastic games of fixed duration. Dyn Games Appl 3:279–312
Lukoyanov NYu, Plaksin AR (2013) Finite-dimensional modeling guides in time-delay systems. Trudy Inst Mat i Mekh UrO RAN 19:182–195 (in Russian)
Neyman A (2012) Continuous-time stochastic games. DP #616. Center for the Study of Rationality, Hebrew University, Jerusalem
Subbotin AI (1995) Generalized solutions of first-order PDEs. The dynamical perspective. Birkhaüser, Boston
Subbotin AI, Chentsov AG (1981) Optimization of guarantee in control problems. Nauka, Moscow (in Russian)
Zachrisson LE (1964) Markov games. In: Dresher M, Shapley LS, Tucker AW (eds) Advances in game theory. Princeton University Press, Princeton, pp 211–253
Acknowledgments
The author would like to thank Vassili Kolokoltsov for insightful discussions and the anonymous reviewer for the valuable comments. The research was supported by Russian Foundation for Basic Research (Project N15-01-07909).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Averboukh, Y. Extremal Shift Rule for Continuous-Time Zero-Sum Markov Games. Dyn Games Appl 7, 1–20 (2017). https://doi.org/10.1007/s13235-015-0173-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-015-0173-z