Abstract
We introduce asymptotic analysis of stochastic games with short-stage duration. The play of stage k, k≥0, of a stochastic game Γ δ with stage duration δ is interpreted as the play in time kδ≤t<(k+1)δ and, therefore, the average payoff of the n-stage play per unit of time is the sum of the payoffs in the first n stages divided by nδ, and the λ-discounted present value of a payoff g in stage k is λ kδ g. We define convergence, strong convergence, and exact convergence of the data of a family (Γ δ ) δ>0 as the stage duration δ goes to 0, and study the asymptotic behavior of the value, optimal strategies, and equilibrium. The asymptotic analogs of the discounted, limiting-average, and uniform equilibrium payoffs are defined. Convergence implies the existence of an asymptotic discounted equilibrium payoff, strong convergence implies the existence of an asymptotic limiting-average equilibrium payoff, and exact convergence implies the existence of an asymptotic uniform equilibrium payoff.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Henceforth, whenever we discuss a value concept of a family (Γ δ ), we will omit the statement of the implicit condition that it is a family of two-person zero-sum games.
A continuous-time strategy σ is a mixed-action-valued measurable function defined on \(S\times\mathbb{R}\).
Without the assumption of finitely many actions, a uniform value need not exist [12]. The assumption of finitely many states is obviously needed.
Moreover, the stage-dependent duration δ m , payoff g m , and transition function p m can depend on past history.
The finiteness follows from the fact that the minimum and the maximum of a linear function over a simplex is attained in one of the finitely many extreme points of the simplex.
The ε in [8] is ε/8 here, and ε 1 there is sufficiently small.
References
Bewley T, Kohlberg E (1976) The asymptotic theory of stochastic games. Math Oper Res 1:197–208
Guo X, Hernandez-Lerma O (2005) Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J Appl Probab 42:303–320
Jasso-Fuentes H (2005) Noncooperative continuous-time Markov games. Morfismos 9:39–54
Levy Y (2012) Continuous-time stochastic games of fixed duration. Dyn Games Appl (this issue)
Mertens J-F, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–66
Neyman A (2003) Real algebraic tools in stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO ASI series. Kluwer Academic, Dordrecht, pp 57–75
Neyman A (2003) Stochastic games: existence of the minmax. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO ASI series. Kluwer Academic, Dordrecht, pp 173–193
Neyman A (2012) Continuous-time stochastic games. DP 616, Center for the Study of Rationality, Hebrew University of Jerusalem
Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100
Solan E (2003) Continuity of the value of competitive Markov decision processes. J Theor Probab 16:831–845
Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behav 38:362–399
Vigeral G (2012) A zero-sum stochastic game with compact action sets and no asymptotic value. Dyn Games Appl (this issue)
Zachrisson LE (1964) Markov games. In: Advances in game theory. Princeton University Press, Princeton
Acknowledgements
This research was supported in part by Israel Science Foundation grant 1596/10.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Neyman, A. Stochastic Games with Short-Stage Duration. Dyn Games Appl 3, 236–278 (2013). https://doi.org/10.1007/s13235-013-0083-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-013-0083-x