Abstract
In this paper, we introduce a class of two-person finite discounted AR–AT (Additive Reward–Additive Transition) semi-Markov games (SMGs). We provide counterexamples to show that AR–AT and AR–AT–PT (Additive Reward–Additive Transition Probability and Time) SMGs do not satisfy the ordered field property. Some results on AR–AT–AITT (Additive Reward–Additive Transition and Action Independent Transition Time) and AR–AIT–ATT (Additive Reward–Action Independent Transition and Additive Transition Time) games are obtained in this paper. For the zero-sum games, we prove the ordered field property and the existence of pure stationary optimals for the players. Moreover, such games are formulated as a vertical linear complementarity problem (VLCP) and have been solved by Cottle-Dantzig’s algorithm under a mild assumption. We illustrate that the nonzero-sum case of such games do not necessarily have pure stationary equilibria. However, there exists a stationary equilibria which has at most two pure actions in each state for each player.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cottle RW, Dantzig GB (1970) A generalization of the linear complementarity problem. J Comb Theory 8:79–90
Cottle RW, Pang JS, Stone RE (1992) The linear complementarity problem. Academic Press, New York
Filar JA, Vrieze K (1997) Competitive Markov decision processes. Springer, New York
Himmelberg CJ, Parthasarathy T, Raghavan TES, van Vleck FS (1976) Existence of p-equilibrium and optimal stationary strategies in stochastic games. Proc Am Math Soc 60:245–251
Jianyong L, Xiaobo Z (2004) On average reward semi-Markov decision processes with a general multichain structure. Math Oper Res 29(2):339–352
Kaplansky I (1945) A contribution to von Neumanns theory of games. Ann Math 46(2):474–479
Lemke C (1965) Bimatrix Equilibrium Points and Mathematical Programming. Management Science 11:681–689
Luque-Vasquez F (2002) Zero-sum semi-Markov games in Borel spaces: discounted and average payoff. Bol. Soc. Mat. Mexicana 8:227–241
Mohan SR, Neogy SK, Parthasarathy T, Sinha S (1999) Vertical linear complementarity and discounted zero-sum stochastic games with ARAT structure. Math Prog Ser A 86:637–648
Mohan SR, Neogy SK, Parthasarathy T (2001) Pivoting algorithms for some classes of stochastic games: a survey. Int Game Theory Rev 3(2 & 3):253–281
Mohan SR, Neogy SK, Sridhar R (1996) The generalized linear complementarity problem revisited. Math Program 74:197–218
Mondal P, Sinha S (2013) Ordered field property in a subclass of finite SeR-SIT semi-Markov games. Int Game Theory Rev 15(4):1340026-1–1340026-20. doi:10.1142/S0219198913400264
Mondal P, Sinha S (2015) Ordered field property for semi-Markov games when one player controls transition probabilities and transition times. Int Game Theory Rev. 17(1):1540022-1–1540022-26. doi:10.1142/S0219198915400228
Nash JF (1951) Non cooperative games. Ann Math 54:286–295
Neogy SK, Das AK, Das P (2008) Complementarity problem involving a Vertical Block Matrix and its Solution using Neural Network Model. Mathematical Programming and Game Theory for decision making. ISI platinum jubilee series on statistical science and interdisciplinary research 1:113–130
Polowczuk W (2000) Nonzero-sum semi-Markov games with countable state spaces. Appl Math 27(4):395–402
Raghavan TES, Tijs SH, Vrieze OJ (1985) On stochastic games with additive reward and transition structure. J Optim Theory Appl 47:375–392
Ross SM (1970) Applied probability models with optimization applications. Holden-Day, San Francisco
Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100
Schweitzer PJ (1971) Iterative solution of the functional equations of undiscounted Markov renewal programming. J Math Anal Appl 34(3):495–501
Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408
Acknowledgments
The authors wish to thank the unknown referees who have patiently gone through this paper and whose suggestions have improved its presentation and readability considerably. This research was supported by a DST project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mondal, P., Sinha, S., Neogy, S.K. et al. On discounted AR–AT semi-Markov games and its complementarity formulations. Int J Game Theory 45, 567–583 (2016). https://doi.org/10.1007/s00182-015-0470-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00182-015-0470-1