Abstract
Under the framework given by a growth condition, a Lyapunov property and some continuity assumptions, the present work shows the existence of lower semicontinuous solutions to the Shapley equation for zero-sum semi-Markov games with Borel spaces, weakly continuous transition probabilities and possible unbounded payoff. It is also shown the existence of stationary optimal strategies for the minimizing player and stationary \(\varepsilon \)-optimal strategies for the maximizing player. These results are proved using a fixed-point approach. Moreover, it is shown the existence of a deterministic stationary minimax strategy for a minimax semi-Markov inventory problem under mild assumptions on the demand distribution.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The authors thank to the referee for bringing their attention to this paper.
References
Feinberg, E.A., Kasyanov, P.O., Liang, Y.: Fatou’s lemma in its classical form and Lebesgue’s convergence theorems for varying measures with applications to Markov decision processes. Theory Probab. Appl. 65, 270–291 (2020)
Gatsis, K., Ribeiro, A., Pappas, G.J.: Optimal power management in wireless control systems. IEEE Trans. Autom. Control 59, 1495–1510 (2014)
González-Trejo, J.I., Hernández-Lerma, O., Hoyos-Reyes, L.F.: Minimax control of discrete-time stochastic systems. SIAM J. Control Optim. 41, 1626–1659 (2002)
Guo, X.P., Zhu, Q.: Average optimality for Markov decision processes in Borel spaces: a new condition and approach. J. Appl. Probab. 43, 318–334 (2006)
Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)
Hernández-Lerma, O., Vega-Amaya, O.: Infinite-horizon Markov control processes with undiscounted cost criteria: from average to overtaking optimality. Appl. Math. 25, 153–178 (1998)
Hernández-Lerma, O., Vega-Amaya, O., Carrasco, G.: Sample-path optimality and variance-minimization of average cost Markov control processes. SIAM J. Control Optim. 38(1), 79–93 (1999)
Jaśkiewicz, A.: Zero-sum semi-Markov games. SIAM J. Control Optim. 41, 723–739 (2002)
Jaśkiewicz, A.: A fixed point approach to solve the average cost optimality equation for semi-Markov decision processes with Feller transition probabilities. Commun. Stat., Theory Methods 36, 2559–2575 (2007)
Jaśkiewicz, A.: Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities. J. Optim. Theory Appl. 141, 321–347 (2009)
Jaśkiewicz, A., Nowak, A.S.: Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J. Control Optim. 45, 773–789 (2006)
Jaśkiewicz, A., Nowak, A.S.: On the optimality equation for average cost Markov control processes with Feller transition probabilities. J. Math. Anal. Appl. 316, 495–509 (2006)
Jaśkiewicz, A., Nowak, A.S.: Optimality in Feller semi-Markov control processes. Oper. Res. Lett. 34, 713–718 (2006)
Jaśkiewicz, A., Nowak, A.S.: Robust Markov control processes. J. Math. Anal. Appl. 420, 1337–1353 (2014)
Jaśkiewicz, A., Nowak, A.S.: Zero-sum stochastic games. In: Başar, T., Zaccour, G. (eds.) Handbook of Dynamic Game Theory. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-27335-8_8-1
Luque-Vásquez, F., Minjárez-Sosa, J.A., Rosas-Rosas, L.C.: Semi-Markov control models with partially known holding times distribution: discounted and average criteria. Acta Appl. Math. 114, 135–156 (2011)
Mesquita, A.R., Hespanha, J.P., Nair, G.N.: Redundant data transmission in control/estimation over lossy networks. Automatica 48, 1612–1620 (2012)
Nowak, A.S.: Measurable selection theorems for minimax stochastic optimization problems. SIAM J. Control Optim. 23, 466–476 (1985)
Ross, S.M.: Applied Probability Models with Optimization Applications. Dover, New York (1970)
Tanaka, K., Wakuta, K.: On semi-Markov games. Sci. Rep. Niigata Univ. Ser. A 13, 55–64 (1976)
Vega-Amaya, O.: The average cost optimality equation: a fixed point approach. Bol. Soc. Mat. Mex. 9, 185–195 (2003)
Vega-Amaya, O.: Zero-sum semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J. Control Optim. 42, 1876–1894 (2003)
Vega-Amaya, O.: On the regularity property of semi-Markov processes with Borel state spaces. In: Hernández-Hernández, D., Minjárez-Sosa, A. (eds.) Optimization, Control, and Applications of Stochastic Systems, pp. 301–309. Springer, Berlin (2012)
Vega-Amaya, O.: Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited. J. Math. Anal. Appl. 464, 152–163 (2018)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Work partially supported by Consejo Nacional de Ciencia y Tecnología (CONACYT-Mexico) under grant Ciencia Frontera 2019-87787.
Rights and permissions
About this article
Cite this article
Vega-Amaya, Ó., Luque-Vásquez, F. & Castro-Enríquez, M. Zero-Sum Average Cost Semi-Markov Games with Weakly Continuous Transition Probabilities and a Minimax Semi-Markov Inventory Problem. Acta Appl Math 177, 9 (2022). https://doi.org/10.1007/s10440-022-00470-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10440-022-00470-5