Abstract
In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates are allowed to be unbounded. Under certain Lyapunov condition, we establish the existence and uniqueness of the solution to the Hamilton–Jacobi–Bellman equation. Also, we prove the existence of optimal risk-sensitive control in the class of Markov control and completely characterized the optimal control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bauerle N, Rieder U (2014) More risk-sensitive Markov decision processes. Math Oper Res 39:105–120
Bertsekas D, Shreve S (1996) Stochastic optimal control: the discrete-time case. Academic Press Inc, New York
Ghosh MK, Saha S (2014) Risk-sensitive control of continuous-time Markov chains. Stoch 86:655–675
Guo XP, Hernandez-Lerma O (2009) Continuous-time Markov decision processes: theory and applications. Stochastic modelling and applied probability. Springer, Berlin
Guo X, Liao ZW (2019) Risk-sensitive discounted continuous-time Markov decision processes with unbounded rates. SIAM J Control Optim 57:3857–3883
Guo X, Piunovskiy A (2011) Discounted continuous-time Markov decision processes with constraints: unbounded transition and loss rates. Math Oper Res 36:105–132
Guo X, Zhang J (2019) Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces. Discrete Event Dyn Syst 29:445–471
Guo X, Zhang Y (2020) On risk-sensitive piecewise deterministic Markov decision processes. Appl Math Optim 81:685–710
Guo X, Huang Y, Song X (2012) Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent polices. SIAM J Control Optim 50:23–47
Guo X, Liu Q, Zhang Y (2019) Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates. 4OR 17:427–442
Hernandez-Lerma O, Lasserre J (1999) Further topics on discrete-time Markov control processes. Springer, New York
Kitaev MY (1995) Semi-Markov and jump Markov controlled models: average cost criterion. SIAM Theory Probab Appl 30:272–288
Kitaev MY, Rykov VV (1995) Controlled queueing systems. CRC Press, Boca Raton
Kumar KS, Pal C (2013) Risk-sensitive control of jump process on denumerable state space with near monotone cost. Appl Math Optim 68:311–331
Kumar KS, Pal C (2015) Risk-sensitive control of continuous-time Markov processes with denumerable state space. Stoch Anal Appl 33:863–881
Masi GB, Stettner L (2000) Infinite horizon risk-sensitive control of discrete time Markov processes with small risk. Syst Control Lett 40:15–20
Masi GB, Stettner L (2007) Infinite horizon risk-sensitive control of discrete time Markov processes under minorization property. SIAM J Control Optim 46:231–252
Pal C, Pradhan S (2019) Risk-sensitive control of pure jump processes on a general state space. Int J Probab Stoch Process 91(2):155–174
Piunovskiy A, Zhang Y (2020) Continuous-time Markov decision processes. Springer, Berlin
Piunovskiy A, Zhang Y (2011) Discounted continuous-time Markov decision processes with unbounded rates: the convex analytic approach. SIAM J Control Optim 49:2032–2061
Prieto-Rumeau T, Hernandez-Lerma O (2012) Selected topics in continuous-time controlled Markov chains and Markov games. Imperical College Press, London
Wei Q (2016) Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion. Math Methods Oper Res 84:461–487
Whittle P (1990) Risk-sensitive optimal control, Wiley-Inter Science series in systems and optimization. Wiley, Chichester
Zhang Y (2017) Continuous-time Markov decision processes with exponential utility. SIAM J Control Optim 55:2636–2660
Acknowledgements
We thank the anonymous referees for their valuable comments and helpful suggestions that have improved the presentation of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Golui, S., Pal, C. Risk-sensitive discounted cost criterion for continuous-time Markov decision processes on a general state space. Math Meth Oper Res 95, 219–247 (2022). https://doi.org/10.1007/s00186-022-00779-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-022-00779-9