Abstract
In this technical note, we revisit the risk-sensitive optimal control problem for Markov jump linear systems (MJLSs). We first demonstrate the inherent difficulty in solving the risk-sensitive optimal control problem even if the system is linear and the cost function is quadratic. This is due to the nonlinear nature of the coupled set of Hamilton-Jacobi-Bellman (HJB) equations, stemming from the presence of the jump process. It thus follows that the standard quadratic form of the value function with a set of coupled Riccati differential equations cannot be a candidate solution to the coupled HJB equations. We subsequently show that there is no equivalence relationship between the problems of risk-sensitive control and H ∞ control of MJLSs, which are shown to be equivalent in the absence of any jumps. Finally, we show that there does not exist a large deviation limit as well as a risk-neutral limit of the risk-sensitive optimal control problem due to the presence of a nonlinear coupling term in the HJB equations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
O. Costa, M. Fragoso, and M. Todorov, Continuous-Time Markov Jump Linear Systems, Springer, 2013.
W. M. Wonham, “Random differential equations in control theory,” Probabilistic Methods in Applied Mathematics (ed. by A.T. Bharucha-Reid), vol. 2, pp. 131–212, Academic Press, San Diego, 1970.
Y. Ji and H. Chizeck, “Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control,” IEEE Transactions on Automatic Control, vol. 35, no. 7, pp. 777–788, Jul 1990.
Z. Pan and T. Başar, “H ∞ control of Markovian jump systems and solutions to associated piecewise-deterministic differential games,” In G.J. Olsder, editor, Annals of Dynamic Games, vol. 2, pp. 61–94, 1995, Birkhäuser.
P. Shi and L. Fanbiao, “A survey on Markovian jump systems: modeling and design,” International Journal of Control, Automation, and Systems, vol. 13, no. 1, pp. 1–16, 2015.
A. Teel, A. Subbaraman, and A. Sferlazza, “Stability analysis for stochastic hybrid systems: A survey,” Automatica, vol. 50, no. 10, pp. 2435–2456, 2014.
T. Runolfsson, “Risk-sensitive control of stochastic hybrid systems on infinite time horizon,” Mathematical Problmes in Engineering, vol. 5, pp. 459–478, 2000.
W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, 2nd ed., Springer, 2006.
L. Li and V. Ugrinovskii, “On necessary and sufficient conditions for H ∞ output feedback control of Markov jump linear systems,” IEEE Transactions on Automatic Control, vol. 52, no. 7, pp. 1287–1292, July 2007.
T. Başar, “Nash equilibria of risk-sensitive nonlinear stochastic differential games,” Journal of Optimization Theory and Applications, vol. 100, no. 3, pp. 479–498, 1999.
W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, 1986.
D. Kahaner, C. Moler, and S. Nash, Numerical Methods and Software, Prentice Hall, 1977.
Q. Song, G. Yin, and Z. Zhang, “Numerical methods for controlled regime-switching diffusions and regimeswitching jump diffusions,” Automatica, vol. 42, no. 7, pp. 1147–1157, 2006.
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Jiuxiang Dong under the direction of Editor Yoshito Ohta. This research was supported by Large-scale Optimization for Multi-agent Systems (No. 1.160045.01), Ulsan National Institute of Science and Technology (UNIST).
Rights and permissions
About this article
Cite this article
Moon, J., Başar, T. Risk-sensitive control of Markov jump linear systems: Caveats and difficulties. Int. J. Control Autom. Syst. 15, 462–467 (2017). https://doi.org/10.1007/s12555-015-0114-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-015-0114-z