Journal of Global Optimization

, Volume 57, Issue 4, pp 1245–1262 | Cite as

Infinite horizon \(H_2/H_\infty \) optimal control for discrete-time Markov jump systems with (\(x,u,v\))-dependent noise

  • Ting Hou
  • Weihai ZhangEmail author
  • Hongji Ma


In this paper, an infinite horizon \(H_2/H_\infty \) control problem is addressed for a broad class of discrete-time Markov jump systems with (\(x,u,v\))-dependent noises. First of all, under the condition of exact detectability, the stochastic Popov–Belevich–Hautus (PBH) criterion is utilized to establish an extended Lyapunov theorem for a generalized Lyapunov equation. Further, a necessary and sufficient condition is presented for the existence of state-feedback \(H_2/H_\infty \) optimal controller on the basis of two coupled matrix Riccati equations, which may be solved by a backward iterative algorithm. A numerical example with simulations is supplied to illustrate the proposed theoretical results.


Infinite horizon \(H_2/H_\infty \) control Markov jump Exact detectability Coupled matrix Riccati equations 



This work was supported by the Mathematical Tianyuan Youth Foundation of China (No. 11126094), the National Natural Science Foundation of China (No. 61174078), the Key Project of Natural Science Foundation of Shandong Province (No. ZR2009GZ001), the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20103718110006), the Research Fund for the Taishan Scholar Project of Shandong Province of China, and the SDUST Research Fund (No. 2011KYTD105).


  1. 1.
    Weiner, N.: The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications. Weiley, New York (1949)Google Scholar
  2. 2.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. ASME J. Basic Eng. 82(1), 34–45 (1960)CrossRefGoogle Scholar
  3. 3.
    Kalman, R.E.: On the general theory of control systems. IFAC First International Conference, pp. 481–493 (1960)Google Scholar
  4. 4.
    Oksendal, B.: Stochastic Differential Equations: An Introduction with Application, 5th edn. Springer, New York (1998)CrossRefGoogle Scholar
  5. 5.
    Costa, O.L.V., Fragoso, M.D., Marques, R.P.: Discrete-Time Markovian Jump Linear Systems. Springer, New York (2005)Google Scholar
  6. 6.
    Boukas, E.K.: Control of Singular Systems with Random Abrupt Changes. Springer, Berlin (2008)Google Scholar
  7. 7.
    Yong, J., Zhou, X.Y.: Stochastic Control: Hamilton Sytsems and HJB Equations. Springer, New York (1999)CrossRefGoogle Scholar
  8. 8.
    Zhang, W., Zhang, H., Chen, B.S.: Generalized Lyapunov equation approach to state-dependent stochastic stabilization/detectability criterion. IEEE Trans. Autom. Control 53(7), 1630–1642 (2008)CrossRefGoogle Scholar
  9. 9.
    Petersen, I.R., Ugrinovskii, V., Savkin, A.V.: Robust Control Design Using \(H_\infty \) Methods. Springer, Berlin (2000)CrossRefGoogle Scholar
  10. 10.
    Wang, Z., Ho, D.W.C., Dong, H., Gao, H.: Robust \(H_{\infty }\) finite horizon control for a class of stochastic nonlinear time-varying systems subject to sensor and actuator saturations. IEEE Trans. Autom. Control 55(7), 1716–1722 (2010)CrossRefGoogle Scholar
  11. 11.
    Zames, G.: Feedback and optimal sensitivity: Model reference transformation, multiplicative seminorms, and approximative inverses. IEEE Trans. Autom. Control 26(2), 301–320 (1981)CrossRefGoogle Scholar
  12. 12.
    Doyle, J.C., Glover, K., Khargonekar, P.P., Francis, B.: State-space solutions to standard \(H_{2}\) and \(H_{\infty }\) problems. IEEE Trans. Autom. Control 34(8), 831–847 (1989)CrossRefGoogle Scholar
  13. 13.
    Limebeer, D.J.N., Anderson, B.D.O., Hendel, B.: A Nash game approach to mixed \(H_2/H_{\infty }\) control. IEEE Trans. Autom. Control 39(1), 69–82 (1994)CrossRefGoogle Scholar
  14. 14.
    Khargonekar, P.P., Rotea, M.A.: Mixed \(H_2/H_{\infty }\) control: a convex optimization approach. IEEE Trans. Autom. Control 36(7), 824–837 (1991)CrossRefGoogle Scholar
  15. 15.
    Hinrichsen, D., Pritchard, A.J.: Stochastic \(H_{\infty }\). SIAM J. Control Optim. 36(5), 1504–1538 (1998)CrossRefGoogle Scholar
  16. 16.
    Zhang, W., Chen, B.S.: State feedback \(H_{\infty }\) control for a class of nonlinear stochastic systems. SIAM J. Control Optim. 44(6), 1973–1991 (2006)CrossRefGoogle Scholar
  17. 17.
    Yin, G., Zhou, X.Y.: Markowitz’s mean-variance portfolio selection with regime switching: from discrete-time models to their continuous-time limits. IEEE Trans. Autom. Control 49(3), 349–360 (2004)CrossRefGoogle Scholar
  18. 18.
    Li, X., Zhou, X.Y., Ait Rami, M.: Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon. J. Glob. Optim. 27(2), 149–175 (2003)CrossRefGoogle Scholar
  19. 19.
    Mao, X., Yuan, C.: Stochastic Differential Equations with Markovian Swithcing. Imperial College Press, London (2006)CrossRefGoogle Scholar
  20. 20.
    Dragan, V., Morozan, T., Stoica, A.M.: Mathematical Methods in Robust Control of Discrete-Time Linear Stochastic Systems. Springer, New York (2010)CrossRefGoogle Scholar
  21. 21.
    Lin, Z., Lin, Y., Zhang, W.: A unified design for state and output feedback \(H_{\infty }\) control of nonlinear stochastic Markov jump systems with state and disturbance-dependent noise. Automatica 45(12), 2955–2962 (2009)CrossRefGoogle Scholar
  22. 22.
    Siqueria, A.A.G., Terra, M.H.: A fault-tolerant manipulator robot based on \(H_{2}, H_{\infty }\) and \(H_2/H_{\infty }\) Markov controls. IEEE Trans. Mechatron. 14(2), 257–263 (2009)CrossRefGoogle Scholar
  23. 23.
    Zhang, W., Huang, Y., Xie, L.: Infinite horizon stochastic \(H_{2}/H_{\infty }\) control for discrete-time systems with state and disturbance dependent noise. Automatica 44(3), 2306–2316 (2008)CrossRefGoogle Scholar
  24. 24.
    Hou, T., Zhang, W., Ma, H.: Finite horizon \(H_2/H_\infty \) control for discrete-time stochastic systems with Markovian jumps and multiplicative noise. IEEE Trans. Autom. Control 55(5), 1185–1191 (2010)CrossRefGoogle Scholar
  25. 25.
    Hou, T.: Stability and Robust \(H_2/H_\infty \) Control for Discrete-Time Markov Jump Systems. Ph.D. thesis, Shandong University of Science and Technology, Qingdao, China (2010)Google Scholar
  26. 26.
    Morozan, T.: Stochastic uniformly observability and Riccati equations of stochastic control. Revue Roumaine de Mathématique Pures et Appliquées 9, 771–781 (1993)Google Scholar
  27. 27.
    Ma H., Zhang W., Hou, T.: Infinite horizon \(H_2/H_\infty \) control for discrete-time time-varying Markov jump systems with multiplicative noise. The 18th IFAC World Congress, Milano, Italy, pp. 4626–4631 (2011)Google Scholar
  28. 28.
    Schneider, H.: Positive operator and an inertia theorem. Numer. Math. 7(1), 11–17 (1965)CrossRefGoogle Scholar
  29. 29.
    Costa, E.F., do Val, J.B.R.: On the detectability and observability of discrete-time Markov jump linear systems. Syst. Control Lett. 44(2), 135–145 (2001)CrossRefGoogle Scholar
  30. 30.
    Ni, Y., Zhang, W., Fang, H.: On the observability and detectability of linear stochastic systems with Markov jumps and multiplicative noise. J. Syst. Sci. Complex. 23(1), 102–115 (2010)CrossRefGoogle Scholar
  31. 31.
    Chen, B.S., Zhang, W.: Stochastic \(H_{2}/H_{\infty }\) control with state-dependent noise. IEEE Trans. Autom. Control 49(1), 45–57 (2004)CrossRefGoogle Scholar
  32. 32.
    Zhang, W., Chen, B.S.: General D-stability and D-stabilization for linear stochastic systems: contunuous-time case. The 8th IEEE International Conference on Control and Automation, June 9–11, Xiamen, China, 131–136 (2010)Google Scholar
  33. 33.
    Zhang, W., Chen, B.S.: \({\cal H}\)-representation and applications to generalized Lyapunov equations and linear stochastic systems. IEEE Trans. Autom. Control. 57(12), 3009–3022 (2012)Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.College of ScienceShandong University of Science and TechnologyQingdaoChina
  2. 2.College of Information and Electrical EngineeringShandong University of Science and TechnologyQingdaoChina

Personalised recommendations