Journal of Global Optimization

, Volume 27, Issue 2–3, pp 149–175 | Cite as

Indefinite Stochastic Linear Quadratic Control with Markovian Jumps in Infinite Time Horizon

  • Xun Li
  • Xun Yu Zhou
  • Mustapha Ait Rami


This paper studies a stochastic linear quadratic (LQ) control problem in the infinite time horizon with Markovian jumps in parameter values. In contrast to the deterministic case, the cost weighting matrices of the state and control are allowed to be indinifite here. When the generator matrix of the jump process – which is assumed to be a Markov chain – is known and time-invariant, the well-posedness of the indefinite stochastic LQ problem is shown to be equivalent to the solvability of a system of coupled generalized algebraic Riccati equations (CGAREs) that involves equality and inequality constraints. To analyze the CGAREs, linear matrix inequalities (LMIs) are utilized, and the equivalence between the feasibility of the LMIs and the solvability of the CGAREs is established. Finally, an LMI-based algorithm is devised to slove the CGAREs via a semidefinite programming, and numerical results are presented to illustrate the proposed algorithm.

Stochastic LQ control coupled generalized algebraic Riccati equations linear matrix inequality semidefinite programming mean-square stability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ait Rami, M. and El Ghaoui, L. (1996), LMI optimization for nonstandard Riccati equations arising in stochastic control, IEEE Transactions on Automatic Control, 41, 1666–1671.Google Scholar
  2. 2.
    Ait Rami, M., Moore, J. and Zhou, X.Y. (2001), Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM Journal on Control and Optimization, 40, 1296–1311.Google Scholar
  3. 3.
    Ait Rami, M. and Zhou, X.Y. (2000), Linear matrix inequalities, Riccati equations, indefinite stochastic quadratic controls, IEEE Transactions on Automatic Control, 45, 1131-1143.Google Scholar
  4. 4.
    Albert, A. (1969), Conditions for positive and nonnegative definiteness in terms of pseudoinverses, SIAM Journal on Applied Mathematics, 17, 434-440.Google Scholar
  5. 5.
    Björk, T. (1980), Finite dimensional optimal filters for a class of Itô-processes with jumping parameters, Stochastics, 4, 167–183.Google Scholar
  6. 6.
    Bensoussan, A. (1982), Lectures on stochastic control, Lecture Notes in Mathematics, 972, 1-62.Google Scholar
  7. 7.
    Boyd, S., El Ghaoui, L., Feron, E. and Balakrishnan, V., Linear Matrix Inequality in Systems and Control Theory, SIAM, Philadelphia, 1994.Google Scholar
  8. 8.
    Chen, S., Li X. and Zhou, X.Y. (1998), Stochastic linear quadratic regulators with indefinite control weight costs, SIAM Journal on Control and Optimization, 36, 1685-1702.Google Scholar
  9. 9.
    Davis, M.H.A. (1977), Linear Estimation and Stochastic Control, Chapman and Hall, London.Google Scholar
  10. 10.
    El Ghaoui, L. and Ait Rami, M. (1996), Robust state-feedback stabilization of jump linear systems via LMIs, International Journal of Robust and Nonlinear Control, 6, 1015-1022.Google Scholar
  11. 11.
    El Ghaoui, L., Nikoukhah, R. and Delebecque, F. (1995), LMITOOL: A front-end for LMI optimization in matlab. Available via anonymous ftp to, under /pub/elghaoui/lmitool.Google Scholar
  12. 12.
    Ji, Y. and Chizeck, H.J. (1990), Controllability, stabilizability, and continuous-time Markovian jump linear quadratic, IEEE Transactions on Automatic Control, 35, 777-788.Google Scholar
  13. 13.
    Ji, Y. and Chizeck, H.J. (1992), Jump linear quadratic Gaussian control in continuous-time, IEEE Transactions on Automatic Control, 37, 1884-1892.Google Scholar
  14. 14.
    Krasosvkii, N.N. and Lidskii, E.A. (1961), Analytical design of controllers in systems with random attributes I, II, III, Automation and Remote Control, 22, 1021-1025, 1141-1146, 1289-1294.Google Scholar
  15. 15.
    Mariton, M. (1990), Jump Linear Systems in Automatic Control, Marcel Dekker, New York.Google Scholar
  16. 16.
    Mariton, M. and Bertrand, P. (1985), A homotopy algorithm for solving coupled Riccati equations, Optimal Control Applications & Methods, 6, 351-357.Google Scholar
  17. 17.
    Nesterov, Y. and Nemirovsky, A. (1993), Interior point polynomial methods in convex programming: Theory and applications, SIAM.Google Scholar
  18. 18.
    Nikoukhah, R., Delebecque, F. and El Ghaoui, L. (1995), LMITOOL: A package for LMI optimization in scilab. INRIA Rocquencourt, France.Google Scholar
  19. 19.
    Penrose, R. (1955), A generalized inverse of matrices, Proceedings of the Cambridge Philosophical Society, 51, 406-413.Google Scholar
  20. 20.
    Porkolab, L. and Khachiyan, L. (1997), On the complexity of semidefinite programs, Journal of Global Optimization, 10, 351-365.Google Scholar
  21. 21.
    Vandenberghe, L. and Boyd, S. (1995), A primal-dual potential reduction method for problems involving matrix inequalities, Mathematical Programming, 69, 205-236.Google Scholar
  22. 22.
    Vandenberghe, L. and Boyd, S. (1996), Semidefinite programming, SIAM Review, 38, 49-95.Google Scholar
  23. 23.
    Wonham, W.M. (1968), On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6, 681-697.Google Scholar
  24. 24.
    Wonham,W.M. (1970), Random differential equations in control theory, Probabilistic Methods in Applied Mathematics, Academic, New York, 2, 131-212.Google Scholar
  25. 25.
    Zhang, Q. and Yin, G. (1999), On nearly optimal controls of hybrid LQG problems, IEEE Transactions on Automatic Control, 44, 2271–2282.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Xun Li
    • 1
  • Xun Yu Zhou
    • 1
  • Mustapha Ait Rami
    • 1
  1. 1.Department of Systems Engineering and Engineering ManagementThe Chinese University of Hong KongShatin, N.T.Hong Kong

Personalised recommendations