A Game Theoretic Algorithm to Solve Riccati and Hamilton—Jacobi—Bellman—Isaacs (HJBI) Equations in H Control

  • Brian D. O. AndersonEmail author
  • Yantao FengEmail author
  • Weitian ChenEmail author
Part of the Springer Optimization and Its Applications book series (SOIA, volume 39)


In this chapter, we propose a new algorithm to solve Riccati equations and certain Hamilton—Jacobi—Bellman—Isaacs (HJBI) equations arising in \(H_{\infty}\) control. The need for the algorithm is motivated by the existence of \(H_{\infty}\) problems for which standard Riccati solvers break down, but which can be handled by the algorithm. By using our algorithm, we replace the problem of solving \(H_{\infty}\) Riccati equations or HJBI equations by the problem of solving a sequence of H 2 Riccati equations or Hamilton—Jacobi—Bellman (HJB) equations. The algorithms have some advantages such as a simple initialization, local quadratic rate of convergence, and a natural game theoretic interpretation. Some numerical examples are given to demonstrate advantages of our algorithm.


Riccati HJBI iterative game theoretic convergence 



This work has been supported in part by Australian Research Council Discovery Project Grant DP0664427.


  1. 1.
    Abou-Kandil, H., Freiling, G., Ionescu, V., Jank, G.: Matrix Riccati Equations in Control and Systems Theory, Birkhäuser, Basel, Switzerland (2003).zbMATHCrossRefGoogle Scholar
  2. 2.
    Anderson, B.D.O., Hitz, K.L., Diem, N.D.: Recursive algorithms for spectral factorization. IEEE Trans. Circuits Syst. CAS-21 (6), 742–750, November (1974).MathSciNetCrossRefGoogle Scholar
  3. 3.
    Anderson, B.D.O.: Second order convergence algorithms for the steady-state Riccati equation. Int. J. Contr. 28(2), 295–306 (1978).zbMATHCrossRefGoogle Scholar
  4. 4.
    Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users Guide: Third Edition, ser. Software Environments Tools. Philadelphia: SIAM, 1999.Google Scholar
  5. 5.
    Arnold, III W.F., Laub, A.J.: Generalized eigenproblem algorithms and software for algebraic Riccati equations, Proceedings of the IEEE, 72(12), 1746–1754 (1984).CrossRefGoogle Scholar
  6. 6.
    Bartels, R.H., Stewart, W., Solution of the matrix \(AX+XB=C\). Commun. ACM, 15, 820–826 (1972).CrossRefGoogle Scholar
  7. 7.
    Beard, R.W., McLain, T.W.: Successive Galerkin approximation algorithm for nonlinear optimal and robust control. Int. J. Control, 71(5), 717–743 (1998).MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Bellman, R.E.: Dynamic programming, Princeton, Princeton University Press, (1957).zbMATHGoogle Scholar
  9. 9.
    Bellman, R.E., Kalaba, R.E.: Quasilinearization and Nonlinear Boundary-Value Problems, American Elsevier, New York (1965).zbMATHGoogle Scholar
  10. 10.
    Bellman, R.E.: Introduction to the Mathematical Theory of Control Process, Academic, New York (1971).Google Scholar
  11. 11.
    Benner, P.: Computational methods for linear-quadratic optimization. Suppl. Rend. Circ. Mat. Palermo, Ser. II, 58, 21–56 (1999).MathSciNetGoogle Scholar
  12. 12.
    Benner, P., Hernández, V., Pastor, A.: On the Kleinman iteration for nonstabilizable systems. J. Math. Control Signals Syst. 16(1) 76–93 (2003).zbMATHGoogle Scholar
  13. 13.
    Cherfi, L., Abou-Kandil, H., Bourles, H.: Iterative Method for General Algebraic Riccati Equation, ACSE 05 conference, 19–21 Dec. 2005, CICC, Cairo, Egypt.Google Scholar
  14. 14.
    Damm, T.: Rational Matrix Equations in Stochastic Control, volume 297 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, (2004).Google Scholar
  15. 15.
    Datta, B.N., Numerical Methods for Linear Control Systems, Elsevier New York, (2004).zbMATHGoogle Scholar
  16. 16.
    Dieci, L.: Some numerical considerations and Newton’s method revisited for solving algebraic Riccati equations. IEEE Trans. Automa. Control, 36, 608–616 (1991).MathSciNetCrossRefGoogle Scholar
  17. 17.
    Feng, Y., Anderson, B.D.O., Rotkowitz, M.: A game theoretic algorithm to compute local stabilizing solutions to HJBI equations in nonlinear \(H_{\infty}\) control. Automatica, 45(4), 881–888 April (2009).MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, The John Hopkins University Press, Baltimore and London (1996).zbMATHGoogle Scholar
  19. 19.
    Green, M., Limebeer, D.J.N.: Linear Robust Control, Prentice Hall, Englewood Cliffs, NJ (1995).zbMATHGoogle Scholar
  20. 20.
    Guo, C.H.: A note on the minimal nonnegative solution of a nonsymmetric algebraic Riccati equation. Linear Algebra Appl. 357, 299–302 (2002).MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Guo, C.H.: Efficient methods for solving a nonsymmetric algebraic Riccati equation arising in stochastic fluid models. J. Comput Appl Math, 192, 353–373 s(2006).MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Hitz, K.L., Anderson, B.D.O.: An iterative method of computing the limiting solution of the matrix Riccati differential equation. Proc. IEE, 119(9), 1402–1406, September (1972).Google Scholar
  23. 23.
    Kalman, R.E.: The Theory of Optimal Control and the Calculus of Variations. R. Bellman (ed.), Mathematical Optimization Techniques, University of California Press, Berkeley, 309–330 (1963), Chap. 16.Google Scholar
  24. 24.
    Kleinman, D.L.: On an iterative technique for Riccati equation computation. IEEE Trans. Automat. Control 13, 114–115 (1968).CrossRefGoogle Scholar
  25. 25.
    Lancaster, P., Rodman, L.: The Algebraic Riccati Equation, Oxford University Press, Oxford, (1995).Google Scholar
  26. 26.
    Lanzon, A., Feng, Y., Anderson, B.D.O.: An iterative algorithm to solve Algebraic Riccati equations with an indefinite quadratic term. Proceedings of European Control Conference 2007, 3033–3039, Greece.Google Scholar
  27. 27.
    Lanzon, A., Feng, Y., Anderson, B.D.O., Rotkowitz, M.: Computing the positive stabilizing solution to algebraic riccati equations with an indefinite quadratic term via a recursive method. IEEE Trans. Automat. Contr. 53(10), 2280–2291, November (2008).MathSciNetCrossRefGoogle Scholar
  28. 28.
    Laub, A.J.: A Schur method for solving algebraic Riccati equation. IEEE Trans. Automat. Control 24, 913–921 (1979).MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Leake, R.J., Liu, R.: Construction of suboptimal control sequences. SIAM J. Control, 5(1) 54–63 (1967).MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7(1), 75–100 (1969).MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Martensson, K.: On the matrix Riccati equation: Inf. Sci. 3, 17–49 (1971).MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Mehrmann, V., Tan, E.: Defect correction methods for the solution of algebraic Riccati equations. IEEE Trans. Automat. Control 33, 695–698 (1988).MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Mehrmann, V.: The Autonomous Linear Quadratic Control Problem, Theory and Numerical Solution, number 163 in Lecture Notes in Control and Information Sciences. Springer-Verlag, Heidelberg (July 1991).Google Scholar
  34. 34.
    Mehrmann, V.: Numerical Methods for Eigenvalue and Control Problems. Frontiers in Numerical Analysis. Springer New York, NY (2002).Google Scholar
  35. 35.
    Sima, V.: Algorithms for Linear-Quadratic Optimization, volume 200 in Pure and Applied Mathematics, Marcel Dekker, New York, (1996).Google Scholar
  36. 36.
    van der Schaft, A.J.: L 2-gain analysis of nonlinear systems and nonlinear systems and nonlinear state feedback \(H_{\infty}\) control. IEEE Trans. Automat. Control 37, 770–784 (1992).MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Willems, J.C.: Least Squares Stationary Optimal Control and the Algebraic Riccati Equation. IEEE Trans. Automat. Control, AC-16(6), 621–634 (1971).MathSciNetCrossRefGoogle Scholar
  38. 38.
    Wise, K.A., Sedwick, J.L.: Successive approximation solution of the HJI equation. Proceeding IEEE Conference on Decision and Control, 1387–1991 (1994).Google Scholar
  39. 39.
    Zhou, K., Doyle, J.C., Glover, K.: Robust and Optimal Control, Prentice Hall, Upper Saddle River, NJ (1996).zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Research School of Information Sciences and Engineeringthe Australian National UniversityCanberraAustralia
  2. 2.National ICT AustraliaCanberraAustralia

Personalised recommendations