Skip to main content
Log in

An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems

  • Published:
Decisions in Economics and Finance Aims and scope Submit manuscript

Abstract

In this paper, we present iterative or successive approximation methods for solving the coupled Hamilton–Jacobi–Isaacs equations (HJIEs) arising in nonzero-sum differential game for affine nonlinear systems. We particularly consider the ones arising in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) control. However, the approach is perfectly general and can be applied to any others including those arising in the N-player case. The convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the utility of the method. The results are also specialized to the coupled algebraic Riccati equations arising typically in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) linear control. In this case, a bound within which the optimal solution lies is established. Finally, based on the iterative approach developed, a local existence result for the solution of the coupled-HJIEs is also established.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Ignoring this assumption only makes the manipulations more lengthy. But the results remain the same.

References

  • Abou-Kandil, H., Freiling, G., Jank, G.: Solution and asymptotic behavior of coupled Riccati equations in jump linear systems. IEEE Trans. Autom. Control 39(8), 1631–1636 (1994)

    Article  Google Scholar 

  • Abu-Khalaf, M., Lewis, F.L., Huang, J.: Policy iterations on the Hamilton–Jacobi–Isaacs equation for \({\cal{H}}_{\infty }\) state-feedback control with input saturation. IEEE Trans. Autom. Control 51(12), 1989–1993 (2006)

    Article  Google Scholar 

  • Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. IFAC J. Autom. 41(5), 779–791 (2005)

    Article  Google Scholar 

  • Aliyu, M.D.S.: An approach for solving the Hamilton–Jacobi–Isaac equations arising in nonlinear \(\cal{H}_{\infty }\) control. IFAC J. Autom. 38, 877–884 (2003a)

    Article  Google Scholar 

  • Aliyu, M.D.S.: A transformation approach for solving the Hamilton–Jacobi–Bellman equations in \({\cal{H}}_{2}\) deterministic and stochastic optimal control of affine nonlinear systems. IFAC J. Autom. 39, 1243–1249 (2003b)

    Article  Google Scholar 

  • Aliyu, M.D.S.: Nonlinear \({\cal{H}}_{\infty }\) Control, Hamiltonian Systems and Hamilton–Jacobi Equations. CRC Press, Taylor and Francis, Boca Raton (2011)

    Book  Google Scholar 

  • Aliyu, M.D.S., Smolinsky, L.: A Parameterization approach for solving the Hamilton-Jacobi equation and application to the \({\cal{A}}_{2}\) Toda lattice. Nonlinear Dyn. Syst. Theory 5(4), 323–344 (2005)

    Google Scholar 

  • Barbu, V., Prata, Da G.: Hamilton–Jacobi Equations in Hilbert Space. Pitman Advanced Publishing Program, London (1983)

    Google Scholar 

  • Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Systems & Control Foundations, Birhauser (1997)

    Book  Google Scholar 

  • Basar, T., Bernhard, P.: \({\cal{H}}_{\infty }\) Optimal Control and Related Minimax Design. Birkhauser, New York (1991)

    Google Scholar 

  • Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory. Academic Press, New York (1982)

    Google Scholar 

  • Beard, R.W., Saridas, G.N., Wen, J.T.: Galerkin approximations of the generalized HJB equation. IFAC J. Autom. 33(12), 2159–2177 (1997)

    Article  Google Scholar 

  • Beard, R.W., Saridas, G.N., Wen, J.T.: Successive Galerkin approximation algorithms for nonlinear optimal and robust control. Int. J. Control 71(5), 717–743 (1998)

    Article  Google Scholar 

  • Benton, S.H.: The Hamilton–Jacobi Equation: A Global Approach. Academic Press, New York (1977)

    Google Scholar 

  • Chen, B.-S., Chang, Y.-C.: Nonlinear mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for robust tracking of robotic systems. Int. J. Control 67(6), 837–857 (1998)

    Article  Google Scholar 

  • Coddington, E.A., Levinsone, N.: Ordinary Differential Equations. McGraw Hill, New York (1955)

    Google Scholar 

  • Cox, D., Little, J., O’Shea, D.: Ideals, Varieties and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3rd edn. Springer, New York (2007)

    Book  Google Scholar 

  • Denman, E.D.: An additional algorithm for a system of coupled algebraic matrix Riccati equations. IEEE Trans. Comput. 26(1), 91–93 (1976)

    Article  Google Scholar 

  • Doyle, J.C., Zhou, K., Glover, K., Bodenheimer, B.: Optimal control with mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-II: optimal control. IEEE Trans. Autom. Control 39(8), 1575–1587 (1994)

    Article  Google Scholar 

  • Evans, L.C.: Partial Differential Equations, Graduate Studies in Mathematics. American Mathmtical Society, Providence (1998)

    Google Scholar 

  • Fleming, W.H., Soner, M.: Controlled Markov Processes and Viscosity Solutions. Springer, New York (2006)

    Google Scholar 

  • Freiling, G., Jank, G., AbouKandil, H.: On the global existence of coupled matrix Riccati equations in closed-loop Nash games. IEEE Trans. Autom. Control 41(2), 264–269 (1996)

    Article  Google Scholar 

  • Freiling, G., Lee, S.-R., Jank, G.: Coupled matrix Riccati equations in minimum cost variance control problems. IEEE Trans. Autom. Control 44(3), 556–560 (1999)

    Article  Google Scholar 

  • Glad, S.T.: Robustness of nonlinear state-feedback—a survey. IFAC J. Autom. 23, 425–435 (1987)

    Article  Google Scholar 

  • Isaacs, R.: Differential Games, SIAM Series in Applied Mathematics. Wiley, New York (1965)

    Google Scholar 

  • Jiang, Y., Jiang, Z.-P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)

    Article  Google Scholar 

  • Johnson, M., Bhasin, S., Dixon, W.E.: Nonlinear two-player zero-sum game approximate solution using a policy iteration algorithm. In: Proceedings 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pp. 142–147 (2011)

  • Khalil, H.K.: Nonlinear Systems. Mcmillan Publishers, New York (1992)

    Google Scholar 

  • Kleinmann, D.L.: On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control 13, 114–115 (1968)

    Article  Google Scholar 

  • Lee, C.-H.: An improved lower matrix bound for the solution of unified coupled Riccati equation. IEEE Trans. Autom. Control 50(8), 1221–1223 (2005)

    Article  Google Scholar 

  • Li, T.-Y., Gajic, Z.: Lyapunov Iterations for Solving Coupled Algebraic Riccati Equations of Nash Differential games and algebraic Riccati Equations of zero-sum games, New Trends in Dynamic Games and Applications. Birkhuser, Boston (1995)

    Google Scholar 

  • Limebeer, D.J.N., Anderson, B.D.O., Hendel, B.: A Nash game approach to mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control problem. IEEE Trans. Autom. Control 39(4), 824–839 (1994)

    Google Scholar 

  • Lin, W.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for nonlinear systems. Int. J. Control 64(5), 899–922 (1996)

    Article  Google Scholar 

  • Lions, P.L.: Generalized Solutions of Hamilton–Jacobi Equations. Research Notes in Mathematics. Pitman Advanced Publishing Program, London (1982)

    Google Scholar 

  • Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7, 75–100 (1969)

    Article  Google Scholar 

  • Meyer, G.G.L., Payne, H.J.: An iterative method of solution of the algebraic Riccati equation. IEEE Trans. Autom. Control 17(6), 550–551 (1972)

    Article  Google Scholar 

  • Ohtsuka, T.: Solutions to the Hamilton–Jacobi equation with algebraic gradients. IEEE Trans. Autom. Control 56(8), 1874–1885 (2011)

    Article  Google Scholar 

  • Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, London (1970)

    Google Scholar 

  • Papavassilopouos, G.P., Medanic, J.V., Cruz, J.B.: On the existence of Nash strategies and solutions to coupled Riccati equations in linear-quadratic Nash games. J. Optim. Theory Appl. 28(4), 49–76 (1979)

    Article  Google Scholar 

  • Papavassilopouos, G.P., Olsder, G.J.: On linear-quadratic closed-loop no memory Nash game. J. Optim. Theory Appl. 42(4), 551–560 (1984)

    Article  Google Scholar 

  • Salama, A.I.A., Ghourishankar, V.: A Computational algorithm for solving a system of coupled algebraic Riccati equations. IEEE Trans. Comput. 23(1), 100–102 (1974)

    Article  Google Scholar 

  • Starr, A.W., Ho, W.C.: Nonzero-sum differential games. J. Optim. Theory Appl. 3(4), 207–219 (1969a)

    Article  Google Scholar 

  • Starr, A.W., Ho, W.C.: Further properties of nonzero-sum differential games. J. Optim. Theory Appl. 3(3), 184–206 (1969b)

    Article  Google Scholar 

  • Vamvoudakis, K.G., Lewis, F.G.: Nonzero-sum differential games: online learning solution of coupled Hamilton–Jacobi and coupled Riccati equations. In: Proceedings of IEEE International Symposium on Intelligent Control, pp. 171–178, Denver, Colorado, USA (2011a)

  • Vamvoudakis, K.G., Lewis, F.L.: Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton–Jacobi equations. IFAC J Autom. 47(8), 1556–1569 (2011b)

    Article  Google Scholar 

  • Vamvoudakis, K.G., Lewis, F.L.: Online neural network solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int. J. Robust Nonlinear Control 22(13), 1460–1483 (2012)

    Article  Google Scholar 

  • Vit, K.: Iterative solution of the Riccati equation. IEEE Trans. Autom. Control 17(2), 258–259 (1972)

    Article  Google Scholar 

  • Yong, J., Zhou, X.Y.: Stochastic Controls. Hamiltonian Systems and HJB Equations. Springer, New York (1999)

    Google Scholar 

  • Zeidler, E.: Nonlinear Functional Analysis and Its Applications: Fixed Point Theorems, vol. 1. Springer, Hiedelberg (1985)

    Book  Google Scholar 

  • Zhou, K., Glover, K., Bodenheimer, B., Doyle, J.C.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-I: robust performance analysis. IEEE Trans. Autom. Control 39(8), 1564–1574 (1994)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. D. S. Aliyu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aliyu, M.D.S. An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems. Decisions Econ Finan 40, 1–30 (2017). https://doi.org/10.1007/s10203-017-0184-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10203-017-0184-x

Keywords

Mathematics Subject Classification

Navigation