Abstract
In this paper, we present iterative or successive approximation methods for solving the coupled Hamilton–Jacobi–Isaacs equations (HJIEs) arising in nonzero-sum differential game for affine nonlinear systems. We particularly consider the ones arising in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) control. However, the approach is perfectly general and can be applied to any others including those arising in the N-player case. The convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the utility of the method. The results are also specialized to the coupled algebraic Riccati equations arising typically in mixed \({\mathcal H}_{2}/{\mathcal H}_{\infty }\) linear control. In this case, a bound within which the optimal solution lies is established. Finally, based on the iterative approach developed, a local existence result for the solution of the coupled-HJIEs is also established.
Similar content being viewed by others
Notes
Ignoring this assumption only makes the manipulations more lengthy. But the results remain the same.
References
Abou-Kandil, H., Freiling, G., Jank, G.: Solution and asymptotic behavior of coupled Riccati equations in jump linear systems. IEEE Trans. Autom. Control 39(8), 1631–1636 (1994)
Abu-Khalaf, M., Lewis, F.L., Huang, J.: Policy iterations on the Hamilton–Jacobi–Isaacs equation for \({\cal{H}}_{\infty }\) state-feedback control with input saturation. IEEE Trans. Autom. Control 51(12), 1989–1993 (2006)
Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. IFAC J. Autom. 41(5), 779–791 (2005)
Aliyu, M.D.S.: An approach for solving the Hamilton–Jacobi–Isaac equations arising in nonlinear \(\cal{H}_{\infty }\) control. IFAC J. Autom. 38, 877–884 (2003a)
Aliyu, M.D.S.: A transformation approach for solving the Hamilton–Jacobi–Bellman equations in \({\cal{H}}_{2}\) deterministic and stochastic optimal control of affine nonlinear systems. IFAC J. Autom. 39, 1243–1249 (2003b)
Aliyu, M.D.S.: Nonlinear \({\cal{H}}_{\infty }\) Control, Hamiltonian Systems and Hamilton–Jacobi Equations. CRC Press, Taylor and Francis, Boca Raton (2011)
Aliyu, M.D.S., Smolinsky, L.: A Parameterization approach for solving the Hamilton-Jacobi equation and application to the \({\cal{A}}_{2}\) Toda lattice. Nonlinear Dyn. Syst. Theory 5(4), 323–344 (2005)
Barbu, V., Prata, Da G.: Hamilton–Jacobi Equations in Hilbert Space. Pitman Advanced Publishing Program, London (1983)
Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Systems & Control Foundations, Birhauser (1997)
Basar, T., Bernhard, P.: \({\cal{H}}_{\infty }\) Optimal Control and Related Minimax Design. Birkhauser, New York (1991)
Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory. Academic Press, New York (1982)
Beard, R.W., Saridas, G.N., Wen, J.T.: Galerkin approximations of the generalized HJB equation. IFAC J. Autom. 33(12), 2159–2177 (1997)
Beard, R.W., Saridas, G.N., Wen, J.T.: Successive Galerkin approximation algorithms for nonlinear optimal and robust control. Int. J. Control 71(5), 717–743 (1998)
Benton, S.H.: The Hamilton–Jacobi Equation: A Global Approach. Academic Press, New York (1977)
Chen, B.-S., Chang, Y.-C.: Nonlinear mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for robust tracking of robotic systems. Int. J. Control 67(6), 837–857 (1998)
Coddington, E.A., Levinsone, N.: Ordinary Differential Equations. McGraw Hill, New York (1955)
Cox, D., Little, J., O’Shea, D.: Ideals, Varieties and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3rd edn. Springer, New York (2007)
Denman, E.D.: An additional algorithm for a system of coupled algebraic matrix Riccati equations. IEEE Trans. Comput. 26(1), 91–93 (1976)
Doyle, J.C., Zhou, K., Glover, K., Bodenheimer, B.: Optimal control with mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-II: optimal control. IEEE Trans. Autom. Control 39(8), 1575–1587 (1994)
Evans, L.C.: Partial Differential Equations, Graduate Studies in Mathematics. American Mathmtical Society, Providence (1998)
Fleming, W.H., Soner, M.: Controlled Markov Processes and Viscosity Solutions. Springer, New York (2006)
Freiling, G., Jank, G., AbouKandil, H.: On the global existence of coupled matrix Riccati equations in closed-loop Nash games. IEEE Trans. Autom. Control 41(2), 264–269 (1996)
Freiling, G., Lee, S.-R., Jank, G.: Coupled matrix Riccati equations in minimum cost variance control problems. IEEE Trans. Autom. Control 44(3), 556–560 (1999)
Glad, S.T.: Robustness of nonlinear state-feedback—a survey. IFAC J. Autom. 23, 425–435 (1987)
Isaacs, R.: Differential Games, SIAM Series in Applied Mathematics. Wiley, New York (1965)
Jiang, Y., Jiang, Z.-P.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Control 60(11), 2917–2929 (2015)
Johnson, M., Bhasin, S., Dixon, W.E.: Nonlinear two-player zero-sum game approximate solution using a policy iteration algorithm. In: Proceedings 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pp. 142–147 (2011)
Khalil, H.K.: Nonlinear Systems. Mcmillan Publishers, New York (1992)
Kleinmann, D.L.: On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control 13, 114–115 (1968)
Lee, C.-H.: An improved lower matrix bound for the solution of unified coupled Riccati equation. IEEE Trans. Autom. Control 50(8), 1221–1223 (2005)
Li, T.-Y., Gajic, Z.: Lyapunov Iterations for Solving Coupled Algebraic Riccati Equations of Nash Differential games and algebraic Riccati Equations of zero-sum games, New Trends in Dynamic Games and Applications. Birkhuser, Boston (1995)
Limebeer, D.J.N., Anderson, B.D.O., Hendel, B.: A Nash game approach to mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control problem. IEEE Trans. Autom. Control 39(4), 824–839 (1994)
Lin, W.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) control for nonlinear systems. Int. J. Control 64(5), 899–922 (1996)
Lions, P.L.: Generalized Solutions of Hamilton–Jacobi Equations. Research Notes in Mathematics. Pitman Advanced Publishing Program, London (1982)
Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7, 75–100 (1969)
Meyer, G.G.L., Payne, H.J.: An iterative method of solution of the algebraic Riccati equation. IEEE Trans. Autom. Control 17(6), 550–551 (1972)
Ohtsuka, T.: Solutions to the Hamilton–Jacobi equation with algebraic gradients. IEEE Trans. Autom. Control 56(8), 1874–1885 (2011)
Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, London (1970)
Papavassilopouos, G.P., Medanic, J.V., Cruz, J.B.: On the existence of Nash strategies and solutions to coupled Riccati equations in linear-quadratic Nash games. J. Optim. Theory Appl. 28(4), 49–76 (1979)
Papavassilopouos, G.P., Olsder, G.J.: On linear-quadratic closed-loop no memory Nash game. J. Optim. Theory Appl. 42(4), 551–560 (1984)
Salama, A.I.A., Ghourishankar, V.: A Computational algorithm for solving a system of coupled algebraic Riccati equations. IEEE Trans. Comput. 23(1), 100–102 (1974)
Starr, A.W., Ho, W.C.: Nonzero-sum differential games. J. Optim. Theory Appl. 3(4), 207–219 (1969a)
Starr, A.W., Ho, W.C.: Further properties of nonzero-sum differential games. J. Optim. Theory Appl. 3(3), 184–206 (1969b)
Vamvoudakis, K.G., Lewis, F.G.: Nonzero-sum differential games: online learning solution of coupled Hamilton–Jacobi and coupled Riccati equations. In: Proceedings of IEEE International Symposium on Intelligent Control, pp. 171–178, Denver, Colorado, USA (2011a)
Vamvoudakis, K.G., Lewis, F.L.: Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton–Jacobi equations. IFAC J Autom. 47(8), 1556–1569 (2011b)
Vamvoudakis, K.G., Lewis, F.L.: Online neural network solution of nonlinear two-player zero-sum games using synchronous policy iteration. Int. J. Robust Nonlinear Control 22(13), 1460–1483 (2012)
Vit, K.: Iterative solution of the Riccati equation. IEEE Trans. Autom. Control 17(2), 258–259 (1972)
Yong, J., Zhou, X.Y.: Stochastic Controls. Hamiltonian Systems and HJB Equations. Springer, New York (1999)
Zeidler, E.: Nonlinear Functional Analysis and Its Applications: Fixed Point Theorems, vol. 1. Springer, Hiedelberg (1985)
Zhou, K., Glover, K., Bodenheimer, B., Doyle, J.C.: Mixed \({\cal{H}}_{2}/{\cal{H}}_{\infty }\) performance objectives-I: robust performance analysis. IEEE Trans. Autom. Control 39(8), 1564–1574 (1994)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Aliyu, M.D.S. An iterative computational scheme for solving the coupled Hamilton–Jacobi–Isaacs equations in nonzero-sum differential games of affine nonlinear systems. Decisions Econ Finan 40, 1–30 (2017). https://doi.org/10.1007/s10203-017-0184-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10203-017-0184-x
Keywords
- Differential games
- Coupled Hamilton–Jacobi–Isaacs equations
- Vector identity
- Successive approximation method
- Bounded continuous functions
- Convergence
- Coupled algebraic Riccati equations