Abstract
The current study presents an online iterative adaptive dynamic programming approach to resolve the zero-sum game (ZSG) for nonlinear continuous-time (CT) systems containing a partially unknown dynamic. The Hamilton-Jacobian-Issacs (HJI) equation is solved along the state trajectory according to the value function approximation and the policy improvement online. Relaxed dynamic programming is utilized to ensure the algorithm’s convergence. Model and costate networks were established to conduct the method. Computational simulations are performed to present the efficiency of the algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. IEEE Trans. Neural Networks 9(5), 1054 (1998)
Lewis, F., Vrabie, D., Vamvoudakis, K.: Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst. 32(6), 76–105 (2012)
Song, R., Lewis, F.L., Wei, Q., Zhang, H.: Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Trans. Cybern. 46(5), 1041–1050 (2016)
Khan, S.G., Herrmann, G., Lewis, F.L., Pipe, T., Melhuish, C.: Reinforcement learning and optimal adaptive control: an overview and implementation examples. Annu. Rev. Control. 36(1), 42–59 (2012)
Vrabie, D., Lewis, F.: Adaptive Dynamic Programming algorithm for finding online the equilibrium solution of the two-player zero-sum differential game. In: Proceedings of the International Joint Conference on Neural Networks (2010)
Zhang, H., Wei, Q., Liu, D.: An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica 47(1), 207–214 (2011)
Yasini, S., Sistani, M.B.N., Karimpour, A.: Approximate dynamic programming for two-player zero-sum game related to H∞ control of unknown nonlinear continuous-time systems. Int. J. Control. Autom. Syst. 13(1), 99–109 (2014)
Su, H., Zhang, H., Zhang, K., Gao, W.: Online reinforcement learning for a class of partially unknown continuous-time non-linear systems via value iteration. Optim. Control Appl. Methods 39(2), 1011–1028 (2018)
Rantzer, A.: Relaxed dynamic programming in switching systems. IEE Proc. - Control Theory Appl. 153(5), 567–574 (2006)
Lincoln, B., Rantzer, A.: Relaxing dynamic programming. IEEE Trans. Automat. Contr. 51(8), 1249–1260 (2006)
Wang, D., Liu, D.: Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique. Neurocomputing 121, 218–225 (2013)
Zhang, H., Qin, C., Luo, Y.: Neural-network-based constrained optimal control scheme for a discrete-time switched nonlinear system using dual heuristic programming, IEEE Trans. Autom. Sci. Eng. 11(3), 839–849, 2014
Andrade, G.A., Da Fonseca Neto, J.V., Helena, P., Márcio, M.R., Gonsalves, E.: RLS estimator state space basis for the solution of HJB-Riccati Equation in approximate dynamic programming. In: IFAC Proceedings Volumes (IFAC-PapersOnline), vol. 3, no. PART 1, pp. 1153–1160 (2014)
Xiao, G., Zhang, H., Zhang, K., Wen, Y.: Value iteration based integral reinforcement learning approach for H∞ controller design of continuous-time nonlinear systems. Neurocomputing 285, 51–59 (2018)
Sun, J., Liu, C.: Finite-horizon differential games for missile–target interception system using adaptive dynamic programming with input constraints. Int. J. Syst. Sci. 49(2), 264–283 (2018)
Lewis, F.L., Vrabie, D.: Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 9(3), 32–50 (2009)
Zhu, Y., Zhao, D., Li, X.: Iterative adaptive dynamic programming for solving unknown nonlinear zero-sum game based on online data. IEEE Trans. Neural Networks Learn. Syst. 28(3), 714–725 (2016)
Liu, D., Li, H., Wang, D.: Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110, 92–100 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Beijing HIWING Sci. and Tech. Info Inst
About this paper
Cite this paper
Fu, B., Sun, B., Guo, H., Yang, T., Fu, W. (2023). Online Iterative Adaptive Dynamic Programming Approach for Solving the Zero-Sum Game for Nonlinear Continuous-Time Systems with Partially Unknown Dynamics. In: Fu, W., Gu, M., Niu, Y. (eds) Proceedings of 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022). ICAUS 2022. Lecture Notes in Electrical Engineering, vol 1010. Springer, Singapore. https://doi.org/10.1007/978-981-99-0479-2_262
Download citation
DOI: https://doi.org/10.1007/978-981-99-0479-2_262
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0478-5
Online ISBN: 978-981-99-0479-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)