Event-Triggered Adaptive Dynamic Programming for Continuous-Time Nonlinear Two-Player Zero-Sum Game

  • Shan Xue
  • Biao LuoEmail author
  • Derong Liu
  • Yueheng Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11307)


In this paper, an event-triggered adaptive dynamic programming (ADP) algorithm is developed to solve the two-player zero-sum game problem of continuous-time nonlinear systems. First, a critic neural network is employed to approximate the optimal value function. Then, an event-triggered ADP method is proposed, which guarantees the stability of the closed-loop system. The developed method can save the amount of computation as the control law and disturbance law that update only when the pre-designed triggering condition is violated. Finally, its effectiveness is verified through simulation results.


Event-triggering control Adaptive dynamic programming Two-player zero-sum game Hamilton-Jacobi-Isaacs equation 



This work was supported in part by the National Natural Science Foundation of China under Grants 61873350, 61503377, 61533017 and U1501251.


  1. 1.
    Singh, S., Okun, A., Jackson, A.: Artificial intelligence: learning to play go from scratch. Nature 550(7676), 336–337 (2017)Google Scholar
  2. 2.
    Liu, D., Wei, Q., Wang, D., Yang, X., Li, H.: Adaptive Dynamic Programming with Applications in Optimal Control. Springer, Cham (2017). Scholar
  3. 3.
    Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag. 4(2), 39–47 (2009)CrossRefGoogle Scholar
  4. 4.
    Zhao, Q., Xu, H., Jagannathan, S.: Near optimal output feedback control of nonlinear discrete-time systems based on reinforcement neural network learning. IEEE/CAA J. Autom. Sin. 1(4), 372–384 (2014)CrossRefGoogle Scholar
  5. 5.
    Luo, B., Wu, H.N., Huang, T.: Optimal output regulation for model-free quanser helicopter with multi-step Q-learning. IEEE Trans. Ind. Electron. 65(6), 4953–4961 (2018)CrossRefGoogle Scholar
  6. 6.
    Wang, D., Mu, C., He, H., Liu, D.: Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1358–1370 (2017)CrossRefGoogle Scholar
  7. 7.
    Luo, B., Wu, H.N., Huang, T., Liu, D.: Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50(12), 3281–3290 (2014)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Luo, B., Wu, H.N., Huang, T.: Off-policy reinforcement learning for \( H_\infty \) control design. IEEE Trans. Cybern. 45(1), 65–76 (2015)CrossRefGoogle Scholar
  9. 9.
    Zhu, Y., Zhao, D., He, H., Ji, J.: Event-triggered optimal control for partially unknown constrained-input systems via adaptive dynamic programming. IEEE Trans. Ind. Electron. 64(5), 4101–4109 (2017)CrossRefGoogle Scholar
  10. 10.
    Luo, B., et al.: Policy gradient adaptive dynamic programming for data-based optimal control. IEEE Trans. Cybern. 47(10), 3341–3354 (2017)CrossRefGoogle Scholar
  11. 11.
    Luo, B., Yang, Y., Liu, D.: Adaptive Q-learning for data-based optimal output regulation with experience replay. IEEE Trans. Cybern. (2018)CrossRefGoogle Scholar
  12. 12.
    Dong, L., Tang, Y., He, H., Sun, C.: An event-triggered approach for load frequency control with supplementary ADP. IEEE Trans. Power Syst. 32(1), 581–589 (2017)CrossRefGoogle Scholar
  13. 13.
    Luo, B., Wu, H.N., Li, H.X.: Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 684–696 (2015)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Luo, B., Huang, T., Wu, H.N., Yang, X.: Data-driven \( H_\infty \) control for nonlinear distributed parameter systems. IEEE Trans. Neural Netw. Learn. Syst. 26(11), 2949–2961 (2015)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Wang, D., Mu, C., Liu, D., Ma, H.: On mixed data and event driven design for adaptive-critic-based nonlinear \( H_ {\infty } \) control. IEEE Trans. Neural Netw. Learn. Syst. 29(4), 993–1005 (2018)CrossRefGoogle Scholar
  16. 16.
    Luo, B., Liu, D., Huang, T., Wang, D.: Model-free optimal tracking control via critic-only Q-learning. IEEE Trans. Neural Netw. Learn. Syst. 27(10), 2134–2144 (2016)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Luo, B., Liu, D., Wu, H.N.: Adaptive constrained optimal control design for data-based nonlinear discrete-time systems with critic-only structure. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2099–2111 (2018)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Zhang, Q., Zhao, D., Zhu, Y.: Event-triggered \(H_\infty \) control for continuous-time nonlinear system via concurrent learning. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1071–1081 (2017)CrossRefGoogle Scholar
  19. 19.
    Luo, B., Wu, H.N.: Approximate optimal control design for nonlinear one-dimensional parabolic PDE systems using empirical eigenfunctions and neural network. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 42(6), 1538–1549 (2012)CrossRefGoogle Scholar
  20. 20.
    Xue, S., Luo, B., Liu, D.: Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Trans. Syst. Man Cybern.: Syst. (2018).
  21. 21.
    Luo, B., Liu, D., Huang, T., Liu, J.: Output tracking control based on adaptive dynamic programming with multistep policy evaluation. IEEE Trans. Syst Man Cybern.: Syst. (2017).
  22. 22.
    Luo, B., Wu, H.N., Li, H.X.: Data-based suboptimal neuro-control design with reinforcement learning for dissipative spatially distributed processes. Industr. Eng. Chem. Res. 53(19), 8106–8119 (2014)CrossRefGoogle Scholar
  23. 23.
    Luo, B., Wu, H.N.: Online policy iteration algorithm for optimal control of linear hyperbolic PDE systems. J. Process Control 22(7), 1161–1170 (2012)CrossRefGoogle Scholar
  24. 24.
    Tabuada, P.: Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans. Autom. Control 52(9), 1680–1685 (2007)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Zhong, X., He, H.: An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Trans. Cybern. 47(3), 683–694 (2017)CrossRefGoogle Scholar
  26. 26.
    Vamvoudakis, K.G.: Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J. Autom. Sin. 1(3), 282–293 (2014)CrossRefGoogle Scholar
  27. 27.
    Khalil, H.K.: Noninear Systems, pp. 1–5. Prentice-Hall, New Jersey (1996)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Automation and Electrical EngineeringUniversity of Science and Technology BeijingBeijingChina
  2. 2.The State Key Laboratory of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of SciencesBeijingChina
  3. 3.School of AutomationGuangdong University of TechnologyGuangzhouChina

Personalised recommendations