Skip to main content
SpringerLink
Log in

Event-triggered optimal control for nonlinear stochastic systems via adaptive dynamic programming

  • Original paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

For nonlinear Itô-type stochastic systems, the problem of event-triggered optimal control (ETOC) is studied in this paper, and the adaptive dynamic programming (ADP) approach is explored to implement it. The value function of the Hamilton–Jacobi–Bellman(HJB) equation is approximated by applying critical neural network (CNN). Moreover, a new event-triggering scheme is proposed, which can be used to design ETOC directly via the solution of HJB equation. By utilizing the Lyapunov direct method, it can be proved that the ETOC based on ADP approach can ensure that the CNN weight errors and states of system are semi-globally uniformly ultimately bounded in probability. Furthermore, an upper bound is given on predetermined cost function. Specifically, there has been no published literature on the ETOC for nonlinear Itô-type stochastic systems via the ADP method. This work is the first attempt to fill the gap in this subject. Finally, the effectiveness of the proposed method is illustrated through two numerical examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tong, D., Xu, C., Chen, Q., Zhou, W., Xu, Y.: Sliding mode control for nonlinear stochastic systems with Markovian jumping parameters and mode-dependent time-varying delays. Nonlinear Dyn. 100(2), 1343–1358 (2020)

    Article  MATH  Google Scholar 

  2. Ding, K., Zhu, Q., Li, H.: A generalized system approach to intermittent nonfragile control of stochastic neutral time-varying delay systems. IEEE Trans. Syst. Man Cybern. Syst. https://doi.org/10.1109/TSMC.2020.2965091

  3. Zhu, Q., Wang, H.: Output feedback stabilization of stochastic feedforward systems with unknown control coefficients and unknown output function. Automatica 87, 166–175 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  4. Li, Y., Liu, Y., Tong, S.: Observer-based neuro-adaptive optimized control for a class of strict-feedback nonlinear systems with state constraints. IEEE Trans. Neural Netw. Learn. Syst. https://doi.org/10.1109/TNNLS.2021.3051030

  5. Zhang, G., Zhu, Q.: Guaranteed cost control for impulsive nonlinear Itô stochastic systems with mixed delays. J. Franklin Inst. 357(11), 6721–6737 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  6. Li, Y., Min, X., Tong, S.: Observer-based fuzzy adaptive inverse optimal output feedback control for uncertain nonlinear systems. IEEE Trans. Fuzzy Syst. https://doi.org/10.1109/TFUZZ.2020.2979389

  7. Chao, D., Che, W., Shi, P.: Cooperative fault-tolerant output regulation for multiagent systems by distributed learning control approach. IEEE Trans. Neural Netw. Learn. Syst. 31(11), 4831–4841 (2019)

    MathSciNet  Google Scholar 

  8. Yu, H., Hao, F.: Input-to-state stability of integral-based event-triggered control for linear plants. Automatica 85, 248–255 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hu, W., Liu, L., Feng, G.: Consensus of linear multi-agent systems by distributed event-triggered strategy. IEEE Trans. Cybern. 46(1), 148–157 (2016)

    Article  Google Scholar 

  10. Hu, T., He, Z., Zhang, X., Zhong, S.: Leader-following consensus of fractional-order multi-agent systems based on event-triggered control. Nonlinear Dyn. 99(3), 2219–2232 (2020)

    Article  Google Scholar 

  11. Chao, D., Che, W., Wu, Z.: A dynamic periodic event-triggered approach to consensus of heterogeneous linear multiagent systems with time-varying communication delays. IEEE Trans. Cybern. https://doi.org/10.1109/TCYB.2020.3015746

  12. Yang, X., He, H.: Adaptive critic designs for event-triggered robust control of nonlinear systems with unknown dynamics. IEEE Trans. Cybern. 49(6), 2255–2267 (2019)

    Article  Google Scholar 

  13. Mu, C., Liao, K., Wang, K.: Event-triggered design for discrete-time nonlinear systems with control constraints. Nonlinear Dyn. https://doi.org/10.1007/s11071-021-06218-4

  14. Qi, W., Zong, G., Zheng, W.: Adaptive event-triggered SMC for stochastic switching systems with semi-Markov process and application to boost converter circuit model. IEEE Trans. Circuits Syst. I Regular Papers 68(2), 786–796 (2021)

    Article  MathSciNet  Google Scholar 

  15. Qi, W., Hou, Y., Zong, G., Ahn, C.K.: Finite-time event- triggered control for semi-Markovian switching cyber- physical systems with FDI attacks and applications. IEEE Trans. Circuits Syst. I Regular Papers. https://doi.org/10.1109/TCSI.2021.3071341

  16. Yang, J., Lu, J., Li, L., Liu, Y., Wang, Z., Alsaadi, F.E.: Event-triggered control for the synchronization of Boolean control networks. Nonlinear Dyn. 96(2), 1335–1344 (2019)

    Article  MATH  Google Scholar 

  17. Qu, F.L., Guan, Z.H., He, D.X., Chi, M.: Event-triggered control for networked control systems with quantization and packet losses. J. Franklin Inst. 352(3), 974–986 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Tian, E., Wang, X., Peng, C.: Probabilistic-constrained distributed filtering for a class of nonlinear stochastic systems subject to periodic DoS attacks. IEEE Trans. Circuits Syst. I Regular Papers 67(12), 5369–5379 (2020)

    Article  MathSciNet  Google Scholar 

  19. Kulikov, G.Y., Kulikova, M.V.: Itô-Taylor-based square-root unscented Kalman filtering methods for state estimation in nonlinear continuous-discrete stochastic systems. Europ. J. Control 58, 101–113 (2021)

    Article  MATH  Google Scholar 

  20. Zhu, Q.: Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans. Automatic Control 64(9), 3764–3771 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  21. Luo, S., Deng, F.: On event-triggered control of nonlinear stochastic systems. IEEE Trans. Automatic Control 65(1), 369–375 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  22. Zhang, J., Fridman, E.: Dynamic event-triggered control of networked stochastic systems with scheduling protocols. IEEE Trans. Automatic Control. https://doi.org/10.1109/TAC.2021.3061668

  23. Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal control. Wiley, Hoboken (2012)

    Book  MATH  Google Scholar 

  24. Beard, R.W.: Improving the closed-loop performance of nonlinear systems. Rensselaer Polytechnic Institute (1995)

  25. Werbos, P.: Advanced forecasting methods for global crisis warning and models of intelligence. General Syst. Yearbook 22(6), 25–38 (1977)

    Google Scholar 

  26. Al-Tamimi, A., Lewis, F.L., Abu-Khalaf, M.: Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof, IEEE Transactions on Systems, Man, and Cybernetics. Part B (Cybernetics) 38(4), 943–949 (2008)

    Article  Google Scholar 

  27. Jiang, Y., Jiang, Z.: Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Automatic Control 60(11), 2917–2919 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  28. Wang, D., Mu, C., Liu, D., Ma, H.: On mixed data and event driven design for adaptive-critic-based nonlinear \(H_{\infty }\) control. IEEE Trans. Neural Netw. Learn. Syst. 29(4), 993–1005 (2018)

    Google Scholar 

  29. Xue, S., Luo, B., Liu, D., Li, Y.: Adaptive dynamic programming based event-triggered control for unknown continuous-time nonlinear systems with input constraints. Neurocomputing 396, 191–200 (2020)

    Article  Google Scholar 

  30. Han, X., Zhao, X., Sun, T., Wu, Y.: Event-triggered optimal control for discrete-time switched nonlinear systems with constrained control input. IEEE Trans. Syst. Man Cybern. Syst. https://doi.org/10.1109/TSMC.2020.2987136

  31. Xue, S., Luo, B., Liu, D.: Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Trans. Syst. Man Cybern. Syst. 50(9), 3189–3199 (2020)

    Article  Google Scholar 

  32. Dong, L., Zhong, X., Sun, C., He, H.: Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Trans. Neural Netw. Learn. Syst. 28(8), 1941–1952 (2017)

    Article  MathSciNet  Google Scholar 

  33. Zhu, Y., Zhao, D., He, H., Ji, J.: Event-triggered optimal control for partially unknown constrained-input systems via adaptive dynamic programming. IEEE Trans. Ind. Electron. 64(5), 4101–4109 (2017)

    Article  Google Scholar 

  34. Yang, X., He, H., Liu, D.: Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems. IEEE Trans. Syst. Man Cybern. Syst. 49(9), 1866–1878 (2019)

    Article  Google Scholar 

  35. Chen, X., Chen, X., Bai, W., Guo, Z.: Event-triggered optimal control for macro-micro composite stage system via single-network ADP method. IEEE Trans. Ind. Electron. https://doi.org/10.1109/TIE.2020.2984462

  36. Guo, Z., Yao, D., Bai, W., Li, H., Lu, R.: Event-triggered guaranteed cost fault-tolerant optimal tracking control for uncertain nonlinear system via adaptive dynamic programming. Int. J. Robust Nonlinear Control. https://doi.org/10.1002/rnc.5403

  37. Song, R., Liu, L.: Event-triggered constrained robust control for partly-unknown nonlinear systems via ADP. Neurocomputing 404, 294–303 (2020)

    Article  Google Scholar 

  38. Vamvoudakis, K.G.: Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J. Automatica Sinica 1(3), 282–293 (2014)

    Article  Google Scholar 

  39. Zhao, B., Liu, D.: Event-triggered decentralized tracking control of modular reconfigurable robots through adaptive dynamic programming. IEEE Trans. Ind. Electron. 67(4), 3054–3064 (2020)

    Article  Google Scholar 

  40. Mu, C., Wang, K.: Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism. Nonlinear Dyn. 95(4), 2639–2657 (2019)

    Article  MATH  Google Scholar 

  41. Luo, B., Yang, Y., Liu, D., Wu, H.: Event-triggered optimal control with performance guarantees using adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 31(1), 76–88 (2020)

    Article  MathSciNet  Google Scholar 

  42. Liu, D., Wei, Q., Wang, D., Yang, X., Li, H.: Adaptive Dynamic Programming with Applications in Optimal Control. Springer, Cham (2017)

    Book  MATH  Google Scholar 

  43. Sahoo, A., Xu, H., Jagannathan, S.: Approximate optimal control of affine nonlinear continuous-time systems using event-sampled neurodynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 639–652 (2017)

  44. Liu, D., Yang, X., Wang, D., Wei, Q.: Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Trans. Cybern. 45(7), 1372–1385 (2015)

    Article  Google Scholar 

  45. Wang, H., Liu, P., Bao, J., Xie, X.J., Li, S.: Adaptive neural output-feedback decentralized control for large-scale nonlinear systems with stochastic disturbances. IEEE Trans. Neural Netw. Learn. Syst. 31(3), 972–983 (2020)

    Article  MathSciNet  Google Scholar 

  46. Shankar, S.: Nonlinear Systems. Springer, New York (2007)

    MATH  Google Scholar 

  47. Vamvoudakis, K.G., Miranda, M.F., Hespanha, J.P.: Asymptotically stable adaptive-optimal control algorithm with saturating actuators and relaxed persistence of excitation. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2386–2398 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was jointly supported by the National Natural Science Foundation of China (61773217), the Natural Science Foundation of Hunan Province (2020JJ4054), Hunan Provincial Science and Technology Project Foundation (2019RS1033), Hunan Provincial Science and Technology Project Foundation (2020JJ5344).

Author information

Authors and Affiliations

  1. MOE-LCSM, School of Mathematics and Statistics, Hunan Normal University, Changsha, 410081, China

    Guoping Zhang & Quanxin Zhu

Authors
  1. Guoping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quanxin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quanxin Zhu.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Statement of data

No data were used in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, G., Zhu, Q. Event-triggered optimal control for nonlinear stochastic systems via adaptive dynamic programming. Nonlinear Dyn 105, 387–401 (2021). https://doi.org/10.1007/s11071-021-06624-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-021-06624-8

Keywords

Search

Navigation