Skip to main content
SpringerLink
Log in

Event-triggered receding horizon control via actor-critic design

  • Research Paper
  • Special Focus on Advanced Techniques for Event-Triggered Control and Estimation
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel event-triggered near-optimal control for nonlinear continuoustime systems. The receding horizon principle is utilized to improve the system robustness and obtain better dynamic control performance. In the proposed structure, we first decompose the infinite horizon optimal control into a series of finite horizon optimal problems. Then a learning strategy is adopted, in which an actor network is employed to approximate the cost function and an critic network is used to learn the optimal control law in each finite horizon. Furthermore, in order to reduce the computational cost and transmission cost, an event-triggered strategy is applied. We design an adaptive trigger condition, so that the signal transmissions and controller updates are conducted in an aperiodic way. Detailed stability analysis shows that the nonlinear system with the developed event-triggered optimal control policy is asymptotically stable. Simulation results on a single-link robot arm with different noise types have demonstrated the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Song, R Z, Xiao, W D, Sun, C Y. A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture. Sci China Inf Sci, 2014, 57: 068202

    MATH  Google Scholar 

  2. Li, C H, Zhang, E C, Jiu L, et al. Optimal control on special Euclidean group via natural gradient algorithm. Sci China Inf Sci, 2016, 59: 112203

    Article  Google Scholar 

  3. Wei, W N. Maximum principle for optimal control of neutral stochastic functional differential systems. Sci China Math, 2015, 58: 1265–1284

    Article  MathSciNet  Google Scholar 

  4. Lee, J H. Model predictive control: review of the three decades of development. Int J Control Autom Syst, 2011, 9: 415–424

    Article  Google Scholar 

  5. Bequette, B W. Nonlinear control of chemical processes: a review. Ind Eng Chem Res, 1991, 30: 1391–1413

    Article  Google Scholar 

  6. Mayne, D Q, Rawlings, J B, Rao, C V, et al. Constrained model predictive control: stability and optimality. Automatica, 2000, 36: 789–814

    Article  MathSciNet  Google Scholar 

  7. Hu, L S, Huang B, Cao, Y Y. Robust digital model predictive control for linear uncertain systems with saturations. IEEE Trans Automat Contr, 2004, 49: 792–796

    Article  MathSciNet  Google Scholar 

  8. Evans M, Cannon M, Kouvaritakis B. Robust and stochastic linear MPC for systems subject to multiplicative uncertainty. IFAC Proc Vol, 2012, 45: 335–341

    Article  Google Scholar 

  9. Fang H, Shang, C S, Chen J. An optimization-based shared control framework with applications in multi-robot systems. Sci China Inf Sci, 2018, 61: 014201

    Article  MathSciNet  Google Scholar 

  10. Zhou, L M, Jia L, Wang, Y L. A robust integrated model predictive iterative learning control strategy for batch processes. Sci China Inf Sci, 2019, 62: 219202

    Article  MathSciNet  Google Scholar 

  11. Aguilera, R P, Lezana P, Quevedo, D E. Switched model predictive control for improved transient and steady-state performance. IEEE Trans Ind Inf, 2015, 11: 968–977

    Article  Google Scholar 

  12. Venkat, A N, Hiskens, I A, Rawlings, J B, et al. Distributed MPC strategies with application to power system automatic generation control. IEEE Trans Contr Syst Technol, 2008, 16: 1192–1206

    Article  Google Scholar 

  13. Bertsekas, D P. Dynamic programming and suboptimal control: a survey from ADP to MPC*. Eur J Control, 2005, 11: 310–334

    Article  MathSciNet  Google Scholar 

  14. Wei, Q L, Liu, D R. A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems. Sci China Inf Sci, 2015, 58: 122203

    Article  Google Scholar 

  15. Wang D, Mu, C X. Developing nonlinear adaptive optimal regulators through an improved neural learning mechanism. Sci China Inf Sci, 2017, 60: 058201

    Article  Google Scholar 

  16. Ding, D R, Wang, Z D, Han, Q L, et al. Neural-network-based output-feedback control under round-robin scheduling protocols. IEEE Trans Cybern, 2019, 49: 2372–2384

    Article  Google Scholar 

  17. Ernst D, Glavic M, Capitanescu F, et al. Reinforcement learning versus model predictive control: a comparison on a power system problem. IEEE Trans Syst Man Cybern B, 2009, 39: 517–529

    Article  Google Scholar 

  18. Görges D. Relations between model predictive control and reinforcement learning. IFAC-PapersOnLine, 2017, 50: 4920–4928

    Article  Google Scholar 

  19. Lian, C Q, Xu X, Chen H, et al. Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans Cybern, 2016, 46: 2484–2496

    Article  Google Scholar 

  20. Xu X, Chen H, Lian, C Q, et al. Learning-based predictive control for discrete-time nonlinear systems with stochastic disturbances. IEEE Trans Neural Netw Learn Syst, 2018, 29: 6202–6213

    Article  MathSciNet  Google Scholar 

  21. Dong L, Yan J, Yuan X, et al. Functional nonlinear model predictive control based on adaptive dynamic programming. IEEE Trans Cybern, 2019, 49: 4206–4218

    Article  Google Scholar 

  22. Anta A, Tabuada P. To sample or not to sample: self-triggered control for nonlinear systems. IEEE Trans Automat Contr, 2010, 55: 2030–2042

    Article  MathSciNet  Google Scholar 

  23. Lemmon, M D. Event-triggered feedback in control, estimation, and optimization. In: Networked Control Systems. London: Springer, 2010. 293–358

    Chapter  Google Scholar 

  24. Vamvoudakis, K G. Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J Autom Sin, 2014, 1: 282–293

    Article  Google Scholar 

  25. Duan, G P, Xiao F, Wang L. Hybrid event- and time-triggered control for double-integrator heterogeneous networks. Sci China Inf Sci, 2019, 62: 022203

    Article  MathSciNet  Google Scholar 

  26. Zhang, X M, Han, Q L. Event-triggered dynamic output feedback control for networked control systems. IET Control Theor Appl, 2014, 8: 226–234

    Article  MathSciNet  Google Scholar 

  27. Zhang, B L, Han, Q L, Zhang, X M. Event-triggered H∞ reliable control for offshore structures in network environments. J Sound Vib, 2016, 368: 1–21

    Article  Google Scholar 

  28. Zhang, X M, Han, Q L. A decentralized event-triggered dissipative control scheme for systems with multiple sensors to sample the system outputs. IEEE Trans Cybern, 2016, 46: 2745–2757

    Article  Google Scholar 

  29. Liu, C X, Gao J, Li, H P, et al. Aperiodic robust model predictive control for constrained continuous-time nonlinear systems: an event-triggered approach. IEEE Trans Cybern, 2018, 48: 1397–1405

    Article  Google Scholar 

  30. Li, H P, Shi Y. Event-triggered robust model predictive control of continuous-time nonlinear systems. Automatica, 2014, 50: 1507–1513

    Article  MathSciNet  Google Scholar 

  31. Ding, D R, Han, Q L, Wang, Z D, et al. A survey on model-based distributed control and filtering for industrial cyber-physical systems. IEEE Trans Ind Inf, 2019, 15: 2483–2499

    Article  Google Scholar 

  32. Krichman M, Sontag, E D, Wang Y. Input-output-to-state stability. SIAM J Control Opt, 2001, 39: 1874–1928

    Article  MathSciNet  Google Scholar 

  33. Tabuada P. Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans Automat Contr, 2007, 52: 1680–1685

    Article  MathSciNet  Google Scholar 

  34. Heydari A, Balakrishnan, S N. Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans Neural Netw Learn Syst, 2013, 24: 145–157

    Article  Google Scholar 

  35. Khalil, H K. Nonlinear Systems. New Jersey: Prentice Hall, 2002

    MATH  Google Scholar 

  36. Dong L, Zhong, X N, Sun, C Y, et al. Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Trans Neural Netw Learn Syst, 2017, 28: 1941–1952

    Article  MathSciNet  Google Scholar 

  37. Zhong, X N, He, H B. An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Trans Cybern, 2017, 47: 683–694

    Article  Google Scholar 

  38. Chen H. Model Predictive Control. Beijing: Science Press, 2013

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (Grant Nos. 61803085, 61921004, 61931020) and National Key R&D Program of China (Grant No. 2018AAA0101400).

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Lu Dong, Xin Yuan & Changyin Sun

  2. College of Electronics and Information Engineering, Tongji University, Shanghai, 201804, China

    Lu Dong

  3. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing, 210096, China

    Lu Dong, Xin Yuan & Changyin Sun

Authors
  1. Lu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changyin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changyin Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, L., Yuan, X. & Sun, C. Event-triggered receding horizon control via actor-critic design. Sci. China Inf. Sci. 63, 150210 (2020). https://doi.org/10.1007/s11432-019-2663-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-019-2663-y

Keywords

Search

Navigation