Skip to main content
SpringerLink
Log in

Distributed observer-based hierarchical optimal consensus tracking with dynamic event-triggered adaptive dynamic programming

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Constraints on the communication topology bring challenges to optimal consensus protocol design. This paper investigates an observer-based optimal consensus tracking problem with a dynamic event-triggered mechanism for nonlinear multi-agent systems. Followers have access to only part of the states of the leader. The control process is completed hierarchically with two layers. In the distributed observer layer, an extended distributed observer is designed for each follower to reconstruct states of the leader by exchanging local estimated information with its neighbours. Parameters are selected step by step instead of solving linear matrix inequalities, guaranteeing the solvability of the observer. In the decentralized protocol layer, a dynamic event-triggered adaptive dynamic programming algorithm is designed for tracking the observer state, which serves as a reference. Inter-event time slots are enlarged by introducing a positive inner variable with first-order filter dynamics. Asymptotic stability of the tracking error and ultimately uniformly bounded property are proven rigorously. Finally, an illustrative example validates the effectiveness of the proposed design.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Notes

  1. One can see \(\varPsi _{i}=\{\gamma _{i},K_{i},M_{i}\}\) in Sect. 3.

References

  1. Li, X., Chen, C., Xu, Q., Wen, C.: Resilience for communication faults in reactive power sharing of microgrids. IEEE Trans. Smart Grid 12(4), 2788–2799 (2021)

    Google Scholar 

  2. Zhao, W., Liu, H., Lewis, F.L.: Data-driven fault-tolerant control for attitude synchronization of nonlinear quadrotors. IEEE Trans. Autom. Control 66(11), 5584–5591 (2021)

    MathSciNet  MATH  Google Scholar 

  3. Hua, T., Xiao, J.-W., Liu, X.-K., Lei, Y., Wang, Y.-W.: Event-triggered sub-optimal control for two-time-scale systems with unknown dynamics. Nonlinear Dyn. 1–14 (2022)

  4. Zhang, K., Su, R., Zhang, H., Tian, Y.: Adaptive resilient event-triggered control design of autonomous vehicles with an iterative single critic learning framework. IEEE Trans. Neural Netw. Learn. Syst. 32(12), 5502–5511 (2021)

    MathSciNet  Google Scholar 

  5. Xu, J., Wang, L., Liu, Y., Xue, H.: Event-triggered optimal containment control for multi-agent systems subject to state constraints via reinforcement learning, Nonlinear Dyn. 1–20 (2022)

  6. Zhao, B., Jia, L., Xia, H., Li, Y.: Adaptive dynamic programming-based stabilization of nonlinear systems with unknown actuator saturation. Nonlinear Dyn. 93(4), 2089–2103 (2018)

    MATH  Google Scholar 

  7. Mu, C., Liao, K., Wang, K.: Event-triggered design for discrete-time nonlinear systems with control constraints. Nonlinear Dyn. 103(3), 2645–2657 (2021)

    Google Scholar 

  8. Chen, Z., Chen, S.-Z., Chen, K., Zhang, Y.: Constrained decoupling adaptive dynamic programming for a partially uncontrollable time-delayed model of energy systems. Inf. Sci. 608, 1352–1374 (2022)

    Google Scholar 

  9. Liu, D., Xue, S., Zhao, B., Luo, B., Wei, Q.: Adaptive dynamic programming for control: a survey and recent advances. IEEE Trans. Syst. Man Cybern. Syst. 51(1), 142–160 (2021)

    Google Scholar 

  10. Lewis, F.L., Vrabie, D.: Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst. Mag. 9(3), 32–50 (2009)

    Google Scholar 

  11. Vamvoudakis, K.G., Lewis, F.L., Hudas, G.R.: Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8), 1598–1611 (2012)

    MathSciNet  MATH  Google Scholar 

  12. Zhang, H., Zhang, J., Yang, G.-H., Luo, Y.: Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Trans. Fuzzy Syst. 23(1), 152–163 (2015)

    Google Scholar 

  13. Zhang, J., Zhang, H., Feng, T.: Distributed optimal consensus control for nonlinear multiagent system with unknown dynamic. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3339–3348 (2018)

    MathSciNet  Google Scholar 

  14. Wang, H., Li, M.: Model-free reinforcement learning for fully cooperative consensus problem of nonlinear multiagent systems. IEEE Trans. Neural Netw. Learn. Syst. 33(4), 1482–1491 (2022)

    MathSciNet  Google Scholar 

  15. Shi, J., Yue, D., Xie, X.: Optimal leader-follower consensus for constrained-input multiagent systems with completely unknown dynamics. IEEE Trans. Syst. Man Cybern. Syst. 52(2), 1182–1191 (2022)

    Google Scholar 

  16. Jiao, Q., Modares, H., Xu, S., Lewis, F.L., Vamvoudakis, K.G.: Multi-agent zero-sum differential graphical games for disturbance rejection in distributed control. Automatica 69, 24–34 (2016)

    MathSciNet  MATH  Google Scholar 

  17. Abouheaf, M.I., Lewis, F.L., Vamvoudakis, K.G., Haesaert, S., Babuska, R.: Multi-agent discrete-time graphical games and reinforcement learning solutions. Automatica 50(12), 3038–3053 (2014)

    MathSciNet  MATH  Google Scholar 

  18. Peng, Z., Luo, R., Hu, J., Shi, K., Nguang, S.K., Ghosh, B.K.: Optimal tracking control of nonlinear multiagent systems using internal reinforce Q-learning. IEEE Trans. Neural Netw. Learn. Syst. 33(8), 4043–4055 (2022)

    MathSciNet  Google Scholar 

  19. Lopez, V.G., Lewis, F.L., Wan, Y., Liu, M., Hewer, G., Estabridis, K.: Stability and robustness analysis of minmax solutions for differential graphical games. Automatica 121, 109177 (2020)

    MathSciNet  MATH  Google Scholar 

  20. Liu, M., Wan, Y., Lopez, V.G., Lewis, F.L., Hewer, G.A., Estabridis, K.: Differential graphical game with distributed global nash solution. IEEE Trans. Control Netw. Syst. 8(3), 1371–1382 (2021)

    MathSciNet  Google Scholar 

  21. Qian, Y.-Y., Liu, M., Wan, Y., Lewis, F.L., Davoudi, A.: Distributed adaptive nash equilibrium solution for differential graphical games, IEEE Trans. Cybern. 1–13 (2021)

  22. Cai, H., Su, Y., Huang, J.: “Cooperative output regulation of linear multi-agent systems by distributed observer approach,” in Cooperative Control of Multi-agent Systems. Springer, pp. 231–257 (2022)

  23. Fu, H., Chen, X., Wang, W., Wu, M.: Observer-based adaptive synchronization control of unknown discrete-time nonlinear heterogeneous systems. IEEE Trans. Neural Netw. Learn. Syst. 33(2), 681–693 (2022)

    MathSciNet  Google Scholar 

  24. Jiang, Y., Fan, J., Gao, W., Chai, T., Lewis, F.L.: Cooperative adaptive optimal output regulation of nonlinear discrete-time multi-agent systems. Automatica 121, 109149 (2020)

    MathSciNet  MATH  Google Scholar 

  25. Han, W., Trentelman, H.L., Wang, Z., Shen, Y.: A simple approach to distributed observer design for linear systems. IEEE Trans. Autom. Control 64(1), 329–336 (2019)

    MathSciNet  MATH  Google Scholar 

  26. Silm, H., Efimov, D., Michiels, W., Ushirobira, R., Richard, J.-P.: A simple finite-time distributed observer design for linear time-invariant systems. Syst. Control Lett. 141, 104707 (2020)

    MathSciNet  MATH  Google Scholar 

  27. Xu, H., Wang, J., Wang, B., Brahmia, I.: Distributed observer design for linear systems to achieve omniscience asymptotically under jointly connected switching networks. IEEE Trans. Cybern. 1–12 (2021)

  28. Deng, C., Wen, C., Huang, J., Zhang, X.-M., Zou, Y.: Distributed observer-based cooperative control approach for uncertain nonlinear mass under event-triggered communication. IEEE Trans. Autom. Control 67(5), 2669–2676 (2022)

    MathSciNet  MATH  Google Scholar 

  29. Tabuada, P.: Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans. Autom. Control 52(9), 1680–1685 (2007)

    MathSciNet  MATH  Google Scholar 

  30. Vamvoudakis, K.G., Ferraz, H.: Model-free event-triggered control algorithm for continuous-time linear systems with optimal performance. Automatica 87, 412–420 (2018)

    MathSciNet  MATH  Google Scholar 

  31. Mu, C., Wang, D., Sun, C., Zong, Q.: Robust adaptive critic control design with network-based event-triggered formulation. Nonlinear Dyn. 90(3), 2023–2035 (2017)

    MathSciNet  MATH  Google Scholar 

  32. Zhang, Y., Zhao, B., Liu, D., Zhang, S.: Event-triggered control of discrete-time zero-sum games via deterministic policy gradient adaptive dynamic programming. IEEE Trans. Syst. Man Cybern. Syst. 52(8), 4823–4835 (2022)

    Google Scholar 

  33. Wang, J., Wang, Y., Ji, Z.: Model-free event-triggered optimal control with performance guarantees via goal representation heuristic dynamic programming. Nonlinear Dyn. 1–16 (2022)

  34. Chen, Z., Chen, K., Chen, S.-Z., Zhang, Y.: Event-triggered H\(_{\infty }\) consensus for uncertain nonlinear systems using integral sliding mode based adaptive dynamic programming. Neural Netw. 156, 258–270 (2022)

    Google Scholar 

  35. Girard, A.: Dynamic triggering mechanisms for event-triggered control. IEEE Trans. Autom. Control 60(7), 1992–1997 (2015)

    MathSciNet  MATH  Google Scholar 

  36. Yang, Y., Vamvoudakis, K.G., Modares, H., Yin, Y., Wunsch, D.C.: Safe intermittent reinforcement learning with static and dynamic event generators. IEEE Trans. Neural Netw. Learn. Syst. 31(12), 5441–5455 (2020)

    MathSciNet  Google Scholar 

  37. Yang, X., Xu, M., Wei, Q.: Dynamic event-sampled control of interconnected nonlinear systems using reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 1–15 (2022)

  38. Ming, Z., Zhang, H., Luo, Y., Wang, W.: Dynamic event-based control for stochastic optimal regulation of nonlinear networked control systems. IEEE Trans. Neural Netw. Learn. Syst. 1–10 (2022)

  39. Gao, W., Jiang, Z.-P., Lewis, F.L., Wang, Y.: Leader-to-formation stability of multiagent systems: an adaptive optimal control approach. IEEE Trans. Autom. Control 63(10), 3581–3587 (2018)

    MathSciNet  MATH  Google Scholar 

  40. Ding, L., Han, Q.-L., Ge, X., Zhang, X.-M.: An overview of recent advances in event-triggered consensus of multiagent systems. IEEE Trans. Cybern. 48(4), 1110–1123 (2018)

    Google Scholar 

  41. Abouheaf, M.I., Lewis, F.L., Mahmoud, M.S., Mikulski, D.G.: Discrete-time dynamic graphical games: model-free reinforcement learning solution. Control Theory Technol. 13(1), 55–69 (2015)

    MathSciNet  MATH  Google Scholar 

  42. Kailath, T.: Linear systems, vol. 156. Prentice-Hall Englewood Cliffs, NJ (1980)

  43. Fu, H., Chen, X., Wu, M.: Distributed optimal observer design of networked systems via adaptive critic design. IEEE Trans. Syst. Man Cybern. Syst. 51(11), 6976–6985 (2021)

    Google Scholar 

  44. Sun, Y., Liu, J., Gao, Y., Liu, Z., Zhao, Y.: Adaptive neural tracking control for manipulators with prescribed performance under input saturation. IEEE/ASME Trans. Mechatron. 1–10 (2022)

  45. Wu, L., Liu, J., Vazquez, S., Mazumder, S.K.: Sliding mode control in power converters and drives: a review. IEEE/CAA J. Autom. Sin. 9(3), 392–406 (2022)

    Google Scholar 

  46. Li, Z., Duan, Z.: Cooperative control of multi-agent systems: a consensus region approach. CRC Press, Boca Raton (2017)

    Google Scholar 

  47. Horn, R.A., Johnson, C.R.: Matrix analysis. Cambridge University Press, Cambridge (2012)

    Google Scholar 

  48. Yu, W., Chen, G., Cao, M., Kurths, J.: Second-order consensus for multiagent systems with directed topologies and nonlinear dynamics. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(3), 881–891 (2010)

    Google Scholar 

  49. Kim, T., Lee, C., Shim, H.: Completely decentralized design of distributed observer for linear systems. IEEE Trans. Autom. Control 65(11), 4664–4678 (2020)

    MathSciNet  MATH  Google Scholar 

  50. Huang, J.: Nonlinear Output Regulation: Theory and Applications. SIAM (2004)

  51. Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal Control. John Wiley & Sons, New Jersey (2012)

    MATH  Google Scholar 

  52. Liu, Z., Zhang, O., Gao, Y., Zhao, Y., Sun, Y., Liu, J.: Adaptive neural network-based fixed-time control for trajectory tracking of robotic systems, IEEE Trans. Circ. Syst. II Express Briefs 1–1 (2022)

  53. Finlayson, B.A.: The method of weighted residuals and variational principles. SIAM (2013)

  54. Lewis, F., Jagannathan, S., Yesildirak, A.: Neural Network Control of Robot Manipulators and Non-linear Systems. CRC Press, Boca Raton (2020)

    Google Scholar 

  55. Zhang, Y., Zhao, B., Liu, D., Zhang, S.: Event-triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs. Int. J. Robust Nonlinear Control 32(1), 29–51 (2022)

    MathSciNet  Google Scholar 

  56. Aubin, J.-P.: Applied Functional Analysis. John Wiley & Sons, New Jersey (2011)

    Google Scholar 

  57. Xue, S., Luo, B., Liu, D., Yang, Y.: Constrained event-triggered H\(_{\infty }\) control based on adaptive dynamic programming with concurrent learning. IEEE Trans. Syst. Man Cybern. Syst. 52(1), 357–369 (2022)

    Google Scholar 

  58. Khalil, H.K.: Nonlinear Control, vol. 406. Pearson, New York (2015)

    Google Scholar 

  59. Lewis, F.L., Yesildirak, A., Jagannathan, S.: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor & Francis Inc, USA (1998)

    Google Scholar 

  60. Zhao, Q., Sun, J., Wang, G., Chen, J.: Event-triggered adp for nonzero-sum games of unknown nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 33(5), 1905–1913 (2022)

    Google Scholar 

Download references

Acknowledgements

The authors thank the Associate Editor and anonymous reviewers for their feedback that has improved the quality of this work. Zitao Chen thanks Professor Seung Yeal Ha at Seoul National University and Dr. Jaeyoung Lee at University of Waterloo for topical and intellectual discussions about the research.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62103115, in part by the Natural Science Foundation of Guang-dong Province under 2021A1515011636, and the Science and Technology Research Program of Guangzhou 202102020975. Zitao Chen was supported in part by China Scholarship Council (CSC Number: CSC202208440314).

Author information

Authors and Affiliations

  1. School of Automation, Guangdong University of Technology, Guangzhou, 510006, China

    Zitao Chen & Yun Zhang

  2. Department of Mathematical Sciences, Seoul National University, Seoul, 08826, South Korea

    Zitao Chen

  3. School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou, 510006, China

    Kairui Chen

Authors
  1. Zitao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kairui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kairui Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Chen, K. & Zhang, Y. Distributed observer-based hierarchical optimal consensus tracking with dynamic event-triggered adaptive dynamic programming. Nonlinear Dyn 111, 12319–12337 (2023). https://doi.org/10.1007/s11071-023-08496-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-023-08496-6

Keywords

Search

Navigation