Skip to main content
SpringerLink
Log in

Data-driven cooperative optimal output regulation for linear discrete-time multi-agent systems by online distributed adaptive internal model approach

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

In this study, a data-driven learning algorithm was developed to estimate the optimal distributed cooperative control policy, which solves the cooperative optimal output regulation problem for linear discrete-time multi-agent systems. Notably, the dynamics of all the agent systems and exo-system is completely unknown. By combining adaptive dynamic programming with an internal model, a model-free off-policy learning method is proposed to estimate the optimal control gain and the distributed adaptive internal model by only accessing the measurable data of multi-agent systems. Moreover, different from the traditional cooperative adaptive controller design method, a distributed internal model is approximated online. Convergence and stability analyses show that the estimate controller generated by the proposed data-driven learning algorithm converges to the optimal distributed controller. Finally, simulation results verify the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Su Y F, Huang J. Cooperative output regulation of linear multi-agent systems. IEEE Trans Automat Contr, 2012, 57: 1062–1066

    Article  MathSciNet  MATH  Google Scholar 

  2. Su Y F, Huang J. Cooperative output regulation with application to multi-agent consensus under switching network. IEEE Trans Syst Man Cybern B, 2012, 42: 864–875

    Article  Google Scholar 

  3. Dong Y, Huang J. Cooperative global output regulation for a class of nonlinear multi-agent systems. IEEE Trans Automat Contr, 2014, 59: 1348–1354

    Article  MathSciNet  MATH  Google Scholar 

  4. Tran A T, Sakamoto N, Sato M, et al. Control augmentation system design for quad-tilt-wing unmanned aerial vehicle via robust output regulation method. IEEE Trans Aerosp Electron Syst, 2017, 53: 357–369

    Article  Google Scholar 

  5. Zhuan X, Xia X. Speed regulation with measured output feedback in the control of heavy haul trains. Automatica, 2008, 44: 242–247

    Article  MathSciNet  MATH  Google Scholar 

  6. Wang J, Guo Y. Leaderless cooperative control of robotic sensor networks for monitoring dynamic pollutant plumes. IET Control Theor Appl, 2019, 13: 2670–2680

    Article  MathSciNet  Google Scholar 

  7. Roberto C. Blockchain-based distributed cooperative control algorithm for WSN monitoring. In: Proceedings of International Symposium on Distributed Computing and Artificial Intelligence, 2019. 414–417

  8. Francis B A. The linear multivariable regulator problem. SIAM J Control Optim, 1977, 15: 486–505

    Article  MathSciNet  MATH  Google Scholar 

  9. Huang J. The cooperative output regulation problem of discrete-time linear multi-agent systems by the adaptive distributed observer. IEEE Trans Automat Contr, 2017, 62: 1979–1984

    Article  MathSciNet  MATH  Google Scholar 

  10. Cai H, Lewis F L, Hu G, et al. The adaptive distributed observer approach to the cooperative output regulation of linear multi-agent systems. Automatica, 2017, 75: 299–305

    Article  MathSciNet  MATH  Google Scholar 

  11. Liu T, Huang J. Cooperative output regulation for a class of nonlinear multi-agent systems with unknown control directions subject to switching networks. IEEE Trans Automat Contr, 2018, 63: 783–790

    Article  MathSciNet  MATH  Google Scholar 

  12. Dong S, Liu L, Feng G, et al. Cooperative output regulation quadratic control for discrete-time heterogeneous multiagent Markov jump systems. IEEE Trans Cybern, 2021, 52: 9882–9892

    Article  Google Scholar 

  13. Zhang Y, Su Y F. Cooperative output regulation for linear uncertain MIMO multi-agent systems by output feedback. Sci China Inf Sci, 2018, 61: 092206

    Article  MathSciNet  Google Scholar 

  14. Yan Y M, Huang J. Cooperative robust output regulation problem for discrete-time linear time-delay multi-agent systems. Int J Robust Nonlinear Control, 2018, 28: 1035–1048

    Article  MathSciNet  MATH  Google Scholar 

  15. Huang J. Nonlinear Output Regulation: Theory and Applications. Philadelphia: SIAM, 2004

    Book  MATH  Google Scholar 

  16. Wieland P, Sepulchre R, Allgöwer F. An internal model principle is necessary and sufficient for linear output synchronization. Automatica, 2011, 47: 1068–1074

    Article  MathSciNet  MATH  Google Scholar 

  17. Yu W W, Wang H, Hong H F, et al. Distributed cooperative anti-disturbance control of multi-agent systems: an overview. Sci China Inf Sci, 2017, 60: 110202

    Article  MathSciNet  Google Scholar 

  18. Su Y, Hong Y, Huang J. A general result on the robust cooperative output regulation for linear uncertain multi-agent systems. IEEE Trans Automat Contr, 2013, 58: 1275–1279

    Article  MathSciNet  MATH  Google Scholar 

  19. Francis B A, Wonham W M. The internal model principle of control theory. Automatica, 1976, 12: 457–465

    Article  MathSciNet  MATH  Google Scholar 

  20. Su Y, Huang J. Cooperative global output regulation of heterogeneous second-order nonlinear uncertain multi-agent systems. Automatica, 2013, 49: 3345–3350

    Article  MathSciNet  MATH  Google Scholar 

  21. Su Y, Huang J. Cooperative semi-global robust output regulation for a class of nonlinear uncertain multi-agent systems. Automatica, 2014, 50: 1053–1065

    Article  MathSciNet  MATH  Google Scholar 

  22. Yan Y, Chen Z. Cooperative output regulation of linear discrete-time time-delay multi-agent systems by adaptive distributed observers. Neurocomputing, 2019, 331: 33–39

    Article  Google Scholar 

  23. Saberi A, Stoorvogel A A, Sannuti P, et al. On optimal output regulation for linear systems. Int J Control, 2003, 76: 319–333

    Article  MathSciNet  MATH  Google Scholar 

  24. Lee J W, Khargonekar P P. Optimal output regulation for discrete-time switched and markovian jump linear systems. SIAM J Control Optim, 2008, 47: 40–72

    Article  MathSciNet  MATH  Google Scholar 

  25. Ullah S, Liquat M. Optimal output regulation on sample-data systems. In: Proceedings of International Conference on Control, Electronics, Renewable Energy and Communications, Bandung, 2015

  26. Tran A T, Sakamoto N, Kikuchi Y, et al. Pilot induced oscillation suppression controller design via nonlinear optimal output regulation method. Aerospace Sci Tech, 2017, 68: 278–286

    Article  Google Scholar 

  27. Yan Y, Huang J. Cooperative output regulation of discrete-time linear time-delay multi-agent systems under switching network. Neurocomputing, 2017, 241: 108–114

    Article  Google Scholar 

  28. Song X L, Ding F, Xiao F, et al. Data-driven optimal cooperative adaptive cruise control of heterogeneous vehicle platoons with unknown dynamics. Sci China Inf Sci, 2020, 63: 190204

    Article  MathSciNet  Google Scholar 

  29. Gao W, Jiang Z P, Lewis F L, et al. Cooperative optimal output regulation of multi-agent systems using adaptive dynamic programming. In: Proceedings of the 2017 American Control Conference (ACC), Seattle, 2017

  30. Gao W, Liu Y, Odekunle A, et al. Adaptive dynamic programming and cooperative output regulation of discrete-time multi-agent systems. Int J Control Autom Syst, 2018, 16: 2273–2281

    Article  Google Scholar 

  31. Zhang H, Liang H, Wang Z, et al. Optimal output regulation for heterogeneous multiagent systems via adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst, 2017, 28: 18–29

    Article  Google Scholar 

  32. Peng Z N, Hu J P, Ghosh B K. Data-driven containment control of discrete-time multi-agent systems via value iteration. Sci China Inf Sci, 2020, 63: 189205

    Article  MathSciNet  Google Scholar 

  33. Peng Z N, Zhang J F, Hu J P, et al. Optimal containment control of continuous-time multi-agent systems with unknown disturbances using data-driven approach. Sci China Inf Sci, 2020, 63: 209205

    Article  Google Scholar 

  34. Jiang Y, Gao W, Wu J, et al. Reinforcement learning and cooperative H∞ output regulation of linear continuous-time multi-agent systems. Automatica, 2023, 148: 110768

    Article  MathSciNet  MATH  Google Scholar 

  35. Peng Y, Chen Q, Sun W. Reinforcement Q-learning algorithm for H∞ tracking control of unknown discrete-time linear systems. IEEE Trans Syst Man Cybern Syst, 2020, 50: 4109–4122

    Article  Google Scholar 

  36. Liu Y, Zhang H, Yu R, et al. H∞ tracking control of discrete-time system with delays via data-based adaptive dynamic programming. IEEE Trans Syst Man Cybern Syst, 2020, 50: 4078–4085

    Article  Google Scholar 

  37. Fu Y, Fu J, Chai T. Robust adaptive dynamic programming of two-player zero-sum games for continuous-time linear systems. IEEE Trans Neural Netw Learn Syst, 2015, 26: 3314–3319

    Article  MathSciNet  Google Scholar 

  38. Gao W, Jiang Z P, Lewis F L, et al. Leader-to-formation stability of multiagent systems: an adaptive optimal control approach. IEEE Trans Automat Contr, 2018, 63: 3581–3587

    Article  MathSciNet  MATH  Google Scholar 

  39. Gao W, Mynuddin M, Wunsch D C, et al. Reinforcement learning-based cooperative optimal output regulation via distributed adaptive internal model. IEEE Trans Neural Netw Learn Syst, 2021, 33: 5229–5240

    Article  MathSciNet  Google Scholar 

  40. Jiang Y, Fan J, Gao W, et al. Cooperative adaptive optimal output regulation of nonlinear discrete-time multi-agent systems. Automatica, 2020, 121: 109149

    Article  MathSciNet  MATH  Google Scholar 

  41. Gao W, Jiang Y, Davari M. Data-driven cooperative output regulation of multi-agent systems via robust adaptive dynamic programming. IEEE Trans Circuits Syst II, 2019, 66: 447–451

    Google Scholar 

  42. Roy S B, Bhasin S, Kar I N. Combined MRAC for unknown MIMO LTI systems with parameter convergence. IEEE Trans Automat Contr, 2018, 63: 283–290

    Article  MathSciNet  MATH  Google Scholar 

  43. Lancaster P, Rodman L. Algebraic Riccati Equations. New York: Oxford University Press Inc., 1995

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Key R&D Program of China (Grant No. 2021ZD0112600), National Natural Science Foundation of China (Grant Nos. 61873219, 62173283), and Natural Science Foundation of Fujian Province of China (Grant No. 2021J01051).

Author information

Authors and Affiliations

  1. Department of Automation, Xiamen University, Xiamen, 361005, China

    Kedi Xie, Xiao Yu & Weiyao Lan

  2. Department of Electrical Engineering, City University of Hong Kong, Hong Kong, 999077, China

    Yi Jiang

  3. Key Laboratory of Control and Navigation (Xiamen University), Fujian Province University, Xiamen, 361005, China

    Xiao Yu & Weiyao Lan

Authors
  1. Kedi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiyao Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Yu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, K., Jiang, Y., Yu, X. et al. Data-driven cooperative optimal output regulation for linear discrete-time multi-agent systems by online distributed adaptive internal model approach. Sci. China Inf. Sci. 66, 170202 (2023). https://doi.org/10.1007/s11432-022-3687-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3687-1

Keywords

Search

Navigation