Skip to main content
SpringerLink
Log in

Distributed Consensus Control for Nonlinear Multi-agent Systems

  • Chapter
  • First Online:
Developments in Advanced Control and Intelligent Automation for Complex Systems

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 329))

  • 543 Accesses

Abstract

This chapter considers the distributed optimal consensus problem of discrete-time (DT) nonlinear multi-agent systems (MASs) with unknown dynamics. For this type of system, obtaining a coupled Hamilton–Jacobi–Bellman (HJB) equation is essential to solving the distributed optimal consensus problem. However, it is difficult to solve the coupled HJB equation of a system with unknown dynamics. In this chapter, a local value function is defined that takes into account local consensus errors, the behavior of agents, and the behavior of their neighbors. Based on adaptive dynamic programming (ADP) with the local value function, an action dependent heuristic dynamic programming based distributed consensus control method is put forward to realize the optimal consensus control (OCC). Furthermore, an ADP-based distributed model reference adaptive control method is also presented to achieve OCC for heterogeneous nonlinear MASs. Simulation examples are given to demonstrate the feasibility of the optimal consensus methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

MASs:

Multi-agent systems

ADP:

Adaptive dynamic programming

HJB:

Hamilton–Jacobi–Bellman

RL:

Reinforcement learning

CT:

Continuous-time

DT:

Discrete-time

OCC:

Optimal consensus control

HDP:

Heuristic dynamic programming

ADHDP:

Action-dependent heuristic dynamic programming

NNs:

Neural networks

MRAC:

Model reference adaptive control

LQR:

Linear quadratic regulator

References

  1. Chen, J., Cao, X.H., Cheng, P., Xiao, Y., Sun, Y.X.: Distributed collaborative control for industrial automation with wireless sensor and actuator networks. IEEE Trans. Ind. Electron. 57(12), 4219–4230 (2010)

    Article  Google Scholar 

  2. Zhu, J.D., Lu, J.H., Yu, X.H.: Flocking of multi-agent non-holonomic systems with proximity graphs. IEEE Trans. Circuits Syst. I: Regul. Pap. 60(1), 199–210 (2013)

    Article  MathSciNet  Google Scholar 

  3. Xiao, F., Wang, L., Chen, J., Gao, Y.P.: Finite-time formation control for multi-agent systems. Automatica 45(11), 2605–2611 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Wang, X.H., Yadav, V., Balakrishnan, S.N.: Cooperative UAV formation flying with obstacle/collision avoidance. IEEE Trans. Control Syst. Technol. 15(4), 672–679 (2007)

    Article  Google Scholar 

  5. Wei, Q.L., Liu, D.R., Shi, G., Liu, Y.: Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Trans. Ind. Electron. 62(7), 4203–4214 (2015)

    Article  Google Scholar 

  6. Fax, J.A., Murray, R.M.: Information flow and cooperative control of vehicle formations. IEEE Trans. Autom. Control 49(9), 1465–1476 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  7. Rehan, M., Jameel, A., Ahn, C.K.: Distributed consensus control of one-sided Lipschitz nonlinear multiagent systems. IEEE Trans. Syst. Man Cybern.: Syst. 48(8), 1297–1308 (2018)

    Article  Google Scholar 

  8. Wang, F., Chen, X., He, Y., Wu, M.: Finite-time consensus problem for second-order multi-agent systems under switching topologies. Asian J. Control 19(5), 1756–1766 (2017)

    MathSciNet  MATH  Google Scholar 

  9. Bu, X.H., Yu, Q.X., Hou, Z.S., Qian, W.: Model free adaptive iterative learning consensus tracking control for a class of nonlinear multiagent systems. IEEE Trans. Syst. Man Cybern.: Syst. 49(4), 677–686 (2019)

    Article  Google Scholar 

  10. Meng, W.C., Yang, Q.M., Sarangapani, J., Sun, Y.X.: Distributed control of nonlinear multiagent systems with asymptotic consensus. IEEE Trans. Syst. Man Cybern.: Syst. 47(5), 749–757 (2017)

    Article  Google Scholar 

  11. Liu, W., Huang, J.: Adaptive leader-following consensus for a class of higher-order nonlinear multi-agent systems with directed switching networks. Automatica 79, 84–92 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  12. Movric, K.H., Lewis, F.L.: Cooperative optimal control for multi-agent systems on directed graph topologies. IEEE Trans. Autom. Control 59(3), 769–774 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  13. Vamvoudakis, K.G., Lewis, F.L.: Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica 47(8), 1556–1569 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  14. Werbos, P.: Approximate dynamic programming for realtime control and neural modelling. Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, pp. 493–525 (1992)

    Google Scholar 

  15. Wang, D., Liu, D.R., Li, H.L., Luo, B., Ma, H.W.: An approximate optimal control approach for robust stabilization of a class of discrete-time nonlinear systems with uncertainties. IEEE Trans. Syst. Man Cybern.: Syst. 46(5), 713–717 (2016)

    Article  Google Scholar 

  16. Wei, Q.L., Lewis, F.L., Liu, D.R., Song, R.Z., Lin, H.Q.: Discrete-time local value iteration adaptive dynamic programming: convergence analysis. IEEE Trans. Syst. Man Cybern.: Syst. 48(6), 875–891 (2018)

    Article  Google Scholar 

  17. Wang, Z., Liu, X.P., Liu, K.F., Li, S., Wang, H.Q.: Backstepping-based Lyapunov function construction using approximate dynamic programming and sum of square techniques. IEEE Trans. Cybern. 47(10), 3393–3403 (2017)

    Google Scholar 

  18. Vamvoudakis, K.G., Lewis, F.L., Hudas, G.R.: Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8), 1598–1611 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhong, X.N., He, H.B.: GrHDP solution for optimal consensus control of multiagent discrete-time systems. IEEE Trans. Syst. Man Cybern.: Syst. (2018). https://doi.org/10.1109/TSMC.2018.2814018

  20. Vamvoudakis, K.G.: Q-learning for continuous-time graphical games on large networks with completely unknown linear system dynamics. Int. J. Robust Nonlinear Control 27(16), 2900–2920 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wei, Q.L., Liu, D.R., Lewis, F.L.: Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games. Inf. Sci. 317, 96–113 (2015)

    Article  MATH  Google Scholar 

  22. Zhang, H.G., Zhang, J.L., Yang, G.H., Luo, Y.H.: Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Trans. Fuzzy Syst. 23(1), 152–163 (2015)

    Article  Google Scholar 

  23. Tatari, F., Naghibi-Sistani, M.B., Vamvoudakis, K.G.: Distributed learning algorithm for non-linear differential graphical games. Trans. Inst. Meas. Control 39(2), 173–182 (2017)

    Article  Google Scholar 

  24. Kamalapurkar, R.K., Dinh, H.Y., Walters, P., Dixon, W.: Approximate optimal cooperative decentralized control for consensus in a topological network of agents with uncertain nonlinear dynamics. In: Proceedings of 2013 American Control Conference, pp. 1320–1325 (2013)

    Google Scholar 

  25. Zhang, J.L., Zhang, H.G., Feng, T.: Distributed optimal consensus control for nonlinear multiagent system with unknown dynamic. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3339–3348 (2018)

    Article  MathSciNet  Google Scholar 

  26. Zhang, H.G., Jiang, H., Luo, Y.H., Xiao, G.Y.: Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans. Ind. Electron. 64(5), 4091–4100 (2017)

    Article  Google Scholar 

  27. Abouheaf, M.I., Lewis, F.L., Vamvoudakis, K.G., Haesaert, S., Babuska, R.: Multi-agent discrete-time graphical games and reinforcement learning solutions. Automatica 50(12), 3038–3053 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  28. Li, J.N., Modares, H., Chai, T.Y., Lewis, F.L., Xie, L.H.: Off-policy reinforcement learning for synchronization in multiagent graphical games. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2434–2445 (2017)

    Article  MathSciNet  Google Scholar 

  29. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  30. Murray, J.J., Cox, C.J., Lendaris, G.G., Saeks, R.: Adaptive dynamic programming. IEEE Trans. Syst. Man Cybern. Part C 32(2), 140–153 (2002)

    Article  Google Scholar 

  31. Chen, K.R., Wang, J.W., Zhang, Y., Liu, Z.: Leader-following consensus for a class of nonlinear strick-feedback multiagent systems with state time-delays. IEEE Trans. Syst. Man Cybern.: Syst. (2018). https://doi.org/10.1109/TSMC.2018.2813399

  32. Modares, H., Nageshrao, S.P., Lopes, G.A.D., Babuška, R., Lewis, F.L.: Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning. Automatica 71, 334–341 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  33. Yang, Y.L., Modares, H., Wunsch, D.C., Yin, Y.X.: Leader-follower output synchronization of linear heterogeneous systems with active leader using reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2139–2153 (2018)

    Article  MathSciNet  Google Scholar 

  34. Zhang, H.G., Liang, H.J., Wang, Z.S., Feng, T.: Optimal output regulation for heterogeneous multiagent systems via adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 18–29 (2017)

    Article  Google Scholar 

  35. Zuo, S., Song, Y.D., Lewis, F.L., Davoudi, A.: Optimal robust output containment of unknown heterogeneous multiagent system using off-policy reinforcement learning. IEEE Trans. Cybern. 48(11), 3197–3207 (2018)

    Article  Google Scholar 

  36. Modares, H., Lewis, F.L., Kang, W., Davoudi, A.: Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics. IEEE Trans. Autom. Control 63(1), 117–131 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  37. Kiumarsi, B., Lewis, F.L.: Output synchronization of heterogeneous discrete-time systems: a model-free optimal approach. Automatica 84, 86–94 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Luo, B., Liu, D.R., Wu, H.N., Wang, D., Lewis, F.L.: Policy gradient adaptive dynamic programming for data-based optimal control. IEEE Trans. Cybern. 47(10), 3341–3354 (2017)

    Article  Google Scholar 

  39. Chen, X., Xie, P.H., Xiong, Y.H., He, Y., Wu, M.: Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design. Math. Probl. Eng. (2015)

    Google Scholar 

  40. Wang, W., Chen, X.: Model-free optimal containment control of multi-agent systems based on actor-critic framework. Neurocomputing 314, 242–250 (2018)

    Article  Google Scholar 

  41. Zhao, D.B., Xia, Z.P., Wang, D.: Model-free optimal control for affine nonlinear systems with convergence analysis. IEEE Trans. Autom. Sci. Eng. 12(4), 1461–1468 (2015)

    Article  Google Scholar 

  42. Ari, E.O., Kocaoglan, E.: An SRWNN-based approach on developing a self-learning and self-evolving adaptive control system for motion platforms. Int. J. Control 89(2), 380–396 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  43. Kumar, R., Srivastava, S., Gupta, J.R.P.: Diagonal recurrent neural network based adaptive control of nonlinear dynamical systems using Lyapunov stability criterion. ISA Trans. 67, 407–427 (2017)

    Article  Google Scholar 

  44. Khanesar, M.A., Oniz, Y., Kaynak, O., Gao, H.J.: Direct model reference adaptive fuzzy control of networked SISO nonlinear systems. IEEE/ASME Trans. Mechatron. 21(1), 205–213 (2016)

    Google Scholar 

  45. Wang, N., Sun, Z., Yin, J.C., Zou, Z.J., Su, S.F.: Fuzzy unknown observer-based robust adaptive path following control of underactuated surface vehicles subject to multiple unknowns. Ocean Eng. 176, 57–64 (2019)

    Article  Google Scholar 

  46. Wang, N., Deng, Q., Xie, G.M., Pan, X.X.: Hybrid finite-time trajectory tracking control of a quadrotor. ISA Trans. 90, 278–286 (2019)

    Article  Google Scholar 

  47. Wang, N., Xie, G.M., Pan, X.X., Su, S.F.: Full-state regulation control of asymmetric underactuated surface vehicles. IEEE Trans. Ind. Electron. 66(11), 8741–8750 (2019)

    Article  Google Scholar 

  48. Wang, N., Su, S.F., Pan, X.X., Yu, X., Xie, G.M.: Yaw-guided trajectory tracking control of an asymmetric underactuated surface vehicle. IEEE Trans. Ind. Inform. 15(6), 3502–3513 (2019)

    Article  Google Scholar 

  49. Fu, H., Chen, X., Wang, W.: A model reference adaptive control with ADP-to-SMC strategy for unknown nonlinear systems. In: Proceedings of 2017 11th Asian Control Conference, pp. 1537–1542 (2018)

    Google Scholar 

  50. Zhang, H.G., Feng, T., Liang, H.J., Luo, Y.H.: LQR-based optimal distributed cooperative design for linear discrete-time multiagent systems. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 599–611 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Automation, China University of Geosciences, Wuhan, 430074, China

    Xin Chen & Min Wu

  2. Hubei Key Laboratory of Advance Control and Intelligent Automation for Complex Systems, Wuhan, 430074, China

    Xin Chen & Min Wu

  3. Department of Electrical & Computer Engineering, University of Alberta, Edmonton, AB, T6R 2V4, Canada

    Witold Pedrycz

  4. Institute of Control and Computation Engineering, University of Zielona Gora, 65-516, Zielona Gora, Poland

    Krzysztof Galkowski & Wojciech Paszke

Authors
  1. Xin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Witold Pedrycz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Krzysztof Galkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wojciech Paszke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Chen .

Editor information

Editors and Affiliations

  1. School of Automation, China University of Geosciences, Wuhan, China

    Min Wu

  2. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  3. School of Automation, China University of Geosciences, Wuhan, China

    Luefeng Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, X., Wu, M., Pedrycz, W., Galkowski, K., Paszke, W. (2021). Distributed Consensus Control for Nonlinear Multi-agent Systems. In: Wu, M., Pedrycz, W., Chen, L. (eds) Developments in Advanced Control and Intelligent Automation for Complex Systems. Studies in Systems, Decision and Control, vol 329. Springer, Cham. https://doi.org/10.1007/978-3-030-62147-6_8

Download citation

Publish with us

Policies and ethics

Search

Navigation