Abstract
In this paper, for a class of multi-agent systems with unknown dynamic functions and unknown actuator faults, the optimized finite-time containment control problem based on optimized backstepping technology is investigated. The optimal control strategy is obtained through a simplified reinforcement learning algorithm with the structure of identifier-critic-actor. Based on such a structure, the identifier, the critic and the actor are applied to estimate unknown dynamic functions, evaluate system performance and implement control behavior, respectively. The updating laws of the actor and critic are derived based on the gradient descent method of a simple positive function rather than the square of Bellman residual, which makes the updating laws simpler and eliminates the harsh persistent excitation condition. In addition, in order to eliminate the effect of actuator faults on system stability, a network-based fault observer is constructed to observe online actuator faults. Furthermore, the designed finite-time optimal distributed containment controllers ensure that the followers can ultimately converge to the convex hull composed by leaders within a predetermined finite time. Finally, a numerical simulation result and a practical example are presented to verify the effectiveness of the proposed control method.
Similar content being viewed by others
Data availability statements
Data available on request from the authors.
References
Li, X., Shi, P.: Cooperative fault-tolerant tracking control of heterogeneous hybrid-order mechanical systems with actuator and amplifier faults. Nonlinear Dyn. 98(1), 447–462 (2019)
Zhao, Y., Duan, Z.: Finite-time containment control without velocity and acceleration measurements. Nonlinear Dyn. 82(1), 259–268 (2015)
Liu, D., Liu, Z., Chen, C.L.P., Zhang, Y.: Distributed adaptive fuzzy control approach for prescribed-time containment of uncertain nonlinear multi-agent systems with unknown hysteresis. Nonlinear Dyn. 105(1), 257–275 (2021)
Wang, W., Liang, H., Zhang, Y., Li, T.: Adaptive cooperative control for a class of nonlinear multi-agent systems with dead zone and input delay. Nonlinear Dyn. 96(4), 2707–2719 (2019)
Li, H., Wu, Y., Chen, M., Lu, R.: Adaptive multigradient recursive reinforcement learning event-triggered tracking control for multiagent systems. IEEE Trans. Neural Netw. Learn. Syst. (2021). https://doi.org/10.1109/TNNLS.2021.3090570
Saber, R.O., Murray, R.M.: Consensus Protocols for Networks of Dynamic Agents. IEEE, Piscatway (2003)
Jadbabaie, A., Lin, J., Morse, A.S.: Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Trans. Autom. Control 48(6), 988–1001 (2003)
Ren, W., Cao, Y.: Distributed Coordination of Multi-agent Networks: Emergent Problems, Models, and Issues, vol. 1. Springer, Berlin (2011)
Liang, H., Du, Z., Huang, T., Pan, Y.: Neuroadaptive performance guaranteed control for multiagent systems with power integrators and unknown measurement sensitivity. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3160532
Cacace, F., Mattioni, M., Monaco, S., Celsi, L.R.: Topology-induced containment for general linear systems on weakly connected digraphs. Automatica 131, 109734 (2021)
Gambuzza, L.V., Frasca, M., Sorrentino, F., Pecora, L.M., Boccaletti, S.: Controlling symmetries and clustered dynamics of complex networks. IEEE Trans. Netw. Sci. Eng. 8(1), 282–293 (2020)
Cristofaro, A., Mattioni, M.: Hybrid consensus for multi-agent systems with time-driven jumps. Nonlinear Anal. Hybrid Syst. 43, 101113 (2021)
Gambuzza, L.V., Frasca, M.: Distributed control of multiconsensus. IEEE Trans. Autom. Control 66(5), 2032–2044 (2020)
Li, J., Ren, W., Xu, S.: Distributed containment control with multiple dynamic leaders for double-integrator dynamics using only position measurements. IEEE Trans. Autom. Control 57(6), 1553–1559 (2012)
Qin, H., Chen, H., Sun, Y.: Distributed finite-time fault-tolerant containment control for multiple ocean bottom flying nodes. J. Frankl. Inst. 357(16), 242–264 (2020)
Zeng, H., He, Y., Teo, H.L.: Monotone-delay-interval-based Lyapunov functionals for stability analysis of systems with a periodically varying delay. Automatica 138, 110030 (2022)
Shangguan, X., Zhang, C., He, Y., Jin, L., Jiang, L., Spencer, J.W., Wu, M.: Robust load frequency control for power system considering transmission delay and sampling period. IEEE Trans. Ind. Inform. 17(8), 5292–5303 (2021)
Panteley, E., Loría, A.: Synchronization and dynamic consensus of heterogeneous networked systems. IEEE Trans. Autom. Control 62(8), 3758–3773 (2017)
Wang, C., Wen, C., Hu, Q., Wang, W., Zhang, X.: Distributed adaptive containment control for a class of nonlinear multiagent systems with input quantization. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2419–2428 (2018)
Wang, W., Liang, H., Pan, Y., Li, T.: Prescribed performance adaptive fuzzy containment control for nonlinear multiagent systems using disturbance observer. IEEE Trans. Cybern. 50(9), 3879–3891 (2020)
Mattioni, M., Monaco, S.: Cluster partitioning of heterogeneous multi-agent systems. Automatica 138, 110136 (2022)
Liberzon, D.: Calculus of Variations and Optimal Control Theory. Princeton University Press, Princeton (2011)
Sun, C., Ye, M., Hu, G.: Distributed time-varying quadratic optimization for multiple agents under undirected graphs. IEEE Trans. Autom. Control 62(7), 3687–3694 (2017)
Liu, Y., Geng, Z.: Finite-time optimal formation tracking control of vehicles in horizontal plane. Nonlinear Dyn. 76(1), 481–495 (2014)
Li, Y., Liu, Y., Tong, S.: Observer-based neuro-adaptive optimized control for a class of strict-feedback nonlinear systems with state constraints. IEEE Trans. Neural Netw. Learn. Syst. (2021). https://doi.org/10.1109/TNNLS.2021.3051030
Bellman, R.: Dynamic programming. Science 153(3731), 34–37 (1966)
Werbos, P.: Approximate Dynamic Programming for Realtime Control and Neural Modelling. Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, pp. 493–525. Van Nostrand Reinhold, New York (1992)
Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K.G., Lewis, F.L., Dixon, W.E.: A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49(1), 82–92 (2013)
Wen, G., Chen, C.L.P., Feng, J., Zhou, N.: Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm. IEEE Trans. Fuzzy Syst. 26(5), 2719–2731 (2018)
Wen, G., Chen, C.L.P., Ge, S.S.: Simplified optimized backstepping control for a class of nonlinear strict-feedback systems with unknown dynamic functions. IEEE Trans. Cybern. 51(9), 4567–4580 (2020)
Wang, F., Chen, B., Lin, C., Zhang, J., Meng, X.: Adaptive neural network finite-time output feedback control of quantized nonlinear systems. IEEE Trans. Cybern. 48(6), 1839–1848 (2018)
Li, H., Zhao, S., He, W., Lu, R.: Adaptive finite-time tracking control of full state constrained nonlinear systems with dead-zone. Automatica 100, 99–107 (2019)
Du, P., Pan, Y., Li, H., Lam, H.K.: Nonsingular finite-time event-triggered fuzzy control for large-scale nonlinear systems. IEEE Trans. Fuzzy Syst. 29(8), 2088–2099 (2021)
Shen, Q., Jiang, B., Shi, P., Zhao, J.: Cooperative adaptive fuzzy tracking control for networked unknown nonlinear multiagent systems with time-varying actuator faults. IEEE Trans. Fuzzy Syst. 22(3), 494–504 (2014)
Chen, S., Ho, D.W., Li, L., Liu, M.: Fault-tolerant consensus of multi-agent system with distributed adaptive protocol. IEEE Trans. Cybern. 45(10), 2142–2155 (2015)
Pan, Y., Li, Q., Liang, H., Lam, H.K.: A novel mixed control approach for fuzzy systems via membership functions online learning policy. IEEE Trans. Fuzzy Syst. (2021). https://doi.org/10.1109/TFUZZ.2021.3130201
Guo, X., Xu, W., Wang, J., Park, J.H.: Distributed neuroadaptive fault-tolerant sliding-mode control for 2-D plane vehicular platoon systems with spacing constraints and unknown direction faults. Automatica 129, 109675 (2021)
Guo, X., Xu, W., Wang, J., Park, J.H., Yan, H.: BLF-based neuroadaptive fault-tolerant control for nonlinear vehicular platoon with time-varying fault directions and distance restrictions. IEEE Trans. Intell. Transp. Syst. (2021). https://doi.org/10.1109/TITS.2021.3113928
Jin, Y., Zhang, Y., Jing, Y., Fu, J.: An average dwell-time method for fault-tolerant control of switched time-delay systems and its application. IEEE Trans. Ind. Electron. 66(4), 3139–3147 (2019)
Li, Y.: Finite time command filtered adaptive fault tolerant control for a class of uncertain nonlinear systems. Automatica 106, 117–123 (2019)
Wang, W., Wen, C.: Adaptive compensation for infinite number of actuator failures or faults. Automatica 47(10), 2197–2210 (2011)
Jiang, B., Staroswiecki, M., Cocquempot, V.: Fault accommodation for nonlinear dynamic systems. IEEE Trans. Autom. Control 51(9), 1578–1583 (2006)
Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal Control. Wiley, London (2012)
Zeng, Z., Wang, J., Liao, X.: Global exponential stability of a general class of recurrent neural networks with time-varying delays. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 50(10), 1353–1358 (2003)
Zeng, Z., Wang, J.: Complete stability of cellular neural networks with time-varying delays. IEEE Trans. Circuits Syst. I Regul. Pap. 53(4), 944–955 (2006)
Yang, D., Li, T., Xie, X., Zhang, H.: Event-triggered integral sliding-mode control for nonlinear constrained-input systems with disturbances via adaptive dynamic programming. IEEE Trans. Syst. Man Cybern. Syst. 50(11), 4086–4096 (2020)
Jia, T., Pan, Y., Liang, H., Lam, H.K.: Event-based adaptive fixed-time fuzzy control for active vehicle suspension systems with time-varying displacement constraint. IEEE Trans. Fuzzy Syst. (2021). https://doi.org/10.1109/TFUZZ.2021.3075490
Pan, Y., Wu, Y., Lam, H.K.: Security-based fuzzy control for nonlinear networked control systems with DoS attacks via a resilient event-triggered scheme. IEEE Trans. Fuzzy Syst. (2022). https://doi.org/10.1109/TFUZZ.2022.3148875
Zhang, D., Ye, Z., Feng, G., Li, H.: Intelligent event-based fuzzy dynamic positioning control of nonlinear unmanned marine vehicles under DoS attack. IEEE Trans. Cybern. (2021). https://doi.org/10.1109/TCYB.2021.3128170
Funding
This work was partially supported by the National Natural Science Foundation of China (62003052).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
To prove Theorem 1, the following steps are given.
Step 1: According to the system (1), the derivative of the containment error \({z}_{i,1}\) can be rewritten as follows
Then, \({{\tilde{\varXi }}}_{f_{i,1}}\) \(={{\hat{\varXi }}}_{f_{i,1}}-{\varXi }_{f_{i,1}}^{*}, {\tilde{\varXi }}_{c_{i,1}}={\hat{\varXi }}_{c_{i,1}}-\varXi _{J_{i,1}}^{*}\) and \({\tilde{\varXi }} _{a_{i,1}}={\hat{\varXi }}_{a_{i,1}}-\varXi _{J_{i,1}}^{*}\) are defined as the identifier, critic, and actor NN weight errors, and the Lyapunov function candidate for subsystem \(z_{i,1}\) is selected as follows
Then, the time derivative of \(V_{i,1}\) is given by
Furthermore, \({\dot{V}}_{i,1}\) along (12), (14) and (16) is as follows
Based on (7) and (15), one has
By adopting Young’s inequality, the following inequalities hold
Substituting the inequalities (55) into (54), one has
On the basis of \({\tilde{\varXi }}_{f_{i,1}}={\hat{\varXi }}_{f_{i,1}}-\varXi _{f_{i,1}}^{*}\), we can obtain
Similarly, owing to \({\tilde{\varXi }}_{{{\bar{o}}}_{i,1}}={\hat{\varXi }}_{{{\bar{o}}}_{i,1}}-\varXi ^*_{{{\bar{o}}}_{i,1}}\) with \({{\bar{o}}}=a,c\), the following equation can be derived
Substituting (57) and (58) into (56), one gets
By using Young’s inequality, one has
Further, it yields
Based on (17), (59) can be rewritten as
where \(c_{i,1}=(\gamma _{c_{i,1}}/2+\gamma _{a_{i,1}}/2)( \varXi _{J_{i,1}}^{*T}S_{J_{i,1}}) ^{2}+\varepsilon _{f_{i,1}}^{2}/2 + (\sigma _{i,1}/2)\varXi _{f_{i,1}}^{*T}\varXi _{f_{i,1}}^{*} \). Since all terms in \(c_{i,1}\) are bounded, there exists a positive constant \(C_{i,1}\) such that \(|c_{i,1}(t)| \le C_{i,1}\).
Denote \(\zeta _{\varGamma _{i,1}^{-1}}^{\max }\) and \(\zeta _{S_{J_{i,1}}}^{\min }\) as the maximum eigenvalue and the minimum eigenvalue of \(\varGamma _{i,1}^{-1}\) and \(S_{J_{i,1}}S_{J_{i,1}}^T\), respectively. Then, one has
Substituting (61) into (60), we obtain
Step m (\(2\le m\le n-1\)): The Lyapunov function of the mth subsystem is defined as follows
Then, the derivative of Lyapunov candidate function \(V_{i,m}\) is as follows
Using Young’s inequality, the following facts hold
Owing to the fact that \({\tilde{\varXi }}_{o_{i,m}}={\hat{\varXi }}_{o_{i,m}}-\varXi _{o_{i,m}}^{*}\) with \(o=f,a,c\) and based on Young’s inequality, we have
By using Young’s inequality, one has
Substituting the above equations into (62), it yields
where \(c_{i,m}=\frac{\gamma _{c_{i,m}}}{2}( \varXi _{J_{i,m}}^{*T}S_{J_{i,m}}) ^{2}+\frac{\gamma _{a_{i,m}}}{2}( \varXi _{J_{i,m}}^{*T}S_{J_{i,m}}) ^{2}+\frac{\sigma _{i,m}}{2} \varXi _{f_{i,m}}^{*T}\varXi _{f_{i,m}}^{*}+\frac{1}{2}\varepsilon _{f_{i,m}}^{2}\) is a bounded function by a positive constant \(C_{i,m}\), that is, \(|c_{i,m}(t)|\le C_{i,m}\), \(\zeta _{\varGamma _{i,m}^{-1}}^{\max }\) and \(\zeta _{S_{J_{i,m}}}^{\min }\) are denoted as the maximum eigenvalue and the minimum eigenvalue of \(\varGamma _{i,m}^{-1}\) and \(S_{J_{i,m}}S_{J_{i,m}}^T\), respectively.
Similar to Step 1 and based on the condition (30), (63) becomes
Step n: In the last Step, we choose the following Lyapunov function
where \({{\tilde{\varXi }}}_{f_{i,n}}\) \(={{\hat{\varXi }}}_{f_{i,n}}-{\varXi }_{f_{i,n}}^{*} ,{\tilde{\varXi }}_{c_{i,n}}={\hat{\varXi }}_{c_{i,n}}-\varXi _{c_{i,n}}^{*}\), \({\tilde{\varXi }} _{a_{i,n}}={\hat{\varXi }}_{a_{i,n}}-\varXi _{a_{i,n}}^{*}\) and \({\tilde{u}}_{f_{i}}\) \(={u}_{f_{i}}-{{\hat{u}}}_{f_{i}}\).
According to (47), the time derivative of \(V_{i,n}\) is given by
Substituting (49) into (64), one has
where \(F_{i,n}=f_{i,n}-\dot{{\hat{\alpha }}}_{i,n-1}^{*}+(5/4)z_{i,n}\).
Then, according to Young’s inequality and Assumption 2, we can obtain
where \(\varpi _{i,1}\) and \(\varpi _{i,2}\) are constants.
Combining (49) and (65)–(67), and similar to Step 1, (65) yields
Next, the proof of Theorem 1 (1) is given as follows. The total Lyapunov function candidate is selected as
Then, the time derivative of V is given by
where \(D_i=\sum _{k=1}^{n}C_{i,k}+\frac{\gamma _{t_{i}}}{2}\varpi ^2_{i,1}+\frac{1}{2}\varpi ^2_{i,2}\) is a constant.
Moreover, by Lemma 4 and denoting \(p=1,q={\tilde{\varXi }}_{f_{i, k}}^{T} \varGamma _{i, k}^{-1} {\tilde{\varXi }}_{f_{i, k}}\), and \(\imath =1-l\), \(\jmath =l\), \(\tau =l^{\frac{l}{ 1-l}}\), the following inequality holds
Similar to (69), we can get
Remark 3
The above inequalities are based on Lemma 3 and Lemma 4, which are proposed in [40] and are common in the results related to finite-time control. These lemmas are used to ensure that the power of system weight vector errors and the fault observation error meets the requirement of finite-time control.
Further, (68) becomes
where \(a=\min \{-2^l\beta _{i,k},\frac{\sigma _{i,k}}{2{\zeta _{{\varGamma _{i,k}^{-1}}}^{\max }}}, \frac{\gamma _{c_{i,k}}}{2}{\zeta _{{S_{{J_{i,k}}}}}^{\min }},\frac{\gamma _{t_i}-1}{2}\}\) and \(b=\min \{ \frac{\sigma _{i,k}}{2{\zeta _{{\varGamma _{i,k}^{-1}}}^{\max }}}\tau l, \frac{\gamma _{c_{i,k}}}{2}{\zeta _{{S_{{J_{i,k}}}}}^{\min }}\tau l, D_{i,k},\frac{\gamma _{t_i}-1}{2}\tau l\}\).
Define \(T^R=[V^{1-l}z(0)-([(b/(1-p)a)])^{(1-l)/l}][1/(1-l)pa]\) and \(p\in (0,1)\), then, on the basis of Lemma 2, \(\forall t\ge T^R, V^l\le [b/(1-p)a]\) holds, i.e., all signals in the closed-loop system are SGPFS.
In addition, according to the definition of V in this section, one has
where \({\underline{z}}=[z_{1,1},z_{2,1},\ldots ,z_{Q,1}]^{T}\!\in \! {\mathbb {R}}^{Q}\). Denote \({\underline{y}}\! =\! [y_{1},\ldots ,y_{Q}]^{T}\in {\mathbb {R}}^{N}\) and \({\underline{y}}_{d}=[y_{Q+1,d},\ldots ,y_{Q+P,d}]^{T}\in {\mathbb {R}}^{P}\), then, the following fact holds
Therefore, one gets
where \(\lambda _{min}({\mathbb {L}}_{1})\) represents the minimum eigenvalue of the matrix \({\mathbb {L}}_{1}\). From formula (71), it means that all followers eventually converge to the convex hull space formed by multiple leaders, and the goal of containment control is achieved.
Thus, the proof of Theorem 1 is completed.
Rights and permissions
About this article
Cite this article
Cui, J., Pan, Y., Xue, H. et al. Simplified optimized finite-time containment control for a class of multi-agent systems with actuator faults. Nonlinear Dyn 109, 2799–2816 (2022). https://doi.org/10.1007/s11071-022-07586-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11071-022-07586-1