Soft Computing

, Volume 22, Issue 8, pp 2705–2715 | Cite as

Decentralized adaptive optimal stabilization of nonlinear systems with matched interconnections

  • Chaoxu Mu
  • Changyin Sun
  • Ding Wang
  • Aiguo Song
  • Chengshan Qian
Methodologies and Application


In this paper, we investigate the decentralized feedback stabilization and adaptive dynamic programming (ADP)-based optimization for the class of nonlinear systems with matched interconnections. The decentralized control law of the overall system is designed by integrating all controllers of the isolated subsystems, and it satisfies the optimality on the basis of optimal control laws of all the subsystems. For solving the optimal control problems of these isolated subsystems, the policy iteration algorithm is used to approximately solve the Hamilton–Jacobi–Bellman equations in the framework of ADP with the neural network implementation, where a set of critic neural networks is constructed to estimate the optimal cost functions, and the approximate optimal control laws can be obtained after the learning of critic neural networks. The weight estimation errors of the critic networks and the stability of all isolated subsystems are proved based on the Lyapunov theory. Finally, the performance of the proposed decentralized optimal control strategy is verified by simulation results.


Adaptive dynamic programming (ADP) Interconnected nonlinear systems Neural networks Decentralized control Matched interconnections 



This work was supported by National Natural Science Foundation of China under Grants 61304018, 61304086, 61533017, 61533008, 61520106009, and U1501251, China Postdoctoral Science Foundation under Grant 2014M561559, Tianjin Natural Science Foundation under Grant 14JCQNJC05400, Beijing Natural Science Foundation under Grant 4162065, Tianjin Key Laboratory of Process Measurement and Control under Grant TKLPMC-201612, and the Early Career Development Award of SKLMCCS.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animals rights

This paper does not contain any studies with human participants or animals performed by any of the authors.


  1. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41:779–791MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bakule L (2008) Decentralized control: an overview. Ann Rev Control 32:87–98CrossRefGoogle Scholar
  3. Beard RW, Saridis GN, Wen JT (1997) Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33:2159–2177MathSciNetCrossRefzbMATHGoogle Scholar
  4. Gu B, Sheng VS, Tay KY, Romano W, Li S (2015a) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26:1403–1416MathSciNetCrossRefGoogle Scholar
  5. Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015b) Incremental learning for \(\nu \)-support vector regression. Neural Netw 67:140–150CrossRefGoogle Scholar
  6. Hou ZG, Gupta MM, Nikiforuk PN, Tan M, Cheng L (2007) A recurrent neural network for hierarchical control of interconnected dynamic systems. IEEE Trans Neural Netw 18:466–481CrossRefGoogle Scholar
  7. Jiang Y, Jiang ZP (2015) Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans Autom Control 60:2917–2929MathSciNetCrossRefzbMATHGoogle Scholar
  8. Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32:76–105MathSciNetCrossRefGoogle Scholar
  9. Liu D, Li H, Wang D (2013) Data-based self-learning optimal control: research progress and prospects. Acta Autom Sin 39:1858–1870MathSciNetCrossRefzbMATHGoogle Scholar
  10. Liu D, Wang D, Li H (2014) Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans Neural Netw Learn Syst 25:418–428CrossRefGoogle Scholar
  11. Luo B, Wu HN, Huang T, Liu D (2014) Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50:3281–3290MathSciNetCrossRefzbMATHGoogle Scholar
  12. Mehraeen S, Jagannathan S (2011) Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton-Jacobi-Bellman formulation. IEEE Trans Neural Netw 22:1757–1769CrossRefGoogle Scholar
  13. Modares H, Lewis FL, Naghibi-Sistani MB (2013) Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans Neural Netw Learn Syst 24:1513–1525CrossRefGoogle Scholar
  14. Mu C, Ni Z, Sun C, He H (2016) Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans Cybern. doi: 10.1109/TCYB.2016.2548941 (in press)
  15. Mu C, Ni Z, Sun C, He H (2017a) Air-breathing hypersonic vehicle track ing control based on adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst 28(3):584–598Google Scholar
  16. Mu C, Wang D, He H (2017b) Novel iterative neural dynamic programming for data-based approximate optimal control design. Automatica. doi: 10.1016/j.automatica.2017.03.022 (in press)
  17. Na J, Herrmann G (2014) Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous time unknown nonlinear systems. IEEE/CAA J Autom Sin 1:412–422CrossRefGoogle Scholar
  18. Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17:2101–2108CrossRefGoogle Scholar
  19. Panagi P, Polycarpou MM (2011) Decentralized fault tolerant control of a class of interconnected nonlinear systems. IEEE Trans Autom Control 56:178–184MathSciNetCrossRefzbMATHGoogle Scholar
  20. Prokhorov DV, Wunsch DC (1997) Adaptive critic designs. IEEE Trans Neural Netw 8:997–1007CrossRefGoogle Scholar
  21. Rice JK, Verhaegen M (2009) Distributed control: a sequentially semi-separable approach for spatially heterogeneous linear systems. IEEE Trans Autom Control 54:1270–1283MathSciNetCrossRefzbMATHGoogle Scholar
  22. Saberi A (1998) On optimality of decentralized control for a class of nonlinear interconnected systems. Automatica 24:101–104MathSciNetCrossRefzbMATHGoogle Scholar
  23. Siljak DD (2012) Decentralized control of complex systems. Academic Press, BostonzbMATHGoogle Scholar
  24. Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, CambridgeGoogle Scholar
  25. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888Google Scholar
  26. Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4:39–47CrossRefGoogle Scholar
  27. Wang D, Liu D, Wei Q, Zhao D, Jin N (2012) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48:1825–1832MathSciNetCrossRefzbMATHGoogle Scholar
  28. Wang D, Liu D, Li H (2014a) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11:627–632CrossRefGoogle Scholar
  29. Wang D, Liu D, Li H, Ma H (2014b) Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Inf Sci 282:167–179MathSciNetCrossRefzbMATHGoogle Scholar
  30. Wang D, Liu D, Li H, Ma H, Li C (2016a) A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization. Soft Comput 20:707–716CrossRefzbMATHGoogle Scholar
  31. Wang D, Liu D, Ma H, Mu C (2016b) Decentralized guaranteed cost control of interconnected systems with uncertainties: a learning-based optimal control strategy. Neurocomputing 214:297–306CrossRefGoogle Scholar
  32. Wang D, Mu C, Liu D (2017) Data-driven nonlinear near-optimal regulation based on iterative neural dynamic programming. Acta Autom Sin 43:354–363Google Scholar
  33. Werbos PJ (2008) ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans Syst Man Cybern B Cybern 38:898–900CrossRefGoogle Scholar
  34. Yang X, Liu D, Ma H, Xu Y (2016) Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inf Sci 328:435–454CrossRefGoogle Scholar
  35. Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20:1490–1503CrossRefGoogle Scholar
  36. Zhao D, Zhu Y (2015) MEC-A near-optimal online reinforcement learning algorithm for continuous deterministic systems. IEEE Trans Neural Netw Learn Syst 26:346–356MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Chaoxu Mu
    • 1
    • 2
  • Changyin Sun
    • 1
  • Ding Wang
    • 3
  • Aiguo Song
    • 4
  • Chengshan Qian
    • 5
  1. 1.School of AutomationSoutheast UniversityNanjingChina
  2. 2.Tianjin Key Laboratory of Process Measurement and Control, School of Electrical and Information EngineeringTianjin UniversityTianjinChina
  3. 3.The State Key Laboratory of Management and Control for Complex Systems, Institute of AutomationChinese Academy of SciencesBeijingChina
  4. 4.School of Instrument Science and EngineeringSoutheast UniversityNanjingChina
  5. 5.School of Computer and SoftwareNanjing University of Information Science and TechnologyNanjingChina

Personalised recommendations