Local Tracking Control for Unknown Interconnected Systems via Neuro-Dynamic Programming

  • Bo ZhaoEmail author
  • Derong Liu
  • Mingming Ha
  • Ding Wang
  • Yancai Xu
  • Qinglai Wei
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11307)


This paper develops a neuro-dynamic programming based local tracking control (LTC) scheme for unknown interconnected systems. By using the local input-output data and the desired states of coupling subsystems, a local neural network (NN) identifier is established to obtain the local input gain matrix online. By introducing a modified local cost function, the Hamilton-Jacobi-Bellman equation is solved by a local critic NN with asymptotically convergent weight vector, which is obtained by nested update law, and the LTC can be derived with the desired state aided augmented subsystem. The stability of the closed-loop system is shown by Lyapunov’s direct method. The simulation on the parallel inverted pendulum system illustrates that the developed LTC scheme is effective.


Adaptive dynamic programming Neuro-dynamic programming Local tracking control Optimal control Unknown interconnected systems 



This work was supported in part by the National Natural Science Foundation of China under Grants 61603387, 61773075, 61533017 and 61773373, and in part by the Early Career Development Award of SKLMCCS under Grant 20180201.


  1. 1.
    Wen, G., Chen, C.L.P., Liu, Y.J.: Formation control with obstacle avoidance for a class of stochastic multiagent systems. IEEE Trans. Industr. Electron. 65(7), 5847–5855 (2018)CrossRefGoogle Scholar
  2. 2.
    Zhao, B., Li, Y.: Model-free adaptive dynamic programming based near-optimal decentralized tracking control of reconfigurable manipulators. Int. J. Control Autom. Syst. 16(2), 478–490 (2018)CrossRefGoogle Scholar
  3. 3.
    Werbos, P.J.: Approximate dynamic programming for real time control and neural modeling. In: White, D.A., Sofge, D.A. (eds.) Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches. Van Nostrand Reinhold, New York (1992)Google Scholar
  4. 4.
    Abu-Khalaf, M., Lewis, F.L., Huang, J.: Neurodynamic programming and zero-sum games for constrained control systems. IEEE Trans. Neural Netw. 19(7), 1243–1252 (2008)CrossRefGoogle Scholar
  5. 5.
    Liu, D., Wei, Q., Wang, D., Yang, X., Li, H.: Adaptive Dynamic Programming with Applications in Optimal Control. Springer, Cham (2017). Scholar
  6. 6.
    Wei, Y., Park, J.H., Karimi, H.R., Tian, Y.C.: Improved stability and stabilization results for stochastic synchronization of continuous-time semi-Markovian jump neural networks with time-varying delay. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2488–2501 (2018)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Zhao, B., Wang, D., Shi, G., Liu, D., Li, Y.: Decentralized control for large-scale nonlinear systems with unknown mismatched interconnections via policy iteration. IEEE Trans. Syst. Man. Cyber. Syst. 48(10), 1725–1735 (2018)CrossRefGoogle Scholar
  8. 8.
    Gao, W., Jiang, Y., Jiang, Z.P., Chai, T.: Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming. Automatica 72, 37–45 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Wu, H., Li, M., Guo, L.: Finite-horizon approximate optimal guaranteed cost control of uncertain nonlinear systems with application to mars entry guidance. IEEE Trans. Neural Netw. Learn. Syst. 26, 1456–1467 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Zhao, B., Liu, D., Li, Y.: Online fault compensation control based on policy iteration algorithm for a class of affine non-linear systems with actuator failures. IET Control Theory Appl. 10(15), 1816–1823 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Wen, G., Ge, S.S., Tu, F.: Optimized backstepping for tracking control of strict-feedback systems. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3850–3862 (2018)CrossRefGoogle Scholar
  12. 12.
    Fu, Y., Fu, J., Chai, T.: Robust adaptive dynamic programming of two-player zero-sum games for continuous-time linear systems. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3314–3319 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Mehraeen, S., Jagannathan, S.: Decentralized optimal control of a class of interconnected nonlinear discrete-time systems by using online Hamilton-Jacobi-Bellman formulation. IEEE Trans. Neural Netw. 22(11), 1757–1769 (2011)CrossRefGoogle Scholar
  14. 14.
    Qu, Q., Zhang, H., Feng, T., Jiang, H.: Decentralized adaptive tracking control scheme for nonlinear large-scale interconnected systems via adaptive dynamic programming. Neurocomputing 225, 1–10 (2017)CrossRefGoogle Scholar
  15. 15.
    Zhao, B., Liu, D., Yang, X., Li, Y.: Observer-critic structure-based adaptive dynamic programming for decentralised tracking control of unknown large-scale nonlinear systems. Int. J. Syst. Sci. 48(9), 1978–1989 (2017)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Bo Zhao
    • 1
    Email author
  • Derong Liu
    • 2
  • Mingming Ha
    • 3
  • Ding Wang
    • 1
  • Yancai Xu
    • 1
  • Qinglai Wei
    • 1
  1. 1.The State Key Laboratory of Management and Control for Complex Systems, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.School of AutomationGuangdong University of TechnologyGuangzhouChina
  3. 3.School of Automation and Electrical EngineeringUniversity of Science and Technology BeijingBeijingChina

Personalised recommendations