Skip to main content
SpringerLink
Log in

ADP-based nonlinear optimal output regulation with nonlinear exosystem

  • S.I.: Reinforcement Learning and Adaptive Dynamic Programming for Control Systems
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper discusses the optimal output regulation problem for nonlinear systems with a nonlinear exosystem. Adaptive dynamic programming and internal model principle are integrated to deal with this problem. The control scheme process is designed in the following two steps. In the first step, a nonlinear internal model is constructed to obtain the feedforward controller, which allows for the conversion of the output regulation problem into a stabilization problem. Compared with feedforward design approach, the common assumption that the exosystem is linear and neutral stable has been eliminated successfully. In the second step, the control cost is taken into account and the optimal feedback controller is learned through adaptive dynamic programming. After that, we demonstrate that the closed-loop system is uniformly ultimately bounded by the Lyapunov stability theory. As a distinctive feature, the proposed control framework can tolerate the nonlinear exosystem. Finally, the effectiveness of the control scheme is shown by a simulation example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availibility

The data has been shown in the paper, and there is no additional data to be supplemented.

References

  1. Wang X, Quan Z, Zhang J (2023) Optimal 3-dimension trajectory-tracking guidance for reusable launch vehicle based on back-stepping adaptive dynamic programming. Neural Comput Appl 35(7):5319–5334

    Article  Google Scholar 

  2. Powell WB (2007) Approximate dynamic programming: solving the curse of dimensionality. Wiley, New York

    Book  MATH  Google Scholar 

  3. Zhu J, Zhang P, Hou Y (2021) Adaptive dynamic programming-based optimal regulation on input-constrained nonlinear time-delay systems. Neural Comput Appl 33:13039–13047

    Article  Google Scholar 

  4. Gao W, Jiang Z-P (2016) Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Trans Autom Contr 61(12):4164–4169

    Article  MathSciNet  MATH  Google Scholar 

  5. Jiang Y, Kiumarsi B, Fan J, Chai T, Li J, Lewis FL (2019) Optimal output regulation of linear discrete-time systems with unknown dynamics using reinforcement learning. IEEE Trans Cybernet 50(7):3147–3156

    Article  Google Scholar 

  6. Gao W, Jiang Z-P (2019) Adaptive optimal output regulation of time-delay systems via measurement feedback. IEEE Trans Neural Netw Learn Syst 30(3):938–945

    Article  MathSciNet  Google Scholar 

  7. Fan J, Wu Q, Jiang Y, Chai T, Lewis FL (2020) Model-free optimal output regulation for linear discrete-time lossy networked control systems. IEEE Trans Syst Man Cybern Syst 50(11):4033–4042

    Article  Google Scholar 

  8. Gao W, Jiang Y, Davari M (2019) Data-driven cooperative output regulation of multi-agent systems via robust adaptive dynamic programming. IEEE Trans Circ Syst II Exp Briefs 66(3):447–451

    Google Scholar 

  9. Deng C, Gao W, Wen C, Chen Z, Wang W (2023) Data-driven practical cooperative output regulation under actuator faults and dos attacks. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2023.3263480

    Article  Google Scholar 

  10. Zhou L, Huang M, Tan F, Zhang Y (2023) Mean-square bounded synchronization of complex networks under deception attacks via pinning impulsive control. Nonlinear Dyn 111:11243–11259

    Article  Google Scholar 

  11. Gao W, Jiang Z-P (2018) Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems. IEEE Trans Neural Netw Learn Syst 29(6):2614–2624

    Article  MathSciNet  Google Scholar 

  12. Jiang Y, Fan J, Gao W, Chai T, Lewis FL (2020) Cooperative adaptive optimal output regulation of nonlinear discrete-time multi-agent systems. Automatica 121:109149

    Article  MathSciNet  MATH  Google Scholar 

  13. Huang J (2004) Nonlinear output regulation: theory and applications. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  14. Huang J, Chen Z (2004) A general framework for tackling the output regulation problem. IEEE Trans Autom Control 49(12):2203–2218

    Article  MathSciNet  MATH  Google Scholar 

  15. Khalil HK (2002) Nonlinear Syst, 3rd ed. Pearson New York, Upper Saddle River

    Google Scholar 

  16. Byrnes CI, Priscoli FD, Isidori A, Kang W (1997) Structurally stable output regulation of nonlinear systems. Automatica 33(3):369–385

    Article  MathSciNet  MATH  Google Scholar 

  17. Xi Z, Ding Z (2007) Global adaptive output regulation of a class of nonlinear systems with nonlinear exosystems. Automatica 43(1):143–149

    Article  MathSciNet  MATH  Google Scholar 

  18. Wu Y, Sun X-M, Zhao X, Shen T (2019) Optimal control of boolean control networks with average cost: a policy iteration approach. Automatica 100:378–387

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhao B, Liu D, Luo C (2019) Reinforcement learning-based optimal stabilization for unknown nonlinear systems subject to inputs with uncertain constraints. IEEE Trans Neural Netw Learn Syst 31(10):4330–4340

    Article  MathSciNet  Google Scholar 

  20. Zhang H, Cui L, Zhang X, Luo Y (2018) Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans Cybern 48(12):3337–3348

    Google Scholar 

  21. Lewis FL, Vrabie D, Syrmos VL (2012) Optimal Control. Wiley, New York

    Book  MATH  Google Scholar 

  22. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circ Syst Magaz 9(3):32–50

    Article  Google Scholar 

  23. Haykin S (1998) Neural networks: a comprehensive foundation. Prentice Hall PTR, Englewood Cliffs

    MATH  Google Scholar 

  24. Tan F, Zhou L, Xia J (2022) Adaptive quantitative exponential synchronization in multiplex cohen-grossberg neural networks under deception attacks. J Frank Inst 359(18):10558–10577

    Article  MathSciNet  MATH  Google Scholar 

  25. Song X, Sun P, Song S, Stojanovic V (2023) Quantized neural adaptive finite-time preassigned performance control for interconnected nonlinear systems. Neural Comput Appl. https://doi.org/10.1007/s00521-023-08361-y

    Article  Google Scholar 

  26. Jiang Y, Jiang Z-P (2017) Robust adaptive dynamic programming. Wiley, Hoboken

    Book  MATH  Google Scholar 

  27. Guckenheimer J (1980) Dynamics of the van der pol equation. IEEE Trans Circ Syst 27(11):983–989

    Article  MathSciNet  MATH  Google Scholar 

  28. Meng Q, Ma Q, Shi Y (2023) Adaptive fixed-time stabilization for a class of uncertain nonlinear systems. IEEE Trans Autom Control. https://doi.org/10.1109/TAC.2023.3244151

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62173183 and Grant 62073169.

Author information

Author notes

  1. Haoan Jiang, Peng Jin, Qian Ma, Guopeng Zhou and Guoying Miao have contributed equally to this work.

Authors and Affiliations

  1. School of Automation, Nanjing University of Science and Technology, Xiaolingwei Street, Nanjing, 210094, Jiangsu, China

    Haoan Jiang, Peng Jin & Qian Ma

  2. Department of Research and Development, Hubei Xiangcheng Intelligent Electromechanical Research Institute, Hesheng Street, Xianning, 437100, Hubei, China

    Guopeng Zhou

  3. School of Automation, Nanjing University of Information Science and Technology, Ningliu Street, Nanjing, 210044, Jiangsu, China

    Guoying Miao

Authors
  1. Haoan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guopeng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guoying Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Jin.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, H., Jin, P., Ma, Q. et al. ADP-based nonlinear optimal output regulation with nonlinear exosystem. Neural Comput & Applic (2023). https://doi.org/10.1007/s00521-023-09253-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-023-09253-x

Keywords

Search

Navigation