Skip to main content
SpringerLink
Log in

Neural network optimal control for discrete-time nonlinear systems with known internal dynamics

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

A neural network (NN) optimal control for discrete-time nonlinear dynamic systems with known internal dynamics is designed. The control is described by an algebraic equation with a variable structure. This algebraic equation is derived analytically. A functional block diagram of the controlled system is given and analyzed. Software and hardware implementation aspects of the controller are discussed. The controller does not need any training and has moderate complexity. The discrete-time state variable trajectories of the controlled system are shown to be globally asymptotically stable and convergent to unique steady states. It is proved that these trajectories converge to steady-state neighborhood in a finite number of steps. Sliding mode analysis of controller operation is fulfilled. A correctness of controller operation in the case of disturbances of its nonlinearities is analyzed. Using the controller for a special case of optimal tracking control is discussed. Results of presented computer simulations of optimal control of discrete-time two-dimensional and three-dimensional affine nonlinear systems and optimal tracking control of permanent-magnet motor of linear type applied for accurate positioning and nonlinear cooling continuous stirred tank reactor confirm theoretical statements of the paper and illustrate a performance of the controller.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

All data generated and analyzed during this study are included in this published article.

References

  1. Ross IM (2015) A primer on Pontryagin’s principle in optimal control: second edition. Collegiate Publishers.

  2. Lewis FL, Vrabie DL, Syrmos VL (2012) Optimal control. John Wiley & Sons, Hoboken, NJ, USA. https://doi.org/10.1002/9781118122631

  3. Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2018) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062. https://doi.org/10.1109/TNNLS.2017.2773458

    Article  MathSciNet  PubMed  Google Scholar 

  4. Schiassi E, D’Ambrosio A, Furfaro R (2022) Bellman neural networks for the class of optimal control problems with integral quadratic cost. IEEE Trans Artif Intell. https://doi.org/10.1109/TAI.2022.3206735

    Article  Google Scholar 

  5. Fairbank M, Li S, Fu X, Alonso E, Wunsch D (2014) An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances. Neural Netw 49:74–86. https://doi.org/10.1016/j.neunet.2013.09.010

    Article  PubMed  Google Scholar 

  6. Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst 32(6):76–105. https://doi.org/10.1109/MCS.2012.2214134

    Article  MathSciNet  Google Scholar 

  7. Luo B, Wu HN, Huang T, Liu D (2014) Date-based approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica 50(12):3281–3290. https://doi.org/10.48550/arXiv.1311.0396

    Article  MathSciNet  Google Scholar 

  8. Aguilar CO, Krener AJ (2014) Numerical solutions to the Bellman equation of optimal control. J Optim Theory Appl 160:527–552. https://doi.org/10.1007/s10957-013-0403-8

    Article  MathSciNet  Google Scholar 

  9. Bauer F, Grune L, Semmler W (2006) Adaptive spline interpolation for Hamilton–Jacobi–Bellman equations. Appl Numer Math 56(9):1196–1210. https://doi.org/10.1016/j.apnum.2006.03.011

    Article  MathSciNet  Google Scholar 

  10. Wang Z, Li Y (2021) Nested sparse successive Galerkin approximation for nonlinear optimal control problems. IEEE Control Syst Lett 5(2):511–516. https://doi.org/10.1109/LCSYS.2020.3003732

    Article  MathSciNet  Google Scholar 

  11. Tan KK, Zhao S, Hu JX (2007) Online automatic tuning of a proportional integral derivative controller based on an iterative learning control approach. IET Control Theory Appl 1(1):90–96. https://doi.org/10.1049/iet-cta:20050004

    Article  Google Scholar 

  12. Luo Q, Xue S, Liu D (2021) Adaptive critic designs for decentralized robust control of nonlinear interconnected systems via event triggering mechanism. Int J Syst Sci 53(5):1031–1047. https://doi.org/10.1080/00207721.2021.1987578

    Article  ADS  Google Scholar 

  13. Lin Z, Duan J, Li SE, Ma H, Li J, Chen J, Cheng B, Ma J (2023) Policy-iteration-based finite-horizon approximate dynamic programming for continuous-time nonlinear optimal control. IEEE Trans Neural Netw Learn Syst 34(9):5255–5267. https://doi.org/10.1109/TNNLS.2022.3225090

    Article  MathSciNet  PubMed  Google Scholar 

  14. Wei Q, Liu D, Song R, Yan P (2016) Data-based self-learning optimal control for continuous-time unknown nonlinear systems with disturbances. In: Proceedings of 2016 Chinese Control Decision Conference, pp 6633–6638. https://doi.org/10.1109/CCDC.2016.7532191

  15. Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888. https://doi.org/10.1016/j.automatica.2010.02.018

    Article  MathSciNet  Google Scholar 

  16. Tymoshchuk P (2019) A neural circuit model of adaptive robust tracking control for continuous-time nonlinear systems. In: Proc 28th Intern Conf ICANN, LNCS (vol 11727, pp 819–835). https://doi.org/10.1007/978-3-030-30487-4

  17. Sun C, Saxena A, Sinitsyn NA (2017) Nearly optimal quantum control: an analytical approach. J Phys B At Mol Opt Phys 50:1–8. https://doi.org/10.1088/1361-6455/aa807d

    Article  CAS  Google Scholar 

  18. Wu D (2015) Analytic solutions to optimal control problems with constraints. Appl Math 6:2326–2339. https://doi.org/10.4236/am.2015.614205

    Article  Google Scholar 

  19. Liu Y, Zhou Y, Su S, Xun J, Tang T (2021) An analytical optimal control approach for virtually coupled high-speed trains with local and string stability. Transp Res Part C 125:1–19. https://doi.org/10.1016/j.trc.2020.102886

    Article  Google Scholar 

  20. Tymoshchuk P, Lobur M (2020) Principles of artificial neural networks and their applications: manual. Lviv Polytechnic Publishing House, Lviv, Ukraine

    Google Scholar 

  21. Luo B, Liu D, Huang T, Yang X, Ma H (2017) Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems. Inf Sci 411:66–83. https://doi.org/10.1016/j.ins.2017.05.005

    Article  Google Scholar 

  22. Alanis AY, Ornelas-Tellez F, Sanchez EN (2012) Discrete-time inverse optimal neural control for synchronous generators. Eng Appl Artif Intell 26:697–705. https://doi.org/10.1016/j.engappai.2012.07.008

    Article  Google Scholar 

  23. Wei Q, Liu D (2013) A new self-learning optimal control scheme for discrete-time nonlinear systems using police iterative adaptive dynamic programming. IFAC Proc 46:580–585

    Google Scholar 

  24. Kiumarsi B, Lewis FL, Levine DS (2015) Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure. Neurocomputing 156:157–165. https://doi.org/10.1016/j.neucom.2014.12.067

    Article  Google Scholar 

  25. Postyyan R, Busoniu L, Nesic D, Daafouz J (2017) Stability analysis of discrete-time infinite-horizon optimal control with discounted cost. IEEE Trans Autom Control 62(6):2736–2749. https://doi.org/10.1088/1361-6455/aa807d

    Article  MathSciNet  CAS  Google Scholar 

  26. Luo H, Yu L (2021) Stability analysis for the discrete-time T-S fuzzy system with stochastic disturbance and state delay. Fuzzy Inf Eng 13(1):17–39. https://doi.org/10.1080/16168658.2021.1932151

    Article  MathSciNet  Google Scholar 

  27. Hu NT (2023) Stability analysis for digital redesign of discrete-time switched systems using linear matrix inequality. Mathematics 11:1–22. https://doi.org/10.3390/math11112468

    Article  Google Scholar 

  28. Siegel JW, Xu J (2020) Approximation rates for neural networks with general activation functions. Neural Netw 128:313–321. https://doi.org/10.1016/j.neunet.2020.05.019

    Article  PubMed  Google Scholar 

  29. Tymoshchuk P (2020) Optimal control for continuous-time scalar nonlinear systems with known dynamics. In: 2020 16th international conference on control, automation, robotics and vision (ICARCV) pp 695–700. https://doi.org/10.1109/ICARCV50220.2020.9305440

  30. Haykin S (2011) Neural networks and learning machines. Pearson, Ontario, Canada.

  31. Singh N, Panda SP (2022) Artificial neural network on graphical processing unit and its emphasis on ground water level prediction. Int J Inf Technol 14:3659–3666. https://doi.org/10.1007/s41870-022-00988-9

    Article  Google Scholar 

  32. Szczeœniak A, Myczuda Z (2010) A method of charge accumulation in the logarithmic analog-to-digital converter with a successive approximation. Electrotech Rev 86(10):336–340

    Google Scholar 

  33. Tymoshchuk P, Shatnyi S (2015) Hardware implementation of discrete-time neural circuit of largest/smallest signal identification. In: The experience of designing and application of CAD systems in microelectronics, pp 226–230. https://doi.org/10.1109/CADSM.2015.7230842.

  34. Tymoshchuk P, Shatnyi S (2019) Hardware implementation design of parallelized fuzzy adaptive resonance theory neural network. In: Perspective technologies and methods in MEMS design, pp 61–66. https://doi.org/10.1109/MEMSTECH.2019.8817406

  35. Kaszkurewicz E, Bhaya A (2005) A generalized Persidskii theorem and its applications to non-smooth gradient dynamical systems. IFAC Proc 38(1):604–609. https://doi.org/10.3182/20050703-6-CZ-1902.00756

    Article  Google Scholar 

  36. Haninger K, Hedrick K (2016) Discrete-time implementations of sliding mode control. In: American Control Conference, pp 6519–6524. https://doi.org/10.1109/ACC.2016.7526696

  37. Dierks T, Jagannthan S (2009) Optimal control of affine nonlinear discrete-time systems. In: 17th Mediterranean conf control automation, pp 1390–1395. https://doi.org/10.1109/MED.2009.5164741

  38. Yan Z, Wang J (2011) Model predictive control of nonlinear affine systems based on the General Projection neural network and its application to a continuous stirred tank reactor. In: International conference on information science and technology, pp 1011–1015. https://doi.org/10.1109/ICARCV50220.2020.9305440.

Download references

Author information

Authors and Affiliations

  1. University of North Texas, Denton, TX, 76207, USA

    Pavlo Tymoshchuk

Authors
  1. Pavlo Tymoshchuk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pavlo Tymoshchuk.

Ethics declarations

Conflict of interest

The author has no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tymoshchuk, P. Neural network optimal control for discrete-time nonlinear systems with known internal dynamics. Neural Comput & Applic 36, 4421–4434 (2024). https://doi.org/10.1007/s00521-023-09244-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09244-y

Keywords

Search

Navigation