Online optimal control for dynamic positioning of vessels via time-based adaptive dynamic programming

  • Xiaoyang Gao
  • Tieshan LiEmail author
  • Qihe Shan
  • Yang Xiao
  • Liang’en Yuan
  • Yifan Liu
Original Research


In this paper, a discrete-time online optimal control via time-based adaptive dynamic programming is developed for dynamic positioning (DP) of vessels in the presence of unknown system dynamic, energy conservation and wastage decrease of equipment. Firstly, a model network is established by a neural network (NN) to identify the DP system. And then, a NN optimal control scheme is developed, in which current and recorded data of DP vessel are utilized to train the critic network and action network. The optimal cost and control law are updated once at the sampling instant. The designed DP optimal control policy can maintain the vessel at desired position and heading angle, and guarantee the uniform ultimate boundedness of all the signals in the closed-loop system simultaneously. Finally, simulation results involving a supply vessel demonstrate the effectiveness of the proposed DP optimal control law.


Dynamic positioning Unknown system dynamic Online optimal control Adaptive dynamic programming 



This work is supported in part by the National Natural Science Foundation of China (under Grant Nos. 51939001, 61976033, 61751202, U1813203, 61803064), the Science and Technology Innovation Funds of Dalian (under Grant no. 2018J11CY022), the LiaoNing Revitalization Talents Program (under Grant no. XLYC1807046), Natural Foundation Guidance Plan Project of Liaoning (2019-ZD-0151) and the Fundamental Research Funds for the Central Universities (under Grant nos. 3132019345).


  1. Bai W, Zhou Q, Li T, Li H (2019) Adaptive reinforcement learning nn control for uncertain nonlinear system with input saturation. IEEE Trans Cybern. CrossRefGoogle Scholar
  2. Balchen JG, Jenssen NA, Saelid S (1980) Dynamic positioning of floating vessels based on Kalman filtering and optimal control. In: Proceedings of the 19th IEEE conference on decision and control, Albuquerque, New Mexico, pp 852–864Google Scholar
  3. Bellman RE (1957) Dynamic programming. Princeton University Press, PrincetonzbMATHGoogle Scholar
  4. Benetazzo F, Ippoliti G, Longhi S, Raspa P (2015) Advanced control for fault-tolerant dynamic positioning of an offshore supply vessel. Ocean Eng 106:472–484CrossRefGoogle Scholar
  5. Chen H (2008) Research on design of DP thruster system of deepwater semi-submersible platform. Shanghai Jiao Tong University, ShanghaiGoogle Scholar
  6. Chen T (2018) An innovative fuzzy and artificial neural network approach for forecasting yield under an uncertain learning environment. J Ambient Intell Humaniz Comput 9(4):1013–1025CrossRefGoogle Scholar
  7. Cheng L, Liu W, Yang C, Huang T, Hou ZG, Tan M (2017) A neural-network-based controller for piezoelectric-actuated stick–slip devices. IEEE Trans Ind Electron 65(3):2598–2607CrossRefGoogle Scholar
  8. Cheng L, Liu Y, Hou ZG, Tan M, Du D, Fei M (2019) A rapid spiking neural network approach with an application on hand gesture recognition. IEEE Trans Cogn Dev Syst. CrossRefGoogle Scholar
  9. Dierks T, Jagannathan S, Jagannathan (2012) Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans Neural Netw Learn Syst 23(7):1118–1129CrossRefGoogle Scholar
  10. Dreyfus SE, Law AM (1977) The art and theory of dynamic programming. Academic Press, New YorkzbMATHGoogle Scholar
  11. Du J, Hu X, Krstic M, Sun Y (2016) Robust dynamic positioning of ships with disturbances under input saturation. Automatica 73:207–214MathSciNetCrossRefGoogle Scholar
  12. Du J, Hu X, Krstic M, Sun Y (2018) Dynamic positioning of ships with unknown parameters and disturbances. Control Eng Pract 76:22–30CrossRefGoogle Scholar
  13. Fossen TI, Berge SP (1997) Nonlinear vectorial backstepping design for global exponential tracking of marine vessels in the presence of actuator dynamics. In: Proceedings of the 36th IEEE conference on decision and control, 1997, vol 5. IEEE, pp 4237–4242Google Scholar
  14. Fossen TI (2002) Marine control systems: guidance, navigation and control of ships, rigs and underwater vehicles. Marine Cybernetics AS, TrondheimGoogle Scholar
  15. Fossen TI (2002) Marine control systems: guidance, navigation and control of ships, rigs and underwater vehicles. Marine Cybernetics AS, TrondlheimGoogle Scholar
  16. Fossen TI (2011) Handbook of marine craft hydrodynamics and motion control. Wiley, ChichesterCrossRefGoogle Scholar
  17. Fossen TI, Grovlen A (1998) Nonlinear output feedback control of dynamically positioned ships using vectorial observer backstepping. IEEE Trans Control Syst Technol 6(1):121–128CrossRefGoogle Scholar
  18. Fossen TI, Strand JP (2001) Nonlinear passive weather optimal positioning control (WOPC) system for ships and rigs: experimental results. Automatica 37(5):701–715MathSciNetCrossRefGoogle Scholar
  19. Fu M, Liu T, Xu Y, et al (2016) A neurodynamics-based dynamic surface control algorithm for tracking control of dynamic positioning vehicles. In: Proceedings of the 35th Chinese control conference, Chengdu, China, pp 9167–9171Google Scholar
  20. Grimble MJ, Patton RJ, Wise DA (1980) The design of dynamic ship positioning control systems using stochastic optimal control theory. Optim Control Appl Methods 1(2):167–202CrossRefGoogle Scholar
  21. Kim H (2019) Multiple vehicle tracking and classification system with a convolutional neural network. J Ambient Intell Human Comput. CrossRefGoogle Scholar
  22. Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628CrossRefGoogle Scholar
  23. Shao X (2015) Research on control methods for dynamic positioning ships based on discrete time system theory. Harbin Engineering University, HarbinGoogle Scholar
  24. Shi Q, Li T, Li J, Chen CP, Xiao Y, Shan Q (2019) Adaptive leader-following formation control with collision avoidance for a class of second-order nonlinear multi-agent systems. Neurocomputing 350:282–290CrossRefGoogle Scholar
  25. Shun J (2010) Research on dynamic positioning of submarine cable workboat based on fuzzy control. Shanghai Jiao Tong University, ShanghaiGoogle Scholar
  26. Sun X (2014) Research on model predictive control for the ship dynamic positioning system. Harbin Engineering University, HarbinGoogle Scholar
  27. Sun T, Peng L, Cheng L, Hou ZG, Pan Y (2019) Composite learning enhanced robot impedance control. IEEE Trans Neural Netw Learn Syst. CrossRefGoogle Scholar
  28. Tannuri EA, Agostinho AC, Morishita HM, Moratelli L Jr (2010) Dynamic positioning systems: an experimental analysis of sliding mode control. Control Eng Pract 18(10):1121–1132CrossRefGoogle Scholar
  29. Veksler A, Johansen TA, Borrelli F, Realfsen B (2016) Dynamic positioning with model predictive control. IEEE Trans Control Syst Technol 24(4):1340–1353CrossRefGoogle Scholar
  30. Wang D, Liu D (2018) Learning and guaranteed cost control with event-based adaptive critic implementation. IEEE Trans Neural Netw Learn Syst 99:1–11Google Scholar
  31. Wang D, He H, Liu D (2017a) Adaptive critic nonlinear robust control: a survey. IEEE Trans Cybern 47(10):3429–3451CrossRefGoogle Scholar
  32. Wang D, He H, Zhong X, Liu D (2017b) Event-driven nonlinear discounted optimal regulation involving a power system application. IEEE Trans Ind Electron 64(10):8177–8186CrossRefGoogle Scholar
  33. Wen G, Ge SS, Chen CP, Tu F, Wang S (2018) Adaptive tracking control of surface vessel using optimized backstepping technique. IEEE Trans Cybern 99:1–12CrossRefGoogle Scholar
  34. White DA, Sofge DA (1992) Handbook of intelligent control: neural, fuzzy, and adaptive approaches. Van Nostrand Reinhold, New YorkGoogle Scholar
  35. Xiao G, Zhang H, Luo Y (2015) Online optimal control of unknown discrete-time nonlinear systems by using time-based adaptive dynamic programming. Neurocomputing 165:163–170CrossRefGoogle Scholar
  36. Xu H, Feng H (2016) The principle of ship dynamic positioning system. National Defense Industry Press, PekingGoogle Scholar
  37. Zhang HG, Zhang X, Luo YH, Yang J (2013) An overview of research on adaptive dynamic programming. Acta Autom Sin 39(4):303–311CrossRefGoogle Scholar
  38. Zhao D, Bian X, Ding F (2011) Design of a nonlinear controller for dynamic ship positioning. J Harbin Eng Univ 32(1):57–61Google Scholar
  39. Zhou L, Wang W, Chen H (2008) Review on the study of dynamic positioning control systems for vessels. Ship Ocean Eng 37(2):86–91Google Scholar
  40. Zhu Y, Zhao D, Yang X, Zhang Q (2017) Policy iteration for $ H_\infty $ optimal control of polynomial nonlinear systems via sum of squares programming. IEEE Trans Cybern 48(2):500–509CrossRefGoogle Scholar
  41. Zhu Y, Zhao D, Li X, Wang D (2018a) Control-limited adaptive dynamic programming for multi-battery energy storage systems. IEEE Trans Smart Grid 10(4):4235–4244CrossRefGoogle Scholar
  42. Zhu Y, Zhao D, Zhong Z (2018b) Adaptive optimal control of heterogeneous cacc system with uncertain dynamics. IEEE Trans Control Syst Technol 99:1–8Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Xiaoyang Gao
    • 1
  • Tieshan Li
    • 1
    Email author
  • Qihe Shan
    • 1
  • Yang Xiao
    • 2
  • Liang’en Yuan
    • 1
  • Yifan Liu
    • 1
  1. 1.Navigation CollegeDalian Maritime UniversityDalianChina
  2. 2.Department of Computer ScienceThe University of AlabamaTuscaloosaUSA

Personalised recommendations