An Improved Method in Receding Horizon Control with Updating of Terminal Cost Function
In this chapter, we propose a modified receding horizon control (RHC) scheme for discrete linear time-invariant systems, called updated terminal cost RHC (UTC-RHC). The standard receding horizon control, also known as model predictive control (MPC) or moving horizon control (MHC), is a suboptimal control scheme that solves a finite horizon open-loop optimal control problem in an infinite horizon context and yields a state feedback control law. The ability to deal with constraints on controls and states makes RHC popular in industry, especially in the process control industry. A lot of successful applications have been reported in the past three decades. The stability issue has been an important topic in the literature and a lot of approaches have been proposed. Most of these approaches impose constraints on either the terminal state, or the terminal cost, or the horizon size, or their different combinations.
Unlike the standard RHC, UTC-RHC algorithm updates the terminal cost function of the corresponding finite horizon optimal control problem at every stage. It ensures the closed-loop system to be uniformly exponentially stable without imposing any constraint on the terminal state, the horizon size, or the terminal cost, thus makes the design of a stabilizing controller more flexible. Moreover, the control law generated by UTC-RHC algorithm converges to the optimal control law of the infinite horizon optimal control problem.
KeywordsOptimal Control Problem Model Predictive Control Linear Quadratic Regulator Approximate Dynamic Programming Recede Horizon Control
Unable to display preview. Download preview PDF.
- 2.A.G. Barto, R.S. Sutton and C.W. Anderson, Neuronlike elements that can solve difficult learning control problems, IEEE Trans. on Systems Man and Cybernetics SMC-13, 835–846, 1983.Google Scholar
- 3.D.P. Bertsekas, Dynamic programming and suboptimal control: A survey from ADP to MPC, Laboratory for Information and Decision Systems Report 2632, MIT, 2005.Google Scholar
- 6.R.R. Bitmead and M. Gever, Riccati difference and differential equations: convergence, monotonicity and stability, in The Riccati Equation, S. Bittani, A.J. Laub and J.C. Willems (Eds.), Springer-Verlag, New York, pp 263–292, 1991.Google Scholar
- 7.R.R. Bitmead, M. Gever, I.R. Petersen and R.J. Kaye, Monotonicity and stabilizability properties of solutions of the Riccati difference equation: Propositions, lemmas, theorems, fallacious conjectures and counterexamples, Systems and Control Letters 5, 309–315, 1985.zbMATHCrossRefMathSciNetGoogle Scholar
- 8.C.T. Chen, Linear System Theory and Design, Holt, Rinehart and Winston, New York, 1984.Google Scholar
- 17.W.H. Kwon and S. Han, Receding Horizon Control: Model Predictive Control for State Models, Springer-Verlag, London, 2005.Google Scholar
- 20.T. Landelius, Reinforcement learning and distributed local model synthesis, PhD Thesis, Linkoping University, Sweden, 1997.Google Scholar
- 22.F.L. Lewis and V.L. Syrmos, Optimal Control, 2nd edn. John Wiley and Sons, New York, 1995.Google Scholar
- 27.J.A. Primbs, V. Nevistic and J. Doyle, Nonlinear optimal control: A control Lyapunov function and receding horizon perspective, Asian Journal of Control 1, 14–24, 1999.Google Scholar
- 32.J. Si, A.G. Barto, W. Powell and D.C. Wunsch, Handbook of Learning and Approximate Dynamic Programming, Wiley-Interscience, Hoboken, NJ, 2004.Google Scholar
- 33.R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.Google Scholar
- 34.D. Vrabie, M. Abu-Khalaf, F.L. Lewis and Y. Wang, Continuous time ADP for linear systems with partially unknown dynamics, in Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 247–253, 2007.Google Scholar
- 35.C. Watkins, Learning from delayed rewards, PhD Thesis, Cambridge University, Cambridge, England, 1989.Google Scholar
- 36.P.J. Werbos, A menu of designs for reinforcement learning over time, in Neural Networks for Control, W.T. Miller, R.S. Sutton and P.J. Werbos (Eds.), MIT Press, Cambridge, MA, pp. 67–95, 1990.Google Scholar
- 38.H.W. Zhang, J. Huang and F.L. Lewis, Algorithm and stability of ATC receding horizon control, in Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 28–35, 2009.Google Scholar