# An Improved Method in Receding Horizon Control with Updating of Terminal Cost Function

In this chapter, we propose a modified receding horizon control (RHC) scheme for discrete linear time-invariant systems, called updated terminal cost RHC (UTC-RHC). The standard receding horizon control, also known as model predictive control (MPC) or moving horizon control (MHC), is a suboptimal control scheme that solves a finite horizon open-loop optimal control problem in an infinite horizon context and yields a state feedback control law. The ability to deal with constraints on controls and states makes RHC popular in industry, especially in the process control industry. A lot of successful applications have been reported in the past three decades. The stability issue has been an important topic in the literature and a lot of approaches have been proposed. Most of these approaches impose constraints on either the terminal state, or the terminal cost, or the horizon size, or their different combinations.

Unlike the standard RHC, UTC-RHC algorithm updates the terminal cost function of the corresponding finite horizon optimal control problem at every stage. It ensures the closed-loop system to be uniformly exponentially stable without imposing any constraint on the terminal state, the horizon size, or the terminal cost, thus makes the design of a stabilizing controller more flexible. Moreover, the control law generated by UTC-RHC algorithm converges to the optimal control law of the infinite horizon optimal control problem.

## Keywords

Optimal Control Problem Model Predictive Control Linear Quadratic Regulator Approximate Dynamic Programming Recede Horizon Control## Preview

Unable to display preview. Download preview PDF.

## References

- 1.A. Al-Tamimi, F.L. Lewis and M. Abu-Khalaf, Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof,
*IEEE Trans. on Systems Man and Cybernetics, Part B: Cybernetics***38**, 943–949, 2008.CrossRefGoogle Scholar - 2.A.G. Barto, R.S. Sutton and C.W. Anderson, Neuronlike elements that can solve difficult learning control problems,
*IEEE Trans. on Systems Man and Cybernetics***SMC-13**, 835–846, 1983.Google Scholar - 3.D.P. Bertsekas, Dynamic programming and suboptimal control: A survey from ADP to MPC, Laboratory for Information and Decision Systems Report 2632, MIT, 2005.Google Scholar
- 4.D.P. Bertsekas and J.N. Tsitsiklis,
*Neuro-Dynamic Programming*, Athena Scientific, Belmont, MA, 1996.zbMATHGoogle Scholar - 5.R.R. Bitmead, M. Gever and V. Wertz,
*Adaptive Optimal Control: The Thinking Man's GPC*, Prentice Hall, New York, 1990.zbMATHGoogle Scholar - 6.R.R. Bitmead and M. Gever, Riccati difference and differential equations: convergence, monotonicity and stability, in
*The Riccati Equation*, S. Bittani, A.J. Laub and J.C. Willems (Eds.), Springer-Verlag, New York, pp 263–292, 1991.Google Scholar - 7.R.R. Bitmead, M. Gever, I.R. Petersen and R.J. Kaye, Monotonicity and stabilizability properties of solutions of the Riccati difference equation: Propositions, lemmas, theorems, fallacious conjectures and counterexamples,
*Systems and Control Letters***5**, 309–315, 1985.zbMATHCrossRefMathSciNetGoogle Scholar - 8.C.T. Chen,
*Linear System Theory and Design*, Holt, Rinehart and Winston, New York, 1984.Google Scholar - 9.H. Chen and F. Allgower, A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability,
*Automatica***34**, 1205–1217, 1998.zbMATHCrossRefMathSciNetGoogle Scholar - 10.L. Chisci, A. Lombardi and E. Mosca, Dual receding horizon control of constrained discrete time systems,
*European Journal of Control***2**, 278–285, 1996.zbMATHGoogle Scholar - 11.G. De Nicolao, L. Magni and R. Scattolini, Stabilizing receding horizon control of nonlinear time-varying systems,
*IEEE Trans. on Automatic Control***43**, 1030–1036, 1998.zbMATHCrossRefGoogle Scholar - 12.G. Grimm, M.J. Messina, S.E. Tuna and A.R. Teel, Model predictive control: for want of a local control Lyapunov function, all is not lost,
*IEEE Trans. on Automatic Control***50**, 546– 558, 2005.CrossRefMathSciNetGoogle Scholar - 13.R.A. Howard,
*Dynamic Programming and Markov Processes*, MIT Press, Cambridge, MA, 1960.zbMATHGoogle Scholar - 14.A. Jadbabaie and J. Hauser, On the stability of receding horizon control with a general terminal cost,
*IEEE Trans. on Automatic Control***50**, 674–678, 2005.CrossRefMathSciNetGoogle Scholar - 15.A. Jadbabaie, J. Yu and J. Hauser, Unconstrained receding horizon control of nonlinear systems,
*IEEE Trans. on Automatic Control***46**, 776–783, 2001.zbMATHCrossRefMathSciNetGoogle Scholar - 16.S.S. Keerthi and E.G. Gilbert, Optimal infinite-horizon feedback laws for a great class of constrained discrete-time systems: Stability and moving horizon approximations,
*Journal of Optimization Theory and Applications***57**, 65–293, 1988.CrossRefMathSciNetGoogle Scholar - 17.W.H. Kwon and S. Han,
*Receding Horizon Control: Model Predictive Control for State Models*, Springer-Verlag, London, 2005.Google Scholar - 18.W.H. Kwon and A.E. Pearson, A modified quadratic cost problem and feedback stabilization of a linear system,
*IEEE Trans. on Automatic Control***22**, 838–842, 1977.zbMATHCrossRefMathSciNetGoogle Scholar - 19.W.H. Kwon and A.E. Pearson, On feedback stabilization of time-varying discrete linear systems,
*IEEE Trans. on Automatic Control***23**, 479–481, 1978.zbMATHCrossRefMathSciNetGoogle Scholar - 20.T. Landelius, Reinforcement learning and distributed local model synthesis, PhD Thesis, Linkoping University, Sweden, 1997.Google Scholar
- 21.J.W. Lee, W.H. Kwon and J.H. Choi, On stability of constrained receding horizon control with finite terminal weighting matrix,
*Automatica***34**, 1607–1612, 1998.zbMATHCrossRefGoogle Scholar - 22.F.L. Lewis and V.L. Syrmos,
*Optimal Control*, 2nd edn. John Wiley and Sons, New York, 1995.Google Scholar - 23.D.Q. Mayne, J.B. Rawlings, C.V. Rao and P.O.M. Scokaert, Constrained model predictive control: stability and optimality,
*Automatica***36**, 789–814, 2000.zbMATHCrossRefMathSciNetGoogle Scholar - 24.H. Michalska and D.Q. Mayne, Robust receding horizon control of constrained nonlinear systems,
*IEEE Trans. on Automatic Control***38**, 1623–1633, 1993.zbMATHCrossRefMathSciNetGoogle Scholar - 25.M. Morari and J.H. Lee, Model predictive control: Past, present and future,
*Computers and Chemical Engineering***23**, 667–682, 1999.CrossRefGoogle Scholar - 26.J.A. Primbs and V. Nevistic, Feasibility and stability of constrained finite receding horizon control,
*Automatica***36**, 965–971, 2000.zbMATHCrossRefMathSciNetGoogle Scholar - 27.J.A. Primbs, V. Nevistic and J. Doyle, Nonlinear optimal control: A control Lyapunov function and receding horizon perspective,
*Asian Journal of Control***1**, 14–24, 1999.Google Scholar - 28.S.J. Qin and T.A. Badgwell, A survey of industrial model predictive control technology,
*Control Engineering Practice***11**, 733–764, 2003.CrossRefGoogle Scholar - 29.Z. Quan, S. Han and W.H. Kwon, Stability-guaranteed horizon size for receding horizon control,
*IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences***E90-A**, 523–525, 2007.CrossRefGoogle Scholar - 30.W.J. Rugh,
*Linear System Theory*, 2nd edn. Prentice Hall, Upper Saddle River, NJ, 1996.zbMATHGoogle Scholar - 31.P.O.M. Scokaert, D.W. Mayne and J.B. Rawlings, Suboptimal model predictive control (feasibility implies stability),
*IEEE Trans. on Automatic Control***44**, 648–654, 1999.zbMATHCrossRefMathSciNetGoogle Scholar - 32.J. Si, A.G. Barto, W. Powell and D.C. Wunsch,
*Handbook of Learning and Approximate Dynamic Programming*, Wiley-Interscience, Hoboken, NJ, 2004.Google Scholar - 33.R.S. Sutton and A.G. Barto,
*Reinforcement Learning: An Introduction*, MIT Press, Cambridge, MA, 1998.Google Scholar - 34.D. Vrabie, M. Abu-Khalaf, F.L. Lewis and Y. Wang, Continuous time ADP for linear systems with partially unknown dynamics, in
*Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning*, pp. 247–253, 2007.Google Scholar - 35.C. Watkins, Learning from delayed rewards, PhD Thesis, Cambridge University, Cambridge, England, 1989.Google Scholar
- 36.P.J. Werbos, A menu of designs for reinforcement learning over time, in
*Neural Networks for Control*, W.T. Miller, R.S. Sutton and P.J. Werbos (Eds.), MIT Press, Cambridge, MA, pp. 67–95, 1990.Google Scholar - 37.R.L. Williams II and D.A. Lawrence,
*Linear State-Space Control Systems*, John Wiley and Sons, Hoboken, NJ, 2007.CrossRefGoogle Scholar - 38.H.W. Zhang, J. Huang and F.L. Lewis, Algorithm and stability of ATC receding horizon control, in
*Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning*, pp. 28–35, 2009.Google Scholar