Advertisement

An Improved Method in Receding Horizon Control with Updating of Terminal Cost Function

  • Hongwei Zhang
  • Jie Huang
  • Frank L. Lewis
Chapter
Part of the Intelligent Systems, Control, and Automation: Science and Engineering book series (ISCA, volume 39)

In this chapter, we propose a modified receding horizon control (RHC) scheme for discrete linear time-invariant systems, called updated terminal cost RHC (UTC-RHC). The standard receding horizon control, also known as model predictive control (MPC) or moving horizon control (MHC), is a suboptimal control scheme that solves a finite horizon open-loop optimal control problem in an infinite horizon context and yields a state feedback control law. The ability to deal with constraints on controls and states makes RHC popular in industry, especially in the process control industry. A lot of successful applications have been reported in the past three decades. The stability issue has been an important topic in the literature and a lot of approaches have been proposed. Most of these approaches impose constraints on either the terminal state, or the terminal cost, or the horizon size, or their different combinations.

Unlike the standard RHC, UTC-RHC algorithm updates the terminal cost function of the corresponding finite horizon optimal control problem at every stage. It ensures the closed-loop system to be uniformly exponentially stable without imposing any constraint on the terminal state, the horizon size, or the terminal cost, thus makes the design of a stabilizing controller more flexible. Moreover, the control law generated by UTC-RHC algorithm converges to the optimal control law of the infinite horizon optimal control problem.

Keywords

Optimal Control Problem Model Predictive Control Linear Quadratic Regulator Approximate Dynamic Programming Recede Horizon Control 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A. Al-Tamimi, F.L. Lewis and M. Abu-Khalaf, Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof, IEEE Trans. on Systems Man and Cybernetics, Part B: Cybernetics 38, 943–949, 2008.CrossRefGoogle Scholar
  2. 2.
    A.G. Barto, R.S. Sutton and C.W. Anderson, Neuronlike elements that can solve difficult learning control problems, IEEE Trans. on Systems Man and Cybernetics SMC-13, 835–846, 1983.Google Scholar
  3. 3.
    D.P. Bertsekas, Dynamic programming and suboptimal control: A survey from ADP to MPC, Laboratory for Information and Decision Systems Report 2632, MIT, 2005.Google Scholar
  4. 4.
    D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.zbMATHGoogle Scholar
  5. 5.
    R.R. Bitmead, M. Gever and V. Wertz, Adaptive Optimal Control: The Thinking Man's GPC, Prentice Hall, New York, 1990.zbMATHGoogle Scholar
  6. 6.
    R.R. Bitmead and M. Gever, Riccati difference and differential equations: convergence, monotonicity and stability, in The Riccati Equation, S. Bittani, A.J. Laub and J.C. Willems (Eds.), Springer-Verlag, New York, pp 263–292, 1991.Google Scholar
  7. 7.
    R.R. Bitmead, M. Gever, I.R. Petersen and R.J. Kaye, Monotonicity and stabilizability properties of solutions of the Riccati difference equation: Propositions, lemmas, theorems, fallacious conjectures and counterexamples, Systems and Control Letters 5, 309–315, 1985.zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    C.T. Chen, Linear System Theory and Design, Holt, Rinehart and Winston, New York, 1984.Google Scholar
  9. 9.
    H. Chen and F. Allgower, A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability, Automatica 34, 1205–1217, 1998.zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    L. Chisci, A. Lombardi and E. Mosca, Dual receding horizon control of constrained discrete time systems, European Journal of Control 2, 278–285, 1996.zbMATHGoogle Scholar
  11. 11.
    G. De Nicolao, L. Magni and R. Scattolini, Stabilizing receding horizon control of nonlinear time-varying systems, IEEE Trans. on Automatic Control 43, 1030–1036, 1998.zbMATHCrossRefGoogle Scholar
  12. 12.
    G. Grimm, M.J. Messina, S.E. Tuna and A.R. Teel, Model predictive control: for want of a local control Lyapunov function, all is not lost, IEEE Trans. on Automatic Control 50, 546– 558, 2005.CrossRefMathSciNetGoogle Scholar
  13. 13.
    R.A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, MA, 1960.zbMATHGoogle Scholar
  14. 14.
    A. Jadbabaie and J. Hauser, On the stability of receding horizon control with a general terminal cost, IEEE Trans. on Automatic Control 50, 674–678, 2005.CrossRefMathSciNetGoogle Scholar
  15. 15.
    A. Jadbabaie, J. Yu and J. Hauser, Unconstrained receding horizon control of nonlinear systems, IEEE Trans. on Automatic Control 46, 776–783, 2001.zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    S.S. Keerthi and E.G. Gilbert, Optimal infinite-horizon feedback laws for a great class of constrained discrete-time systems: Stability and moving horizon approximations, Journal of Optimization Theory and Applications 57, 65–293, 1988.CrossRefMathSciNetGoogle Scholar
  17. 17.
    W.H. Kwon and S. Han, Receding Horizon Control: Model Predictive Control for State Models, Springer-Verlag, London, 2005.Google Scholar
  18. 18.
    W.H. Kwon and A.E. Pearson, A modified quadratic cost problem and feedback stabilization of a linear system, IEEE Trans. on Automatic Control 22, 838–842, 1977.zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    W.H. Kwon and A.E. Pearson, On feedback stabilization of time-varying discrete linear systems, IEEE Trans. on Automatic Control 23, 479–481, 1978.zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    T. Landelius, Reinforcement learning and distributed local model synthesis, PhD Thesis, Linkoping University, Sweden, 1997.Google Scholar
  21. 21.
    J.W. Lee, W.H. Kwon and J.H. Choi, On stability of constrained receding horizon control with finite terminal weighting matrix, Automatica 34, 1607–1612, 1998.zbMATHCrossRefGoogle Scholar
  22. 22.
    F.L. Lewis and V.L. Syrmos, Optimal Control, 2nd edn. John Wiley and Sons, New York, 1995.Google Scholar
  23. 23.
    D.Q. Mayne, J.B. Rawlings, C.V. Rao and P.O.M. Scokaert, Constrained model predictive control: stability and optimality, Automatica 36, 789–814, 2000.zbMATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    H. Michalska and D.Q. Mayne, Robust receding horizon control of constrained nonlinear systems, IEEE Trans. on Automatic Control 38, 1623–1633, 1993.zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    M. Morari and J.H. Lee, Model predictive control: Past, present and future, Computers and Chemical Engineering 23, 667–682, 1999.CrossRefGoogle Scholar
  26. 26.
    J.A. Primbs and V. Nevistic, Feasibility and stability of constrained finite receding horizon control, Automatica 36, 965–971, 2000.zbMATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    J.A. Primbs, V. Nevistic and J. Doyle, Nonlinear optimal control: A control Lyapunov function and receding horizon perspective, Asian Journal of Control 1, 14–24, 1999.Google Scholar
  28. 28.
    S.J. Qin and T.A. Badgwell, A survey of industrial model predictive control technology, Control Engineering Practice 11, 733–764, 2003.CrossRefGoogle Scholar
  29. 29.
    Z. Quan, S. Han and W.H. Kwon, Stability-guaranteed horizon size for receding horizon control, IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences E90-A, 523–525, 2007.CrossRefGoogle Scholar
  30. 30.
    W.J. Rugh, Linear System Theory, 2nd edn. Prentice Hall, Upper Saddle River, NJ, 1996.zbMATHGoogle Scholar
  31. 31.
    P.O.M. Scokaert, D.W. Mayne and J.B. Rawlings, Suboptimal model predictive control (feasibility implies stability), IEEE Trans. on Automatic Control 44, 648–654, 1999.zbMATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    J. Si, A.G. Barto, W. Powell and D.C. Wunsch, Handbook of Learning and Approximate Dynamic Programming, Wiley-Interscience, Hoboken, NJ, 2004.Google Scholar
  33. 33.
    R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.Google Scholar
  34. 34.
    D. Vrabie, M. Abu-Khalaf, F.L. Lewis and Y. Wang, Continuous time ADP for linear systems with partially unknown dynamics, in Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 247–253, 2007.Google Scholar
  35. 35.
    C. Watkins, Learning from delayed rewards, PhD Thesis, Cambridge University, Cambridge, England, 1989.Google Scholar
  36. 36.
    P.J. Werbos, A menu of designs for reinforcement learning over time, in Neural Networks for Control, W.T. Miller, R.S. Sutton and P.J. Werbos (Eds.), MIT Press, Cambridge, MA, pp. 67–95, 1990.Google Scholar
  37. 37.
    R.L. Williams II and D.A. Lawrence, Linear State-Space Control Systems, John Wiley and Sons, Hoboken, NJ, 2007.CrossRefGoogle Scholar
  38. 38.
    H.W. Zhang, J. Huang and F.L. Lewis, Algorithm and stability of ATC receding horizon control, in Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 28–35, 2009.Google Scholar

Copyright information

© Springer Science + Business Media B.V. 2009

Authors and Affiliations

  • Hongwei Zhang
    • 1
  • Jie Huang
    • 1
  • Frank L. Lewis
    • 2
  1. 1.Department of Mechanical and Automation EngineeringThe Chinese University of Hong KongHong Kong
  2. 2.Automation and Robotics Research Institute, Department of Electrical EngineeringThe University of Texas at ArlingtonFort WorthUSA

Personalised recommendations

Cite chapter

Buy options