Computational Optimization and Applications

, Volume 4, Issue 1, pp 47–66 | Cite as

An efficient trust region method for unconstrained discrete-time optimal control problems

  • Thomas F. Coleman
  • Aiping Liao


Discrete-time optimal control (DTOC) problems are large-scale optimization problems with a dynamic structure. In previous work this structure has been exploited to provide very fast and efficient local procedures. Two examples are the differential dynamic programming algorithm (DDP) and the stagewise Newton procedure—both require onlyO(N) operations per iteration, whereN is the number of timesteps. Both exhibit a quadratic convergence rate. However, most algorithms in this category do not have a satisfactory global convergence strategy. The most popular global strategy is shifting: this sometimes works poorly due to the lack of automatic adjustment to the shifting element.

In this paper we propose a method that incorporates the trust region idea with the local stagewise Newton's method. This method possesses advantages of both the trust region idea and the stagewise Newton's method, i.e., our proposed method has strong global and local convergence properties yet remains economical. Preliminary numerical results are presented to illustrate the behavior of the proposed algorithm. We also collect in the Appendix some DTOC problems that have appeared in the literature.


discrete-time optimal control stagewise Newton's method trust region method 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints”,SIAM J. Control and Optimization, 20:221–246, 1982.Google Scholar
  2. 2.
    B.D. Craven,Mathematical Programming and Control Theory, Chapman and Hall, London, 1978.Google Scholar
  3. 3.
    T.B. Culver and C.A. Shoemaker “Dynamic optimal control for groundwater remediation with flexible management periods”,Water Resources Research, 28:629–641, 1992.Google Scholar
  4. 4.
    P. Dyer and S.R. Mcreynolds,The computation and theory of optimal control, Academic Press, 1970.Google Scholar
  5. 5.
    R. Fletcher,Practical Methods of Optimization, John Wiley & Sons, Ltd., 1987.Google Scholar
  6. 6.
    D.M. Gay, “Computing optimal local constrained steps”,SIAM J. Sci. Stat. Comput., 2:186–197, 1981.Google Scholar
  7. 7.
    D. Jacobson and D. Mayne,Differential Dynamic Programming, Elsevier Sci. Publ., 1970.Google Scholar
  8. 8.
    L.-Z. Liao and C.A. Shoemaker, “Convergence in unconstrained discrete-time differential dynamic programming”,IEEE Trans. Automat. Contr., 36:692–706, 1991.Google Scholar
  9. 9.
    L.-Z. Liao and C.A. Shoemaker, “Advantages of differential dynamic programming over Newton's method for discrete-time optimal control problems”, Technical Report ctc92tr97, Advanced Computing Research Institute, Cornell University, 1992.Google Scholar
  10. 10.
    D.G. Luenberger,Linear and Nonlinear Programming, Addison-Wesley, Reading, Mass., 1984.Google Scholar
  11. 11.
    D. Mayne, “A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems”,Intnl. J. Control, 3:85–95, 1966.Google Scholar
  12. 12.
    J.J. Moré, “Recent developments in algorithms and software for trust region methods”. In Bachem, A., Grötschel, M., and Korte, B., editors,Mathematical Programming, pages 258–287, New York, 1983. Springer-Verlag.Google Scholar
  13. 13.
    J.J. Moré and D.C. Sorensen, Computing a trust region step,SIAM J. Sci. Stat. Comput., 4:553–572, 1983.Google Scholar
  14. 14.
    D.M. Murray and S.J. Yakowitz, The application of optimal control methodology to nonlinear programming problems,Mathematical Programming, 21:331–347, 1981.Google Scholar
  15. 15.
    D.M. Murray and S.J. Yakowitz, “Differential dynamic programming and Newton's method for discrete optimal control problems”,J. of Optimization Theory and Applications, 43:395–414, 1984.Google Scholar
  16. 16.
    J. Nocedal and Y. Yuan, “Combining trust region and line search techniques”, Technical Report, Dept. of EE and Computer Science, Northwestern University, 1992.Google Scholar
  17. 17.
    J.F.A. De and O. Pantoja, “Differential dynamic programming and Newton's method”,Intnl. J. Control, 47:1539–1553, 1988.Google Scholar
  18. 18.
    G. Di, Pillo, L. Grippo and F. Lampariello, “A class of structured quasi-Newton algorithms for optimal control problems”, In Rauch, H.E., editor,IFAC Applications of nonlinear programming to optimization and control, pages 101–107. International Federation of Automatic Control, Pergamon Press, New York, 1983.Google Scholar
  19. 19.
    M.J.D. Powell, “A new algorithm for unconstrained optimization”, in Rosen, J.B., Mangasarian, O., and Ritter, K., editors,Nonlinear Programming, pages 31–65, New York, 1970, Academic Press.Google Scholar
  20. 20.
    D. Ralph, “A parallel method for discrete-time optimal control problems”, Technical Report ctc93tr118, Advanced Computing Research Institute, Cornell University, 1993.Google Scholar
  21. 21.
    S.J. Wright, “Partitioned dynamic programming for optimal control”,SIAM Journal on Optimization, 1:620–642, 1991.Google Scholar
  22. 22.
    S. Yakowitz and B. Rutherford, “Computational aspects of discrete-time optimal control”,Applied Mathematics and Computation, 15:29–45, 1984.Google Scholar

Copyright information

© Kluwer Academic Publishers 1995

Authors and Affiliations

  • Thomas F. Coleman
    • 1
  • Aiping Liao
    • 2
  1. 1.Computer Science DepartmentCornell UniversityIthaca
  2. 2.Advanced Computing Research Institute and Center for Applied MathematicsCornell UniversityIthaca

Personalised recommendations