Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

  • J. C. Dunn
  • D. P. Bertsekas
Contributed Papers


Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN3 asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.

Key Words

Unconstrained optimal control Newton's method dynamic programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Luenberger, D. G.,Optimization by Vector Space Methods, Wiley, New York, New York, 1969.Google Scholar
  2. 2.
    Dennis, J. E., Jr., andSchnabel, R. B.,Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey, 1983.Google Scholar
  3. 3.
    Murray, D. M., andYakowitz, S.,Differential Dynamic Programming and Newton's Method for Discrete Optimal Control Problems, Journal of Optimization Theory and Applications, Vol. 43, pp. 395–414, 1984.Google Scholar
  4. 4.
    Mayne, D. Q.,A Second-Order Gradient Method for Determining Optimal Trajectories of Nonlinear Discrete-Time Systems, International Journal on Control, Vol. 3, pp. 85–95, 1966.Google Scholar
  5. 5.
    Bellman, R. E.,Dynamic Programming, Princeton University Press, Princeton, New Jersey, 1957.Google Scholar
  6. 6.
    Jacobson, D. H., andMayne, D. Q.,Differential Dynamic Programming, American Elsevier, New York, New York, 1970.Google Scholar
  7. 7.
    Polak, E.,Computational Methods in Optimization, Academic Press, New York, New York, 1971.Google Scholar
  8. 8.
    Bliss, G. A.,Lectures on the Calculus of Variations, University of Chicago Press, Chicago, Illinois, 1946.Google Scholar
  9. 9.
    Hestenes, M.,Calculus of Variations and Optimal Control, Robert E. Krieger Publishing Company, Huntington, New York, 1980.Google Scholar
  10. 10.
    Griewank, A.,A Superlinear Convergence of Secant Methods on Mildly Nonlinear Problems in Hilbert Space (to appear).Google Scholar
  11. 11.
    Kelley, C. T., andSachs, E.,A Pointwise Quasi-Newton Method for Unconstrained Optimal Control Problems, Numerische Mathematik, Vol. 55, pp. 159–176, 1989.Google Scholar
  12. 12.
    Dreyfus, S. E.,Dynamic Programming and the Calculus of Variations, Academic Press, New York, New York, 1965.Google Scholar
  13. 13.
    Keller, H.,Two-Point Boundary-Value Problems, Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, 1976.Google Scholar
  14. 14.
    Merriam, C. W., III,An Algorithm for the Iterative Solution of a Class of Two-Point Boundary-Value Problems, SIAM Journal on Control and Optimization, Series A, Vol. 2, pp. 1–10, 1964.Google Scholar
  15. 15.
    Mitter, S. K.,Successive Approximation Methods for the Solution of Optimal Control Problems, Automatica, Vol. 3, pp. 135–149, 1966.Google Scholar
  16. 16.
    Pantoja, J.,Differential Dynamic Programming and Newton's Method, International Journal on Control, Vol. 47, pp. 1539–1553, 1988.Google Scholar

Copyright information

© Plenum Publishing Corporation 1989

Authors and Affiliations

  • J. C. Dunn
    • 1
  • D. P. Bertsekas
    • 2
  1. 1.Department of MathematicsNorth Carolina State UniversityRaleigh
  2. 2.Department of Electrical Engineering and Computer ScienceMassachusetts Institute of TechnologyCambridge

Personalised recommendations