Advertisement

Journal of Optimization Theory and Applications

, Volume 80, Issue 2, pp 199–225 | Cite as

The maximum principle, Bellman's equation, and Carathéodory's work

  • H. J. Pesch
  • R. Bulirsch
Historical Paper

Abstract

One of the most important and deep results in optimal control theory is the maximum principle attributed to Hestenes (1950) and in particular to Boltyanskii, Gamkrelidze, and Pontryagin (1956). Another prominent result is known as the Bellman equation, which is associated with Isaacs' and Bellman's work (later than 1951). However, precursors of both the maximum principle and the Bellman equation can already be found in Carathéodory's book of 1935 (Ref. 1a), the first even in his earlier work of 1926 which is given in Ref. 2. This is not a widely acknowledged fact. The present tutorial paper traces Carathéodory's approach to the calculus of variations, once called the “royal road in the calculus of variations,” and shows that famous results in optimal control theory, including the maximum principle and the Bellman equation, are consequences of Carathéodory's earlier results.

Key Words

Maximum principle Bellman equation Carathéodory's work calculus of variations optimal control theory history of the calculus of variations history of optimal control theory 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1a.
    Carathéodory, C.,Variationsrechnung und partielle Differentialgleichungen erster Ordnung, Teubner, Leipzig, Germany, 1935.Google Scholar
  2. 1b.
    Carathéodory, C.,Calculus of Variations and Partial Differential Equations of the First Order, Parts 1 and 2, Holden-Day, San Francisco, California, 1965.Google Scholar
  3. 2a.
    Carathéodory, C.,Die Methode der geodätischen Äquidistanten und das Problem von Lagrange, Acta Mathematica, Vol. 47, pp. 199–236, 1926.Google Scholar
  4. 2b.
    Carathéodory, C.,Die Methode der geodätischen Äquidistanten und das Problem von Lagrange, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 212–248, 1954.Google Scholar
  5. 3a.
    Carathéodory, C.,Sui Campi di Estremali Uscenti da un Punto e Riempienti Tutto lo Spazio, Estratto di una Lettera a Leonida Tonelli, Bollettino dell'Unione Matematica Italiana, Vol. 2, pp. 2–6, 48–52, 81–87, 1923.Google Scholar
  6. 3b.
    Carathéodory, C.,Sui Campi di Estremali Uscenti da un Punto e Riempienti Tutto lo Spazio, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 188–202, 1954.Google Scholar
  7. 4.
    Theon Alexandreus,Open image in new window Open image in new window [Commentary on the Beginning of Ptolemy's Mathematical Syntaxis (Almagest)], Commentaire de Théon d'Alexandrie, sur le Premier Livre de la Composition Mathématique de Ptolemée Traduit pour la Première Fois du Grec en Français sur la Manuscrits de la Bibliothèque du Roi, Translated by M. l'Abbé Halma, Merlin, Paris, France, Vol. 1, 1821 (in ancient Greek and French).Google Scholar
  8. 5.
    Boerner, H.,Carathéodory's Eingang zur Variationsrechnung, Jahresbericht der Deutschen Mathematiker Vereinigung, Vol. 56, pp. 31–58, 1953.Google Scholar
  9. 6.
    Bellman, R.,The Theory of Dynamic Programming, Bulletin of the American Mathematical Society, Vol. 60, pp. 503–516, 1954.Google Scholar
  10. 7.
    Bellman, R.,Dynamic Programming and a New Formalism in the Calculus of Variations, Proceedings of the National Academy of Sciences USA, Vol. 40, pp. 231–235, 1954.Google Scholar
  11. 8.
    Isaacs, R.,Games of Pursuit, Paper No. P-257, RAND Corporation, Santa Monica, California, 1951.Google Scholar
  12. 9a.
    Isaacs, R.,Differential Games, John Wiley and Sons, New York, New York, 1965.Google Scholar
  13. 9b.
    Isaacs, R.,Differential Games, 3rd Printing, Krieger, New York, New York, 1975.Google Scholar
  14. 10a.
    Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Acta Eruditorum, pp. 211–217, 1697.Google Scholar
  15. 10b.
    Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Jacobi Bernoulli Basileensis Opera, Cramer and Philibert, Geneva, Switzerland, No. LXXV, pp. 768–778, 1744.Google Scholar
  16. 10c.
    Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Die Streitschriften von Jacob and Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, pp. 271–282, 1991.Google Scholar
  17. 11a.
    Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Acta Eruditorum, p. 269, 1696.Google Scholar
  18. 11b.
    Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Johannis Bernoulli Basileensis Opera Omnia, Bousquet, Lausanne and Geneva, Switzerland, No. XXX, Vol. 1, p. 161, 1742.Google Scholar
  19. 11c.
    Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Die Streitschriften von Jacob und Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, p. 212, 1991.Google Scholar
  20. 12a.
    Euler, L.,Methodus inveniendi Lineas Curvas maximi minimive proprietate gaudentes, sive Solutio Problematis Isoperimetrici latissimo sensu accepti, Bousquet, Lausanne and Geneva, Switzerland, 1744.Google Scholar
  21. 12b.
    Euler, L.,Methodus inveniendi Lineas Curvas maximi minimive proprietate gaudentes, sive Solutio Problematis Isoperimetrici latissimo sensu accepti, Leonhardi Euleri Opera Omnia, Edited by C. Carathéodory, Series Prima XXIV, Vol. 24, Orell Fuessli, Turici, Switzerland, 1952.Google Scholar
  22. 13a.
    Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Acta Eruditorum, pp. 213–228, 1701.Google Scholar
  23. 13b.
    Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Jacobi Bernoulli Basileensis Opera, Cramer and Philibert, Geneva, Switzerland, No. XCVI, pp. 895–920, 1744.Google Scholar
  24. 13c.
    Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Die Streitschriften von Jacob und Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, pp. 485–505, 1991.Google Scholar
  25. 14a.
    Carathéodory, C.,Basel und der Beginn der Variationsrechnung, Publication in Honor of the 60th Birthday of Professor Dr. Andreas Speiser, Zürich, Switzerland, pp. 1–18, 1945.Google Scholar
  26. 14b.
    Carathéodory, C.,Basel und der Beginn der Variationsrechnung, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 2, pp. 108–128, 1955.Google Scholar
  27. 15a.
    Beltrami, E.,Sulla Teoria delle Linee Geodetiche, Rendiconti del Reale Istituto Lombardo, Serie II, Vol. 1, pp. 708–718, 1868.Google Scholar
  28. 15b.
    Beltrami, E.,Sulla Teoria delle Linee Geodetiche, Opere Matematiche di Eugenio Beltrami, Edited by the Facoltà di Scienze della R. Università di Roma, Hoepli, Milano, Italy, Vol. 1, pp. 366–373, 1902.Google Scholar
  29. 16a.
    Carathéodory, C.,Bemerkungen über die Eulerschen Differentialgleichungen der Variationsrechnung, Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-physikalische Klasse, pp. 40–42, 1931.Google Scholar
  30. 16b.
    Carathéodory, C.,Bemerkungen über die Eulerschen Differentialgleichungen der Variationsrechnung, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 249–252, 1954.Google Scholar
  31. 17.
    Tonelli, L.,Fondamenti di Calcolo delle Variazioni, Zanichelli, Bologna, Italy, 1923.Google Scholar
  32. 18.
    Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, Research Memorandum RM-100, RAND Corporation, Santa Monica, California, 1950.Google Scholar
  33. 19.
    Boltyanskii, V. G., Gamkrelidze, R. V., andPontryagin, L. S.,On the Theory of Optimal Processes, Doklady Akademii Nauk SSSR, Vol. 110, pp. 7–10, 1956 (in Russian).Google Scholar
  34. 20a.
    Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., andMishchenko, E. F.,The Mathematical Theory of Optimal Processes, Fizmatgiz, Moscow, Russia, 1961 (in Russian).Google Scholar
  35. 20b.
    Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., andMishchenko, E. F.,The Mathematical Theory of Optimal Processes, Interscience Publishers, New York, New York, 1962.Google Scholar
  36. 21.
    Ioffe, A. D., andTihomirov, V. M.,Theory of Extremal Problems, North-Holland, Amsterdam, Holland, 1979.Google Scholar
  37. 22.
    Isaacs, R.,Some Fundamentals of Differential Games, Topics in Differential Games, Edited by A. Blaquière, North-Holland Publishing Company, Amsterdam, Holland, pp. 1–42, 1973.Google Scholar
  38. 23a.
    Carathéodory, C.,Exemples Particuliers et Théorie Générale dans le Calcul des Variations, L'Enseigement Mathématique, Vol. 34, pp. 255–261, 1935.Google Scholar
  39. 23b.
    Carathéodory, C.,Exemples Particuliers et Théorie Générale dans le Calcul des Variations, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 306–311, 1954.Google Scholar
  40. 24.
    Miele, A., Wang, T., andMelvin, W. W.,Optimal Abort Landing Trajectories in the Presence of Windshear, Journal of Optimization Theory and Applications, Vol. 55, pp. 165–202, 1987.CrossRefMathSciNetGoogle Scholar
  41. 25.
    Miele, A., Wang, T., andMelvin, W. W.,Quasi-Steady Flight to Quasi-Steady Flight Transition for Abort Landing in a Windshear: Trajectory Optimization and Guidance, Journal of Optimization Theory and Applications, Vol. 58, pp. 165–207, 1988.Google Scholar
  42. 26.
    Bulirsch, R., Montrone, F., andPesch, H. J.,Abort Landing in the Presence of Windshear as a Minimax Optimal Control Problem, Part 1: Necessary Conditions, Part 2: Multiple Shooting and Homotopy, Journal of Optimization Theory and Applications, Vol. 70, pp. 1–23, 223–254, 1991.CrossRefGoogle Scholar
  43. 27.
    Berkmann, P., andPesch, H. J.,Abort Landing of an Airplane under Different Windshear Conditions (in preparation).Google Scholar
  44. 28a.
    Carathéodory, C.,The Beginning of Research in the Calculus of Variations, Osiris, Vol. 3, pp. 224–240, 1937.CrossRefGoogle Scholar
  45. 28b.
    Carathéodory, C.,The Beginning of Research in the Calculus of Variations, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 2, pp. 93–107, 1955.Google Scholar
  46. 29.
    Carathéodory, C.,Variationsrechnung und Partielle Differentialgleichungen Erster Ordnung. Mit Erweiterungen zur Variationsrechnung Mehrfacher Integrale, zur Steuerungstheorie und zur Dualitätstheorie, Edited by R. Klötzler, Teubner, Stuttgart, Germany, 1993.Google Scholar

Copyright information

© Plenum Publishing Corporation 1994

Authors and Affiliations

  • H. J. Pesch
    • 1
  • R. Bulirsch
    • 2
  1. 1.Department of MathematicsUniversity of TechnologyMunichGermany
  2. 2.Department of MathematicsUniversity of TechnologyMunichGermany

Personalised recommendations