Skip to main content
SpringerLink
Log in

The maximum principle, Bellman's equation, and Carathéodory's work

  • Historical Paper
  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

One of the most important and deep results in optimal control theory is the maximum principle attributed to Hestenes (1950) and in particular to Boltyanskii, Gamkrelidze, and Pontryagin (1956). Another prominent result is known as the Bellman equation, which is associated with Isaacs' and Bellman's work (later than 1951). However, precursors of both the maximum principle and the Bellman equation can already be found in Carathéodory's book of 1935 (Ref. 1a), the first even in his earlier work of 1926 which is given in Ref. 2. This is not a widely acknowledged fact. The present tutorial paper traces Carathéodory's approach to the calculus of variations, once called the “royal road in the calculus of variations,” and shows that famous results in optimal control theory, including the maximum principle and the Bellman equation, are consequences of Carathéodory's earlier results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Carathéodory, C.,Variationsrechnung und partielle Differentialgleichungen erster Ordnung, Teubner, Leipzig, Germany, 1935.

    Google Scholar 

  2. Carathéodory, C.,Calculus of Variations and Partial Differential Equations of the First Order, Parts 1 and 2, Holden-Day, San Francisco, California, 1965.

    Google Scholar 

  3. Carathéodory, C.,Die Methode der geodätischen Äquidistanten und das Problem von Lagrange, Acta Mathematica, Vol. 47, pp. 199–236, 1926.

    Google Scholar 

  4. Carathéodory, C.,Die Methode der geodätischen Äquidistanten und das Problem von Lagrange, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 212–248, 1954.

    Google Scholar 

  5. Carathéodory, C.,Sui Campi di Estremali Uscenti da un Punto e Riempienti Tutto lo Spazio, Estratto di una Lettera a Leonida Tonelli, Bollettino dell'Unione Matematica Italiana, Vol. 2, pp. 2–6, 48–52, 81–87, 1923.

    Google Scholar 

  6. Carathéodory, C.,Sui Campi di Estremali Uscenti da un Punto e Riempienti Tutto lo Spazio, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 188–202, 1954.

    Google Scholar 

  7. Theon Alexandreus, [Commentary on the Beginning of Ptolemy's Mathematical Syntaxis (Almagest)], Commentaire de Théon d'Alexandrie, sur le Premier Livre de la Composition Mathématique de Ptolemée Traduit pour la Première Fois du Grec en Français sur la Manuscrits de la Bibliothèque du Roi, Translated by M. l'Abbé Halma, Merlin, Paris, France, Vol. 1, 1821 (in ancient Greek and French).

  8. Boerner, H.,Carathéodory's Eingang zur Variationsrechnung, Jahresbericht der Deutschen Mathematiker Vereinigung, Vol. 56, pp. 31–58, 1953.

    Google Scholar 

  9. Bellman, R.,The Theory of Dynamic Programming, Bulletin of the American Mathematical Society, Vol. 60, pp. 503–516, 1954.

    Google Scholar 

  10. Bellman, R.,Dynamic Programming and a New Formalism in the Calculus of Variations, Proceedings of the National Academy of Sciences USA, Vol. 40, pp. 231–235, 1954.

    Google Scholar 

  11. Isaacs, R.,Games of Pursuit, Paper No. P-257, RAND Corporation, Santa Monica, California, 1951.

    Google Scholar 

  12. Isaacs, R.,Differential Games, John Wiley and Sons, New York, New York, 1965.

    Google Scholar 

  13. Isaacs, R.,Differential Games, 3rd Printing, Krieger, New York, New York, 1975.

    Google Scholar 

  14. Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Acta Eruditorum, pp. 211–217, 1697.

  15. Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Jacobi Bernoulli Basileensis Opera, Cramer and Philibert, Geneva, Switzerland, No. LXXV, pp. 768–778, 1744.

    Google Scholar 

  16. Bernoulli, Jacob,Solutio Problematum Fraternorum, peculiari Programmate Cal. Jan. 1697 Groningae, nec non Actorum Lips. mense Jun. & Dec. 1696 & Feb. 1697 propositorum: una cum Propositione reciproca aliorum, Die Streitschriften von Jacob and Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, pp. 271–282, 1991.

    Google Scholar 

  17. Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Acta Eruditorum, p. 269, 1696.

  18. Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Johannis Bernoulli Basileensis Opera Omnia, Bousquet, Lausanne and Geneva, Switzerland, No. XXX, Vol. 1, p. 161, 1742.

    Google Scholar 

  19. Bernoulli, Johann,Problema novum ad cujus solutionem Mathematici invitantur, Die Streitschriften von Jacob und Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, p. 212, 1991.

    Google Scholar 

  20. Euler, L.,Methodus inveniendi Lineas Curvas maximi minimive proprietate gaudentes, sive Solutio Problematis Isoperimetrici latissimo sensu accepti, Bousquet, Lausanne and Geneva, Switzerland, 1744.

    Google Scholar 

  21. Euler, L.,Methodus inveniendi Lineas Curvas maximi minimive proprietate gaudentes, sive Solutio Problematis Isoperimetrici latissimo sensu accepti, Leonhardi Euleri Opera Omnia, Edited by C. Carathéodory, Series Prima XXIV, Vol. 24, Orell Fuessli, Turici, Switzerland, 1952.

    Google Scholar 

  22. Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Acta Eruditorum, pp. 213–228, 1701.

  23. Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Jacobi Bernoulli Basileensis Opera, Cramer and Philibert, Geneva, Switzerland, No. XCVI, pp. 895–920, 1744.

    Google Scholar 

  24. Bernoulli, Jacob,Analysis magni Problematis Isoperimetrici, Die Streitschriften von Jacob und Johann Bernoulli, Edited by the Naturforschende Gesellschaft in Basel, Birkhäuser, Basel, Switzerland, pp. 485–505, 1991.

    Google Scholar 

  25. Carathéodory, C.,Basel und der Beginn der Variationsrechnung, Publication in Honor of the 60th Birthday of Professor Dr. Andreas Speiser, Zürich, Switzerland, pp. 1–18, 1945.

  26. Carathéodory, C.,Basel und der Beginn der Variationsrechnung, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 2, pp. 108–128, 1955.

    Google Scholar 

  27. Beltrami, E.,Sulla Teoria delle Linee Geodetiche, Rendiconti del Reale Istituto Lombardo, Serie II, Vol. 1, pp. 708–718, 1868.

    Google Scholar 

  28. Beltrami, E.,Sulla Teoria delle Linee Geodetiche, Opere Matematiche di Eugenio Beltrami, Edited by the Facoltà di Scienze della R. Università di Roma, Hoepli, Milano, Italy, Vol. 1, pp. 366–373, 1902.

    Google Scholar 

  29. Carathéodory, C.,Bemerkungen über die Eulerschen Differentialgleichungen der Variationsrechnung, Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch-physikalische Klasse, pp. 40–42, 1931.

  30. Carathéodory, C.,Bemerkungen über die Eulerschen Differentialgleichungen der Variationsrechnung, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 249–252, 1954.

    Google Scholar 

  31. Tonelli, L.,Fondamenti di Calcolo delle Variazioni, Zanichelli, Bologna, Italy, 1923.

    Google Scholar 

  32. Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, Research Memorandum RM-100, RAND Corporation, Santa Monica, California, 1950.

    Google Scholar 

  33. Boltyanskii, V. G., Gamkrelidze, R. V., andPontryagin, L. S.,On the Theory of Optimal Processes, Doklady Akademii Nauk SSSR, Vol. 110, pp. 7–10, 1956 (in Russian).

    Google Scholar 

  34. Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., andMishchenko, E. F.,The Mathematical Theory of Optimal Processes, Fizmatgiz, Moscow, Russia, 1961 (in Russian).

    Google Scholar 

  35. Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., andMishchenko, E. F.,The Mathematical Theory of Optimal Processes, Interscience Publishers, New York, New York, 1962.

    Google Scholar 

  36. Ioffe, A. D., andTihomirov, V. M.,Theory of Extremal Problems, North-Holland, Amsterdam, Holland, 1979.

    Google Scholar 

  37. Isaacs, R.,Some Fundamentals of Differential Games, Topics in Differential Games, Edited by A. Blaquière, North-Holland Publishing Company, Amsterdam, Holland, pp. 1–42, 1973.

    Google Scholar 

  38. Carathéodory, C.,Exemples Particuliers et Théorie Générale dans le Calcul des Variations, L'Enseigement Mathématique, Vol. 34, pp. 255–261, 1935.

    Google Scholar 

  39. Carathéodory, C.,Exemples Particuliers et Théorie Générale dans le Calcul des Variations, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 1, pp. 306–311, 1954.

    Google Scholar 

  40. Miele, A., Wang, T., andMelvin, W. W.,Optimal Abort Landing Trajectories in the Presence of Windshear, Journal of Optimization Theory and Applications, Vol. 55, pp. 165–202, 1987.

    Article  MathSciNet  Google Scholar 

  41. Miele, A., Wang, T., andMelvin, W. W.,Quasi-Steady Flight to Quasi-Steady Flight Transition for Abort Landing in a Windshear: Trajectory Optimization and Guidance, Journal of Optimization Theory and Applications, Vol. 58, pp. 165–207, 1988.

    Google Scholar 

  42. Bulirsch, R., Montrone, F., andPesch, H. J.,Abort Landing in the Presence of Windshear as a Minimax Optimal Control Problem, Part 1: Necessary Conditions, Part 2: Multiple Shooting and Homotopy, Journal of Optimization Theory and Applications, Vol. 70, pp. 1–23, 223–254, 1991.

    Article  Google Scholar 

  43. Berkmann, P., andPesch, H. J.,Abort Landing of an Airplane under Different Windshear Conditions (in preparation).

  44. Carathéodory, C.,The Beginning of Research in the Calculus of Variations, Osiris, Vol. 3, pp. 224–240, 1937.

    Article  Google Scholar 

  45. Carathéodory, C.,The Beginning of Research in the Calculus of Variations, Gesammelte Mathematische Schriften, Edited by the Bayerische Akademie der Wissenschaften, C. H. Beck'sche Verlagsbuchhandlung, München, Germany, Vol. 2, pp. 93–107, 1955.

    Google Scholar 

  46. Carathéodory, C.,Variationsrechnung und Partielle Differentialgleichungen Erster Ordnung. Mit Erweiterungen zur Variationsrechnung Mehrfacher Integrale, zur Steuerungstheorie und zur Dualitätstheorie, Edited by R. Klötzler, Teubner, Stuttgart, Germany, 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Technology, Munich, Germany

    H. J. Pesch (Privatdozent für Mathematik)

  2. Department of Mathematics, University of Technology, Munich, Germany

    R. Bulirsch (Professor of Mathematics)

Authors
  1. H. J. Pesch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Bulirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This paper is in honor of the seventieth birthday of Professor Angelo Miele. It was exactly 70 years ago, in the months of August and September 1922, around the time of the birth of Professor Miele, when Constantin Carathéodory wrote a paper in Italian entitled “Sui Campi di Estremali Uscenti da un Punto e Riempienti Tutto lo Spazio—On Extremal Fields Emanating from a Point and Covering All the Space” (Ref. 3). This paper was inspired by the great Italian mathematician Leonida Tonelli, one of Angelo Miele's academic teachers.

The authors would like to thank Professors Sandra Hayes-Widmann and Donald Smith for checking the English of their manuscript, Dr. Uwe Dubielzig for providing them with the entire classical literature on the bull-hide story of the foundation of Carthage and for drawing their attention to Ref. 4, Dr. Ingeborg Neske for directing them to the codex of the City Library of Nuremberg, and the City Library of Nuremberg for permission to publish the page of the codex given in Fig. 1.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pesch, H.J., Bulirsch, R. The maximum principle, Bellman's equation, and Carathéodory's work. J Optim Theory Appl 80, 199–225 (1994). https://doi.org/10.1007/BF02192933

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02192933

Key Words

Search

Navigation