Acta Applicandae Mathematica

, Volume 1, Issue 1, pp 17–41 | Cite as

On the Hamilton-Jacobi-Bellman equations

  • P. L. Lions
Article

Abstract

We consider general problems of optimal stochastic control and the associated Hamilton-Jacobi-Bellman equations. We recall first the usual derivation of the Hamilton-Jacobi-Bellman equations from the Dynamic Programming Principle. We then show and explain various results, including (i) continuity results for the optimal cost function, (ii) characterizations of the optimal cost function as the maximum subsolution, (iii) regularity results, and (iv) uniqueness results. We also develop the recent notion of viscosity solutions of Hamilton-Jacobi-Bellman equations.

AMS (MOS) subject classification (1980)

93E20 35J65 35K60 60J60 

Key words

Optimal stochastic control diffusion processes Hamilton-Jacobi-Bellman equations viscosity solutions Dynamic Programming Principle 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alexandrov, A. D.:Ucen. Zap. Leningard Gos. Univ. Ser. Fiz. Nauk 37 (1939), 3–35 (in Russian).Google Scholar
  2. 2.
    Belbas, S. and Lenhart, S.: A system of nonlinear partial differential equations arising in the optimal control of stochastic systems with switching costs', preprint.Google Scholar
  3. 3.
    Bellman, R.:Dynamic Programming, Princeton Univ. Press, Princeton, N.J., 1957.Google Scholar
  4. 4.
    Bensoussan, A. and Lions, J. L.:Applications des inéquations variationnelles en contrôle stochastique, Dunod, Paris, 1978.MATHGoogle Scholar
  5. 5.
    Brèzis, H. and Evans, L. C.:Arch. Rat. Mech. Anal. 41 (1979), 1–13.CrossRefGoogle Scholar
  6. 6.
    Busemann, H.:Convex Surfaces, Interscience, New York, 1958.MATHGoogle Scholar
  7. 7.
    Coron, J. M. and Lions, P. L.: To appear.Google Scholar
  8. 8.
    Crandall, M. G., Evans, L. C., and Lions, P. L.: ‘Some properties of viscosity solutions of Hamilton-Jacobi equations’, to appear inTrans. Amer. Math. Soc. (1983).Google Scholar
  9. 9.
    Crandall, M. G. and Lions, P. L.:Comptes-Rendus Acad. Sc. Paris Sér. A,252 (1981), 183–186.MathSciNetGoogle Scholar
  10. 10.
    Crandall, M. G. and Lions, P. L.: ‘Viscosity solutions of Hamilton-Jacobi equations’, to appear inTrans. Amer. Math. Soc. (1983).Google Scholar
  11. 11.
    Crandall, M. G. and Lions, P. L.: ‘Two approximations of solutions of Hamilton-Jacobi equations’, preprint.Google Scholar
  12. 12.
    Evans, L. C.:Comm. Pure Appl. Math. (1982).Google Scholar
  13. 13.
    Evans, L. C.: ‘Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators’, to appear inTrans. Amer. Math. Soc. (1983).Google Scholar
  14. 14.
    Evans, L. C. and Friedman, A.:Trans. Amer. Math. Soc. 253 (1979), 365–389.MATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    Evans, L. C. and Lions, P. L.:Comptes-Rendus Acad. Sc. Paris Sér. A,290 (1980), 1049–1052.MATHMathSciNetGoogle Scholar
  16. 16.
    Fleming, W. H. and Rishel, R.:Deterministic and Stochastic Optimal Control, Springer, Berlin, 1975.MATHGoogle Scholar
  17. 17.
    Genis, T. L. and Krylov, N. V.:Th. Proba. Appl. 21 (1976), 148–152.MATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Itô, K.: to appear.Google Scholar
  19. 19.
    Jensen, R. and Lions, P. L.: ‘Some asymptotic problems in fully nonlinear equations and optimal stochastic control’, preprint (to appear inAnn. Sc. Norm. Sup. Pisa).Google Scholar
  20. 20.
    Krylov, N. V.:Th. Proba. Appl. 17 (1972), 114–131.MATHCrossRefGoogle Scholar
  21. 21.
    Krylov, N. V.:Controlled Diffusion Processes, Springer, Berlin, 1980.MATHGoogle Scholar
  22. 22.
    Krylov, N. V.:Math. U.S.S.R. Izv. 7 (1973), 691–708.MATHMathSciNetCrossRefGoogle Scholar
  23. 23.
    Krylov, N. V.:Math. U.S.S.R. Izv. 6 (1972), 249–262.MATHCrossRefGoogle Scholar
  24. 24.
    Krylov, N. V.: ‘Control of the diffusion-type processes’,Proc. Internat. Congress of Mathematicians, Helsinki, 1978.Google Scholar
  25. 25.
    Krylov, N. V.:Math. U.S.S.R. Sbornik,32 (1980), 133–149.CrossRefGoogle Scholar
  26. 26.
    Krylov, N. V.:Th. Proba. Appl. 18 (1973), 54–63.MATHCrossRefGoogle Scholar
  27. 27.
    Krylov, N. V.:Math. U.S.S.R. Izv. 8 (1974), 233–254.CrossRefGoogle Scholar
  28. 28.
    Krylov, N. V.:Sibirsk Mat. 17 (1976), 226–236.MATHMathSciNetCrossRefGoogle Scholar
  29. 29.
    Krylov, N. V. and Pragarauskas, G.:Liet. Mat. Rink 21 (1981), 101–110 (in Russian).MATHMathSciNetGoogle Scholar
  30. 30.
    Lasry, J. M. and Lions, P. L.: To appear.Google Scholar
  31. 31.
    Lenhart, S.: ‘Partial differential equations from dynamic programming equations’, PhD Thesis, Univ. of Kentucky-Lexington, 1981.Google Scholar
  32. 32.
    Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 1: The dynamic programming principle and applications’, to appear inComm. P.D.E. (1983).Google Scholar
  33. 33.
    Lions, P. L.:Comm. Pure Appl. Math. 34 (1981), 121–147.MATHMathSciNetGoogle Scholar
  34. 34.
    Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 2: Viscosity solutions and uniqueness’, to appear inComm. P.D.E. (1983).Google Scholar
  35. 35.
    Lions, P. L.: ‘Optimal stochastic control of diffusion type processes and Hamilton-Jacobi-Bellman equations’, in W. H. Fleming and L. Gorostiza (eds.),Advances in Filtering and Optimal Stochastic Control, Springer, Berlin, 1982.Google Scholar
  36. 36.
    Lions, P. L.: ‘Fully nonlinear elliptic equations and applications’,Nonlinear Analysis, Function Spaces and Applications, Teubner, Leipzig, 1982.Google Scholar
  37. 37.
    Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 3’.Nonlinear partial differential equations and their applications, Collége de France Seminar, Vol. 5, Pitman, London, 1983.Google Scholar
  38. 38.
    Lions, P. L.: ‘Contrôle optimal stochastique et equations de Hamilton-Jacobi-Bellman’,Comptes-Rendus Acad. Sc. Paris Sér. A 295 (1982).Google Scholar
  39. 39.
    Lions, P. L.: ‘A remark on Bony maximum principle, to appear inProc. A.M.S. (1983).Google Scholar
  40. 40.
    Lions, P. L.:Comptes-Rendus Paris,289 (1979), 329–332.MATHGoogle Scholar
  41. 41.
    Lions, P. L.: ‘Equations de Hamilton-Jacobi-Bellman’,Seminaire Goulaouic Schwartz 1979–1980, Ecole Polytechnique, Palaiseau.Google Scholar
  42. 42.
    Lions, P. L.: ‘Optimal stochastic control and Hamilton-Jacobi-Bellman equations’,Mathematical Control Theory, Banach Centre Publications, Warsaw, to appear.Google Scholar
  43. 43.
    Lions, P. L.:Comm. PDE 5 (1980), 753–771.MATHCrossRefGoogle Scholar
  44. 44.
    Lions, P. L.:Comptes-Rendus Paris 288 (1979), 339–342.MATHGoogle Scholar
  45. 45.
    Lions, P. L.:Comptes-Rendus Paris 287 (1978), 747–750.MATHGoogle Scholar
  46. 46.
    Lions, P. L.:Acta Math. 146 (1981), 121–147.CrossRefGoogle Scholar
  47. 47.
    Lions, P. L.: ‘Bifurcation and optimal stochastic control’, to appear inNonlinear Anal. TMA.Google Scholar
  48. 48.
    Lions, P. L.: to appear.Google Scholar
  49. 49.
    Lions, P. L.:Generated Solutions of Hamilton-Jacobi Equations, Pitman, London, 1982.Google Scholar
  50. 50.
    Lions, P. L.: to appear.Google Scholar
  51. 51.
    Lions, P. L.: to appear.Google Scholar
  52. 52.
    Lions, P. L. and Menaldi, J. L.:SIAM J. Control Optim. 20 (1982), 58–81.MATHMathSciNetCrossRefGoogle Scholar
  53. 53.
    Lions, P. L. and Menaldi, J. L.:SIAM J. Control Optim. 20 (1982), 82–95.MATHMathSciNetCrossRefGoogle Scholar
  54. 54.
    Lions, P. L. and Menaldi, J. L.:Comptes-Rendus Paris,287 (1978), 503–506.MATHMathSciNetGoogle Scholar
  55. 55.
    Lions, P. L. and Mercier, B.:RAIRO 14 (1980), 369–393.MATHMathSciNetGoogle Scholar
  56. 56.
    Lions, P. L. and Nisio, M.: ‘A uniqueness result for the semigroup associated with the Hamilton-Jacobi-Bellman operator’,Proc. Japan Acad. Google Scholar
  57. 57.
    Lions, P. L., Papanicolaou, G. and Varadhan, S. R. S.: to appear.Google Scholar
  58. 58.
    Nisio, M.: ‘Some remarks on stochastic optimal controls’,Proc. 3rd. U.S.S.R.-Japan Symp. Proba. Theory, Lecture Notes in Maths 550, Springer, Berlin, 1976.Google Scholar
  59. 59.
    Nisio, M.:Lectures on Stochastic Control Theory, ISI Lecture Notes 9. Macmillan India Bombay.Google Scholar
  60. 60.
    Nisio, M.:Publ. RIMS Kyoto Univ. 12 (1976), 513–537.MATHMathSciNetCrossRefGoogle Scholar
  61. 61.
    Nisio, M.: ‘On stochastic optimal controls and envelope of Markovian semigroups’,Proc. Int. Symp. SDE, Kyoto 1976, Kinskuniga, Tokyo, 1978.Google Scholar
  62. 62.
    Quadrat, J. P.: ‘Analyse numèrique de l'équation de Bellman stochastique’, Rapport Laboria 140 (1974), INRIA, Rocquencourt.Google Scholar
  63. 63.
    Safonov, M. V.:Math. U.S.S.R. Sbornik 31 (1977), 231–248.MATHMathSciNetCrossRefGoogle Scholar
  64. 64.
    Safonov, M. V.:Math. U.S.S.R. Sbornik 34 (1978), 521–526.MATHMathSciNetCrossRefGoogle Scholar
  65. 65.
    Stroock, D. W. and Varadhan, S. R. S.:Multidimensional Diffusion Processes, Springer, Berlin, 1979.MATHGoogle Scholar
  66. 66.
    Trudinger, N. S.: ‘Elliptic equations in nondivergence form’,Proc. Miniconference on Partial Differential Equations, Canberra, 1981.Google Scholar
  67. 67.
    Trudinger, N. S.: ‘Fully nonlinear uniformly elliptic equations under natural structure conditions’, to appear inTrans. Amer. Math. Soc. (1983).Google Scholar

Copyright information

© D. Reidel Publishing Co. 1983

Authors and Affiliations

  • P. L. Lions
    • 1
  1. 1.Ceremade, Universitè Paris IX-DauphineParis Cedex 16France

Personalised recommendations