Viscosity Solutions of Second-Order Equations, Stochastic Control and Stochastic Differential Games

  • P.-L. Lions
  • P. E. Souganidis
Part of the The IMA Volumes in Mathematics and Its Applications book series (IMA, volume 10)

Abstract

In this note we review, explain and detail some recent results concerning the possible relations between various value functions of general optimal stochastic control and stochastic differential games and the viscosity solutions of the associated Hamilton-Jacobi-Bellman HJB and Bellman-Isaacs BI equations. It is well-known that the derivation of these equations is heuristic and it is justified only when the value functions are smooth enough (W.H. Fleming and R. Richel [15]). On the other hand, the equations are fully nonlinear, second-order, elliptic but possibly degenerate. Smooth solutions do not exist in general and nonsmooth solutions (like Lipschitz continuous solutions in the deterministic case) are highly nonunique. (For some simple examples we refer to P.-L. Lions [24]). As far as the first-order Hamilton-Jacobi equations are concerned, to overcome these typical difficulties and related ones like numerical approximations, asymptotic problems etc. M.G. Crandall and P.-L. Lions [8] introduced the notion of viscosity solutions and proved general uniqueness results. A systematic exploration of several equivalent formulations of this notion and an easy and readable account of the typical uniqueness results may be found in M.G. Crandall, L.C. Evans and P.-L. Lions [6]. It was also observed in P.-L. Lions [24] that the classical derivation of the Bellman equation for deterministic control problems can be easily adapted to yield the following general fact: Value functions of deterministic control problems are always viscosity solutions of the associated Hamilton-Jacobi-Bellman equations. The uniqueness of viscosity solutions and the above fact imply then a complete characterization of the value functions. This observation was then extended to differential games by E.N. Barron, L.C. Evans and R. Jensen [3], P.E. Souganidis [36] and L.C. Evans and P.E. Souganidis [14].

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Alexandrov, A.D., Investigations on the maximum principle, Izv. Vyss. Ucebn. Zared. Mathematica I, 5 (1958), 126–157; II, 3 (1959), 3–12; III, 5(1959), 16–32; IV, 3 (1960), 3–15; V, 5 (1960), 16–26; VI, 1 (1961), 3–20.Google Scholar
  2. [2]
    Alexandrov, A.D., Almost everywhere existence of the second differential of a convex function and some properties of convex functions, Ucen. Zap. Lenigrad Gos. Univ., 37 (1939), 3–35.Google Scholar
  3. [3]
    Barron, E.N., L.C. Evans and R. Jensen, Viscosity solutions of Isaacs’ equations and differential games with Lipschitz controls, J. of Diff. Eq., 53 (1984), 213–233.MathSciNetMATHCrossRefGoogle Scholar
  4. [4]
    Bony, J.M., Principe du maximum dans les espaces de Sobolev, C.R. Acad. Sci. Paris, 265 (1967), 333–336.MathSciNetMATHGoogle Scholar
  5. [5]
    Capuzzo-Dolcetta, I. and P.-L. Lions, Hamilton-Jacobi equations and state-constraints problems, to appear.Google Scholar
  6. [6]
    Crandall, M.G., L.C. Evans and P.-L. Lions, Some properties of viscosity-solutions of Hamilton-Jacobi equations, Trans. AMS, 282 (1984), 481–532.MathSciNetCrossRefGoogle Scholar
  7. [7]
    Crandall, M.G., H. Ishii and P.-L. Lions, Uniqueness of viscosity solutions revisited, to appear.Google Scholar
  8. [8]
    Crandall, M.G. and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 277 (1983), 1–42.MathSciNetMATHCrossRefGoogle Scholar
  9. [9]
    Crandall, M.G., and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions, Part I, J. Funct. Anal., 63 (1985), 379–396; Part II, J. Funct. Anal., 65 (1986), 368–405; Parts III and IV, J. Funct. Anal., to appear.MathSciNetCrossRefGoogle Scholar
  10. [10]
    Crandall, M.G. and P.E. Souganidis, Developments in the theory of nonlinear first-order partial differential equations, Proceedings of International Symposium on Differential Equations, Birmingham, Alabama (1983), Knowles and Lewis, eds., North Holland Math. Studies 92, North Holland, Amsterdam, 1984.Google Scholar
  11. [11]
    Evans, L.C., Classical solutions of fully nonlinear convex second-order elliptic equations, Comm. Pure Appl. Math., 25 (1982), 333–363.Google Scholar
  12. [12]
    Evans, L.C., Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators, Trans. AMS, 275 (1983), 245–255.CrossRefGoogle Scholar
  13. [13]
    Evans, L.C. and A. Friedman, Optimal stochastic switching and the Dirichlet problem for the Bellman equation, Trans. AMS, 253 (1979), 365–389.MathSciNetMATHCrossRefGoogle Scholar
  14. [14]
    Evans, L.C. and P.E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations. Ind. Univ. Math. J., 33 (1984), 773–797.MathSciNetMATHCrossRefGoogle Scholar
  15. [15]
    Fleming, W.H. and R. Richel, Deterministic and stochastic optimal control, Springer, Berlin, 1975.MATHGoogle Scholar
  16. [16]
    Fleming, W.H. and P.E. Souganidis, Value functions of two-player, zero-sum stochastic differential games, to appear.Google Scholar
  17. [17]
    Jensen, R., The maximum principle for viscosity solutions fo fully nonlinear second order partial differential equations, to appear.Google Scholar
  18. [18]
    Jensen, R. and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Sc. Num. Sup. Pisa, 11 (1984), 129–176.MathSciNetMATHGoogle Scholar
  19. [19]
    Jensen, R., P.-L. Lions and P.E. Souganidis, A uniqueness result for viscosity solutions of fully nonlinear second order partial differential equation, to appear.Google Scholar
  20. [20]
    Krylov, N.V., Control of a solution of a stochastic integral equation, Th. Prob. Appl., 17 (1972); 114–131.MATHCrossRefGoogle Scholar
  21. [21]
    Krylov, N.V., Controlled diffusion processes, Springer, Berlin, 1980.MATHGoogle Scholar
  22. [22]
    Lasry, J.M. and P.-L. Lions, A remark on regularization on Hilbert space, Israel J. Math, to appear.Google Scholar
  23. [23]
    Lenhart, S., Semilinear approximation technique for maximum type Hamilton-Jacobi equations over finite max-min index set, Nonlinear Anal. T.M.A., 8 (1984), 407–415.MathSciNetMATHGoogle Scholar
  24. [24]
    Lions, P.-L., Generalized solutions of Hamilton-Jacobi equations, Pitman, London, 1982.MATHGoogle Scholar
  25. [25]
    Lions, P.-L., On the Hamilton-Jacobi-Bellman equations. Acta Applicandae, 1 (1983), 17–41.MATHCrossRefGoogle Scholar
  26. [26]
    Lions, P.-L., Some recent results in the optimal control of diffusion processes, Stochastic Analysis, Proceedings of the Taniguchi International Symposium on Stochastic Analysis, Karata and Kyoko, (1982), Kunikuniya, Tokyo, 1984.Google Scholar
  27. [27]
    Lions, P.-L., A remark on the Bony maximum principle, Proc. AMS, 88 (1983), 503–508.MATHCrossRefGoogle Scholar
  28. [28]
    Lions, P.-L., Viscosity solutions of Hamilton-Jacobi equations and boundary conditions, Proceedings of the Conference held at L’Aquila, 1986.Google Scholar
  29. [29]
    Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 2, Comm. P.D.E., 8 (1983), 1229–1276.MATHGoogle Scholar
  30. [30]
    Lions, P.-L., Control of diffusion processes in ℝN, Comm. Pure Appl. Math., 34 (1981), 121–147.MATHCrossRefGoogle Scholar
  31. [31]
    Lions, P.-L., Resolution analytique des problemes de Bellman-Dirichlet. Acta Math., 146 (1981), 151–166.MathSciNetMATHCrossRefGoogle Scholar
  32. [32]
    Lions, P.-L., Fully nonlinear elliptic equations and applications, Nonlinear Analysis, Function Spaces and Applications, Teubner, Leipzig, 1982.Google Scholar
  33. [33]
    Lions, P.-L., Optimal control of diffusion processes and Hamilton- Jacobi-Bellman equations, Part 3, Nonlinear Partial Differential Equations and their Applications, College de France Seminar, Vol. V., Pitman, London, 1983.Google Scholar
  34. [34]
    Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations, SIAM J. of Control and Optimization, 23 (1985), 566–583.MathSciNetMATHCrossRefGoogle Scholar
  35. [35]
    Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations II, SIAM J. of Control and Optimization, 24 (1986), 1086–1089.MathSciNetMATHCrossRefGoogle Scholar
  36. [36]
    Souganidis, P.E., Approximation schemes for viscosity solutions of Hamilton-Jacobi equations with applications to differential games, J. of Nonlinear Analysis, T.M.A., 9 (1985), 217–257.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag New York Inc. 1988

Authors and Affiliations

  • P.-L. Lions
    • 1
  • P. E. Souganidis
    • 2
  1. 1.CeremadeUniversite Paris-DauphineParis Cedex 16France
  2. 2.Lefschetz Center for Dynamical Systems Division of Applied MathematicsBrown UniversityProvidenceUSA

Personalised recommendations