Viscosity Solutions of Second-Order Equations, Stochastic Control and Stochastic Differential Games

• P.-L. Lions
• P. E. Souganidis
Conference paper
Part of the The IMA Volumes in Mathematics and Its Applications book series (IMA, volume 10)

Abstract

In this note we review, explain and detail some recent results concerning the possible relations between various value functions of general optimal stochastic control and stochastic differential games and the viscosity solutions of the associated Hamilton-Jacobi-Bellman HJB and Bellman-Isaacs BI equations. It is well-known that the derivation of these equations is heuristic and it is justified only when the value functions are smooth enough (W.H. Fleming and R. Richel [15]). On the other hand, the equations are fully nonlinear, second-order, elliptic but possibly degenerate. Smooth solutions do not exist in general and nonsmooth solutions (like Lipschitz continuous solutions in the deterministic case) are highly nonunique. (For some simple examples we refer to P.-L. Lions [24]). As far as the first-order Hamilton-Jacobi equations are concerned, to overcome these typical difficulties and related ones like numerical approximations, asymptotic problems etc. M.G. Crandall and P.-L. Lions [8] introduced the notion of viscosity solutions and proved general uniqueness results. A systematic exploration of several equivalent formulations of this notion and an easy and readable account of the typical uniqueness results may be found in M.G. Crandall, L.C. Evans and P.-L. Lions [6]. It was also observed in P.-L. Lions [24] that the classical derivation of the Bellman equation for deterministic control problems can be easily adapted to yield the following general fact: Value functions of deterministic control problems are always viscosity solutions of the associated Hamilton-Jacobi-Bellman equations. The uniqueness of viscosity solutions and the above fact imply then a complete characterization of the value functions. This observation was then extended to differential games by E.N. Barron, L.C. Evans and R. Jensen [3], P.E. Souganidis [36] and L.C. Evans and P.E. Souganidis [14].

Preview

Unable to display preview. Download preview PDF.

References

1. [1]
Alexandrov, A.D., Investigations on the maximum principle, Izv. Vyss. Ucebn. Zared. Mathematica I, 5 (1958), 126–157; II, 3 (1959), 3–12; III, 5(1959), 16–32; IV, 3 (1960), 3–15; V, 5 (1960), 16–26; VI, 1 (1961), 3–20.Google Scholar
2. [2]
Alexandrov, A.D., Almost everywhere existence of the second differential of a convex function and some properties of convex functions, Ucen. Zap. Lenigrad Gos. Univ., 37 (1939), 3–35.Google Scholar
3. [3]
Barron, E.N., L.C. Evans and R. Jensen, Viscosity solutions of Isaacs’ equations and differential games with Lipschitz controls, J. of Diff. Eq., 53 (1984), 213–233.
4. [4]
Bony, J.M., Principe du maximum dans les espaces de Sobolev, C.R. Acad. Sci. Paris, 265 (1967), 333–336.
5. [5]
Capuzzo-Dolcetta, I. and P.-L. Lions, Hamilton-Jacobi equations and state-constraints problems, to appear.Google Scholar
6. [6]
Crandall, M.G., L.C. Evans and P.-L. Lions, Some properties of viscosity-solutions of Hamilton-Jacobi equations, Trans. AMS, 282 (1984), 481–532.
7. [7]
Crandall, M.G., H. Ishii and P.-L. Lions, Uniqueness of viscosity solutions revisited, to appear.Google Scholar
8. [8]
Crandall, M.G. and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 277 (1983), 1–42.
9. [9]
Crandall, M.G., and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions, Part I, J. Funct. Anal., 63 (1985), 379–396; Part II, J. Funct. Anal., 65 (1986), 368–405; Parts III and IV, J. Funct. Anal., to appear.
10. [10]
Crandall, M.G. and P.E. Souganidis, Developments in the theory of nonlinear first-order partial differential equations, Proceedings of International Symposium on Differential Equations, Birmingham, Alabama (1983), Knowles and Lewis, eds., North Holland Math. Studies 92, North Holland, Amsterdam, 1984.Google Scholar
11. [11]
Evans, L.C., Classical solutions of fully nonlinear convex second-order elliptic equations, Comm. Pure Appl. Math., 25 (1982), 333–363.Google Scholar
12. [12]
Evans, L.C., Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators, Trans. AMS, 275 (1983), 245–255.
13. [13]
Evans, L.C. and A. Friedman, Optimal stochastic switching and the Dirichlet problem for the Bellman equation, Trans. AMS, 253 (1979), 365–389.
14. [14]
Evans, L.C. and P.E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations. Ind. Univ. Math. J., 33 (1984), 773–797.
15. [15]
Fleming, W.H. and R. Richel, Deterministic and stochastic optimal control, Springer, Berlin, 1975.
16. [16]
Fleming, W.H. and P.E. Souganidis, Value functions of two-player, zero-sum stochastic differential games, to appear.Google Scholar
17. [17]
Jensen, R., The maximum principle for viscosity solutions fo fully nonlinear second order partial differential equations, to appear.Google Scholar
18. [18]
Jensen, R. and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Sc. Num. Sup. Pisa, 11 (1984), 129–176.
19. [19]
Jensen, R., P.-L. Lions and P.E. Souganidis, A uniqueness result for viscosity solutions of fully nonlinear second order partial differential equation, to appear.Google Scholar
20. [20]
Krylov, N.V., Control of a solution of a stochastic integral equation, Th. Prob. Appl., 17 (1972); 114–131.
21. [21]
Krylov, N.V., Controlled diffusion processes, Springer, Berlin, 1980.
22. [22]
Lasry, J.M. and P.-L. Lions, A remark on regularization on Hilbert space, Israel J. Math, to appear.Google Scholar
23. [23]
Lenhart, S., Semilinear approximation technique for maximum type Hamilton-Jacobi equations over finite max-min index set, Nonlinear Anal. T.M.A., 8 (1984), 407–415.
24. [24]
Lions, P.-L., Generalized solutions of Hamilton-Jacobi equations, Pitman, London, 1982.
25. [25]
Lions, P.-L., On the Hamilton-Jacobi-Bellman equations. Acta Applicandae, 1 (1983), 17–41.
26. [26]
Lions, P.-L., Some recent results in the optimal control of diffusion processes, Stochastic Analysis, Proceedings of the Taniguchi International Symposium on Stochastic Analysis, Karata and Kyoko, (1982), Kunikuniya, Tokyo, 1984.Google Scholar
27. [27]
Lions, P.-L., A remark on the Bony maximum principle, Proc. AMS, 88 (1983), 503–508.
28. [28]
Lions, P.-L., Viscosity solutions of Hamilton-Jacobi equations and boundary conditions, Proceedings of the Conference held at L’Aquila, 1986.Google Scholar
29. [29]
Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 2, Comm. P.D.E., 8 (1983), 1229–1276.
30. [30]
Lions, P.-L., Control of diffusion processes in ℝN, Comm. Pure Appl. Math., 34 (1981), 121–147.
31. [31]
Lions, P.-L., Resolution analytique des problemes de Bellman-Dirichlet. Acta Math., 146 (1981), 151–166.
32. [32]
Lions, P.-L., Fully nonlinear elliptic equations and applications, Nonlinear Analysis, Function Spaces and Applications, Teubner, Leipzig, 1982.Google Scholar
33. [33]
Lions, P.-L., Optimal control of diffusion processes and Hamilton- Jacobi-Bellman equations, Part 3, Nonlinear Partial Differential Equations and their Applications, College de France Seminar, Vol. V., Pitman, London, 1983.Google Scholar
34. [34]
Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations, SIAM J. of Control and Optimization, 23 (1985), 566–583.
35. [35]
Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations II, SIAM J. of Control and Optimization, 24 (1986), 1086–1089.
36. [36]
Souganidis, P.E., Approximation schemes for viscosity solutions of Hamilton-Jacobi equations with applications to differential games, J. of Nonlinear Analysis, T.M.A., 9 (1985), 217–257.