# Viscosity Solutions of Second-Order Equations, Stochastic Control and Stochastic Differential Games

## Abstract

In this note we review, explain and detail some recent results concerning the possible relations between various value functions of general optimal stochastic control and stochastic differential games and the viscosity solutions of the associated Hamilton-Jacobi-Bellman HJB and Bellman-Isaacs BI equations. It is well-known that the derivation of these equations is heuristic and it is justified only when the value functions are smooth enough (W.H. Fleming and R. Richel [15]). On the other hand, the equations are fully nonlinear, second-order, elliptic but possibly degenerate. Smooth solutions do not exist in general and nonsmooth solutions (like Lipschitz continuous solutions in the deterministic case) are highly nonunique. (For some simple examples we refer to P.-L. Lions [24]). As far as the first-order Hamilton-Jacobi equations are concerned, to overcome these typical difficulties and related ones like numerical approximations, asymptotic problems etc. M.G. Crandall and P.-L. Lions [8] introduced the notion of viscosity solutions and proved general uniqueness results. A systematic exploration of several equivalent formulations of this notion and an easy and readable account of the typical uniqueness results may be found in M.G. Crandall, L.C. Evans and P.-L. Lions [6]. It was also observed in P.-L. Lions [24] that the classical derivation of the Bellman equation for deterministic control problems can be easily adapted to yield the following general fact: Value functions of deterministic control problems are always viscosity solutions of the associated Hamilton-Jacobi-Bellman equations. The uniqueness of viscosity solutions and the above fact imply then a complete characterization of the value functions. This observation was then extended to differential games by E.N. Barron, L.C. Evans and R. Jensen [3], P.E. Souganidis [36] and L.C. Evans and P.E. Souganidis [14].

## Keywords

Viscosity Solution Differential Game Stochastic Control Optimal Stochastic Control Stochastic Control Problem## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Alexandrov, A.D., Investigations on the maximum principle, Izv. Vyss. Ucebn. Zared. Mathematica I, 5 (1958), 126–157; II, 3 (1959), 3–12; III, 5(1959), 16–32; IV, 3 (1960), 3–15; V, 5 (1960), 16–26; VI, 1 (1961), 3–20.Google Scholar
- [2]Alexandrov, A.D., Almost everywhere existence of the second differential of a convex function and some properties of convex functions, Ucen. Zap. Lenigrad Gos. Univ., 37 (1939), 3–35.Google Scholar
- [3]Barron, E.N., L.C. Evans and R. Jensen, Viscosity solutions of Isaacs’ equations and differential games with Lipschitz controls, J. of Diff. Eq., 53 (1984), 213–233.MathSciNetMATHCrossRefGoogle Scholar
- [4]Bony, J.M., Principe du maximum dans les espaces de Sobolev, C.R. Acad. Sci. Paris, 265 (1967), 333–336.MathSciNetMATHGoogle Scholar
- [5]Capuzzo-Dolcetta, I. and P.-L. Lions, Hamilton-Jacobi equations and state-constraints problems, to appear.Google Scholar
- [6]Crandall, M.G., L.C. Evans and P.-L. Lions, Some properties of viscosity-solutions of Hamilton-Jacobi equations, Trans. AMS, 282 (1984), 481–532.MathSciNetCrossRefGoogle Scholar
- [7]Crandall, M.G., H. Ishii and P.-L. Lions, Uniqueness of viscosity solutions revisited, to appear.Google Scholar
- [8]Crandall, M.G. and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 277 (1983), 1–42.MathSciNetMATHCrossRefGoogle Scholar
- [9]Crandall, M.G., and P.-L. Lions, Hamilton-Jacobi equations in infinite dimensions, Part I, J. Funct. Anal., 63 (1985), 379–396; Part II, J. Funct. Anal., 65 (1986), 368–405; Parts III and IV, J. Funct. Anal., to appear.MathSciNetCrossRefGoogle Scholar
- [10]Crandall, M.G. and P.E. Souganidis, Developments in the theory of nonlinear first-order partial differential equations, Proceedings of International Symposium on Differential Equations, Birmingham, Alabama (1983), Knowles and Lewis, eds., North Holland Math. Studies 92, North Holland, Amsterdam, 1984.Google Scholar
- [11]Evans, L.C., Classical solutions of fully nonlinear convex second-order elliptic equations, Comm. Pure Appl. Math., 25 (1982), 333–363.Google Scholar
- [12]Evans, L.C., Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators, Trans. AMS, 275 (1983), 245–255.CrossRefGoogle Scholar
- [13]Evans, L.C. and A. Friedman, Optimal stochastic switching and the Dirichlet problem for the Bellman equation, Trans. AMS, 253 (1979), 365–389.MathSciNetMATHCrossRefGoogle Scholar
- [14]Evans, L.C. and P.E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations. Ind. Univ. Math. J., 33 (1984), 773–797.MathSciNetMATHCrossRefGoogle Scholar
- [15]Fleming, W.H. and R. Richel,
*Deterministic and stochastic optimal control*, Springer, Berlin, 1975.MATHGoogle Scholar - [16]Fleming, W.H. and P.E. Souganidis, Value functions of two-player, zero-sum stochastic differential games, to appear.Google Scholar
- [17]Jensen, R., The maximum principle for viscosity solutions fo fully nonlinear second order partial differential equations, to appear.Google Scholar
- [18]Jensen, R. and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Sc. Num. Sup. Pisa, 11 (1984), 129–176.MathSciNetMATHGoogle Scholar
- [19]Jensen, R., P.-L. Lions and P.E. Souganidis, A uniqueness result for viscosity solutions of fully nonlinear second order partial differential equation, to appear.Google Scholar
- [20]Krylov, N.V., Control of a solution of a stochastic integral equation, Th. Prob. Appl., 17 (1972); 114–131.MATHCrossRefGoogle Scholar
- [21]Krylov, N.V.,
*Controlled diffusion processes*, Springer, Berlin, 1980.MATHGoogle Scholar - [22]Lasry, J.M. and P.-L. Lions, A remark on regularization on Hilbert space, Israel J. Math, to appear.Google Scholar
- [23]Lenhart, S., Semilinear approximation technique for maximum type Hamilton-Jacobi equations over finite max-min index set, Nonlinear Anal. T.M.A., 8 (1984), 407–415.MathSciNetMATHGoogle Scholar
- [24]Lions, P.-L.,
*Generalized solutions of Hamilton-Jacobi equations*, Pitman, London, 1982.MATHGoogle Scholar - [25]Lions, P.-L., On the Hamilton-Jacobi-Bellman equations. Acta Applicandae, 1 (1983), 17–41.MATHCrossRefGoogle Scholar
- [26]Lions, P.-L., Some recent results in the optimal control of diffusion processes, Stochastic Analysis, Proceedings of the Taniguchi International Symposium on Stochastic Analysis, Karata and Kyoko, (1982), Kunikuniya, Tokyo, 1984.Google Scholar
- [27]Lions, P.-L., A remark on the Bony maximum principle, Proc. AMS, 88 (1983), 503–508.MATHCrossRefGoogle Scholar
- [28]Lions, P.-L., Viscosity solutions of Hamilton-Jacobi equations and boundary conditions, Proceedings of the Conference held at L’Aquila, 1986.Google Scholar
- [29]Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 2, Comm. P.D.E., 8 (1983), 1229–1276.MATHGoogle Scholar
- [30]Lions, P.-L., Control of diffusion processes in ℝN, Comm. Pure Appl. Math., 34 (1981), 121–147.MATHCrossRefGoogle Scholar
- [31]Lions, P.-L., Resolution analytique des problemes de Bellman-Dirichlet. Acta Math., 146 (1981), 151–166.MathSciNetMATHCrossRefGoogle Scholar
- [32]Lions, P.-L., Fully nonlinear elliptic equations and applications, Nonlinear Analysis, Function Spaces and Applications, Teubner, Leipzig, 1982.Google Scholar
- [33]Lions, P.-L., Optimal control of diffusion processes and Hamilton- Jacobi-Bellman equations, Part 3,
*Nonlinear Partial Differential Equations and their Applications*, College de France Seminar, Vol. V., Pitman, London, 1983.Google Scholar - [34]Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations, SIAM J. of Control and Optimization, 23 (1985), 566–583.MathSciNetMATHCrossRefGoogle Scholar
- [35]Lions, P.-L. and P.E. Souganidis, Differential games, optimal control and directional derivatives of viscosity solutions of Bellman’s and Isaacs’ equations II, SIAM J. of Control and Optimization, 24 (1986), 1086–1089.MathSciNetMATHCrossRefGoogle Scholar
- [36]Souganidis, P.E., Approximation schemes for viscosity solutions of Hamilton-Jacobi equations with applications to differential games, J. of Nonlinear Analysis, T.M.A., 9 (1985), 217–257.MathSciNetCrossRefGoogle Scholar