Applied Mathematics & Optimization

, Volume 71, Issue 1, pp 125–163 | Cite as

Dynamic Programming and Error Estimates for Stochastic Control Problems with Maximum Cost

  • Olivier Bokanowski
  • Athena Picarelli
  • Hasnaa Zidani


This work is concerned with stochastic optimal control for a running maximum cost. A direct approach based on dynamic programming techniques is studied leading to the characterization of the value function as the unique viscosity solution of a second order Hamilton–Jacobi–Bellman (HJB) equation with an oblique derivative boundary condition. A general numerical scheme is proposed and a convergence result is provided. Error estimates are obtained for the semi-Lagrangian scheme. These results can apply to the case of lookback options in finance. Moreover, optimal control problems with maximum cost arise in the characterization of the reachable sets for a system of controlled stochastic differential equations. Some numerical simulations on examples of reachable analysis are included to illustrate our approach.


Hamilton–Jacobi equations Oblique Neuman boundary condition Error estimate Viscosity notion Reachable sets under state constraints Lookback options Maximum cost 

Mathematics Subject Classification

49J20 49L25 65M15 35K55 



This work was partially supported by the EU under the 7th Framework Programme Marie Curie Initial Training Network “FP7-PEOPLE-2010-ITN”, SADCO project, GA number 264735-SADCO.


  1. 1.
    Altarovici, A., Bokanowski, O., Zidani, H.: A general Hamilton–Jacobi framework for non-linear state-constrained control problems. COCV, ESAIM (2013). doi: 10.1051/cocv/2012011
  2. 2.
    Aubin, J.-P., Da Prato, G.: Stochastic viability and invariance. Ann. Scuola Norm. Sup. Pisa Cl. Sci. 17(4), 595–613 (1990)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Aubin, J.-P., Da Prato, G.: The viability theorem for stochastic differential inclusions. App. Stoch. Anal. (1998). doi: 10.1080/07362999808809512
  4. 4.
    Barles, G., Daher, C., Romano, M.: Optimal control on the \(L^\infty \) norm of a diffusion process. Control Optim. SIAM J. (1994). doi: 10.1137/S0363012991223595
  5. 5.
    Barles, G., Daher, C., Romano, M.: Convergence of numerical schemes for parabolic equations arising in finance theory. Models Methods Appl. Sci. Math. (1995). doi: 10.1017/CBO9781139173056.002
  6. 6.
    Barles G., Jakobsen E.R. (2002) On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations. M2AN Math. Model. Numer. Anal. doi: 10.1051/m2an2:002002
  7. 7.
    Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for Hamilton-Jacobi-Bellman equations. Numer. Anal. SIAM J. (2005). doi: 10.1137/S003614290343815X
  8. 8.
    Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for parabolic Hamilton–Jacobi–Bellman equations. Comp. Math. (2007). doi: 10.1090/S0025-5718-07-02000-5
  9. 9.
    Barles G., Lions P.L. (1991) Fully nonlinear Neumann type boundary conditions for first-order Hamilton–Jacobi equations. Nonlinear Anal. Theory Methods Appl. doi: 10.1016/0362-546X(91)90165-W
  10. 10.
    Barles, G., Souganidis, P.E.: Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. (1991). doi: 10.3233/ASY-1991-4305
  11. 11.
    Barron, E.N.: The Bellman equation for control of the running max of a diffusion and applications to lookback options. Anal. Appl. (1993). doi: 10.1080/00036819308840158
  12. 12.
    Bokanowski, O., Forcadel, N., Zidani, H.: Reachability and minimal times for state constrained nonlinear problems without any controllability assumption. Control Optim. SIAM J. (2010). doi: 10.1137/090762075
  13. 13.
    Bonnans, J.F., Maroso, S., Zidani, H.: Error estimates for stochastic differential games: the adverse stopping case. IMA J. Numer. Anal. (2006). doi: 10.1093/imanum/dri034
  14. 14.
    Bonnans, J.F., Maroso, S., Zidani, H.: Error bounds for a stochastic impulse control problem. Math. Optim. Appl. (2007). doi: 10.1007/s00245-006-0865-2
  15. 15.
    Bonnans, J.F., Ottenwaelter, E., Zidani, H.: Numerical schemes for the two dimensional second-order HJB equation. ENSAIM: M2AN 723, 735–738 (2004)Google Scholar
  16. 16.
    Bonnans, J.F., Zidani, H.: Consistency of generalized finite difference schemes for the stochastic HJB equation. Numer. Anal. SIAM J. (2003). doi: 10.1137/S0036142901387336
  17. 17.
    Bouchard, B.: Stochastic targets with mixed diffusion processes and viscosity solutions. Stoch. Process. Appl. (2002). doi: 10.1016/S0304-4149(02)00129-1
  18. 18.
    Bouchard, B., Touzi, N.: Weak dynamic programming principle for viscosity solutions. Control Optim. SIAM J. (2011). doi: 10.1137/090752328
  19. 19.
    Camilli, F., Falcone, M.: An approximation scheme for the optimal control of diffusion processes. RAIRO Modél. Math. Anal. Numér. 29(1), 97–122 (1995)zbMATHMathSciNetGoogle Scholar
  20. 20.
    Capuzzo-Dolcetta, I.: On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming. Math. Optim. Appl. (1983). doi: 10.1007/BF01448394
  21. 21.
    Crandall, M.G., Ishii, H., Lions, P.L.: User’s guide to viscosity solutions of second order partial differential equations. Am. Math. Soc. Bull. (1992). doi: 10.1090/S0273-0979-1992-00266-5
  22. 22.
    Crandall, M.G., Lions, P.L.: Convergent difference schemes for nonlinear parabolic equations and mean curvature motion. Math. Numer. (1996). doi: 10.1007/s002110050228
  23. 23.
    Debrabant, K., Jakobsen, E.R.: Semi-Lagrangian schemes for linear and fully non-linear diffusion equations. Comp. Math. (2013). doi: 10.1090/S0025-5718-2012-02632-9
  24. 24.
    Dupuis P., Ishii H. (1990) On oblique derivative problems for fully nonlinear second-order elliptic partial differential equations on nonsmooth domains. Nonlinear Anal. Theory Methods Appl. doi: 10.1016/0362-546X(90)90048-L
  25. 25.
    Heinricher, A.C., Stockbridge, R.H.: Optimal control of the running max. Control Optim. SIAM J. (1991). doi: 10.1137/0329052
  26. 26.
    Ishii, H.: A boundary value problem of the Dirichlet type for Hamilton–Jacobi equations. Ann. Scuola Norm. Sup. Pisa Cl. Sci. 16(1), 105–135 (1989)zbMATHMathSciNetGoogle Scholar
  27. 27.
    Ishii, H.: Fully nonlinear oblique derivative problems for nonlinear Second-order elliptic PDEs. Duke Math. J. (1991). doi: 10.1215/S0012-7094-91-06228-9
  28. 28.
    Ishii, H., Lions, P.L.: Viscosity solutions of fully nonlinear Second-order elliptic partial differential equations. J. Differ. Equ. (1990). doi: 10.1016/0022-0396(90)90068-Z
  29. 29.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations. St. Petersb. Math. J. 9(3), 639–650 (1998)Google Scholar
  30. 30.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Theory Relat. Fields Probab. (2000). doi: 10.1007/s004400050264
  31. 31.
    Kushner, H.J.: Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Academic Press, New York (1977)zbMATHGoogle Scholar
  32. 32.
    Kushner, H.J., Dupuis, P.: Numerical methods for stochastic control problems in continuous time. Springer, New York (2001)CrossRefzbMATHGoogle Scholar
  33. 33.
    Lions, P.L.: Neumann type boundary conditions for Hamilton–Jacobi equations. Duke Math. J. (1985). doi: 10.1215/S0012-7094-85-05242-1
  34. 34.
    Menaldi, J.-L.: Some estimates for finite difference approximations. Control Optim. SIAM J. (1989). doi: 10.1137/0327031
  35. 35.
    Osher, S., Sethian, A.J.: Fronts propagating with curvature dependent speed: algorithms based on Hamilton–Jacobi formulations. J. Compt. Phys. (1988). doi: 10.1016/0021-9991(88)90002-2
  36. 36.
    Soner, H.M., Touzi, N.: Stochastic target problems, dynamic programming and viscosity solutions. Control Optim. SIAM J. (2002). doi: 10.1137/S0363012900378863
  37. 37.
    Wilmott, P.: Paul Wilmott on Quantitative Finance. Wiley, England (2000). (Revised Ed. of “Derivatives”)Google Scholar
  38. 38.
    Yong, J., Zhou, X.Y.: Stochastic controls: Hamiltonian systems and HJB equations. Stochastic Modeling and Applied Probability. Springer, New York (1999)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Olivier Bokanowski
    • 1
  • Athena Picarelli
    • 2
  • Hasnaa Zidani
    • 3
  1. 1.Laboratoire Jacques-Louis Lions, Université Paris-Diderot (Paris 7) UFR de Mathématiques - Bât. Sophie GermainParis Cedex 13France
  2. 2.Projet Commands, INRIA Saclay & ENSTA ParisTechPalaiseau CedexFrance
  3. 3.Unité de Mathématiques appliquées (UMA), ENSTA ParisTechPalaiseau CedexFrance

Personalised recommendations