Skip to main content
Log in

A semi-Lagrangian scheme for Hamilton–Jacobi–Bellman equations with oblique derivatives boundary conditions

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

We investigate in this work a fully-discrete semi-Lagrangian approximation of second order possibly degenerate Hamilton–Jacobi–Bellman (HJB) equations on a bounded domain \({\mathcal {O}}\subset {\mathbb {R}}^N\) (\(N=1,2,3\)) with oblique derivatives boundary conditions. These equations appear naturally in the study of optimal control of diffusion processes with oblique reflection at the boundary of the domain. The proposed scheme is shown to satisfy a consistency type property, it is monotone and stable. Our main result is the convergence of the numerical solution towards the unique viscosity solution of the HJB equation. The convergence result holds under the same asymptotic relation between the time and space discretization steps as in the classical setting for semi-Lagrangian schemes on \({\mathcal {O}}={\mathbb {R}}^{N}\). We present some numerical results, in dimensions \(N=1, \,2\), on unstructured meshes, that confirm the numerical convergence of the scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. By \(\varDelta t\sim (\varDelta x)^q\), we mean here that \(\varDelta t\) is of the order of \((\varDelta x)^q\), i.e. there exists \(c>0\) such that \(\varDelta t=c(\varDelta x)^q\).

References

  1. Abgrall, R.: Numerical discretization of boundary conditions for first order Hamilton–Jacobi equations. SIAM J. Numer. Anal. 41(6), 2233–2261 (2003). https://doi.org/10.1137/S0036142998345980

    Article  MathSciNet  MATH  Google Scholar 

  2. Achdou, Y., Falcone, M.: A semi-Lagrangian scheme for mean curvature motion with nonlinear Neumann conditions. Interfaces Free Bound. 14(4), 455–485 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bardi, M., Dolcetta, I.C.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Birkauser (1996)

  4. Barles, G.: Fully nonlinear Neumann type boundary conditions for second-order elliptic and parabolic equations. J. Differ. Equ. 106(1), 90–106 (1993). https://doi.org/10.1006/jdeq.1993.1100

    Article  MathSciNet  MATH  Google Scholar 

  5. Barles, G.: Nonlinear Neumann boundary conditions for quasilinear degenerate elliptic equations and applications. J. Differ. Equ. 154(1), 191–224 (1999). https://doi.org/10.1006/jdeq.1998.3568

    Article  MathSciNet  MATH  Google Scholar 

  6. Barles, G., Lions, P.L.: Fully nonlinear Neumann type boundary conditions for first-order Hamilton–Jacobi equations. Nonlinear Anal. 16(2), 143–153 (1991). https://doi.org/10.1016/0362-546X(91)90165-W

    Article  MathSciNet  MATH  Google Scholar 

  7. Barles, G., Rouy, E.: A strong comparison result for the Bellman equation arising in stochastic exit time control problems and its applications. Commun. Partial Differ. Equ. 23(11–12), 1995–2033 (1998). https://doi.org/10.1080/03605309808821409

    Article  MathSciNet  MATH  Google Scholar 

  8. Barles, G., Souganidis, P.E.: Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4(3), 271–283 (1991)

    MathSciNet  MATH  Google Scholar 

  9. Barrett, J.W., Elliott, C.M.: Finite element approximation of the Dirichlet problem using the boundary penalty method. Numer. Math. 49(4), 343–366 (1986). https://doi.org/10.1007/BF01389536

    Article  MathSciNet  MATH  Google Scholar 

  10. Bonaventura, L., Calzola, E., Carlini, E., Ferretti, R.: Second order fully semi-Lagrangian discretizations of advection–diffusion–reaction systems. J. Sci. Comput. 88(1), 1–29 (2021). https://doi.org/10.1007/s10915-021-01518-8

    Article  MathSciNet  MATH  Google Scholar 

  11. Bonaventura, L., Ferretti, R., Rocchi, L.: A fully semi-Lagrangian discretization for the 2D incompressible Navier–Stokes equations in the vorticity-streamfunction formulation. Appl. Math. Comput. 323, 132–144 (2018). https://doi.org/10.1016/j.amc.2017.11.030

    Article  MathSciNet  MATH  Google Scholar 

  12. Bouchard, B.: Optimal reflection of diffusions and barrier options pricing under constraints. SIAM J. Control Optim. 47(4), 1785–1813 (2008). https://doi.org/10.1137/070697161

    Article  MathSciNet  MATH  Google Scholar 

  13. Bourgoing, M.: Viscosity solutions of fully nonlinear second order parabolic equations with \(L^1\) dependence in time and Neumann boundary conditions. Discrete Contin. Dyn. Syst. 21(3), 763–800 (2008). https://doi.org/10.3934/dcds.2008.21.763

    Article  MathSciNet  MATH  Google Scholar 

  14. Camilli, F., Falcone, M.: An approximation scheme for the optimal control of diffusion processes. RAIRO Modél. Math. Anal. Numér. 29(1), 97–122 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  15. Carlini, E., Falcone, M., Ferretti, R.: Convergence of a large time-step scheme for mean curvature motion. Interfaces Free Bound. 12(4), 409–441 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  16. Ciarlet, P.G., Lions, J.L. (eds.): Handbook of Numerical Analysis. Finite Element Methods. Part 1, vol. II. North-Holland, Amsterdam (1991)

    Google Scholar 

  17. Crandall, M.G., Ishii, H., Lions, P.L.: User’s guide to viscosity solutions of second order partial differential equations. Bull. Am. Math. Soc. (N.S.) 27(1), 1–67 (1992). https://doi.org/10.1090/S0273-0979-1992-00266-5

    Article  MathSciNet  MATH  Google Scholar 

  18. Day, M.V.: Neumann-type boundary conditions for Hamilton–Jacobi equations in smooth domains. Appl. Math. Optim. 53(3), 359–381 (2006). https://doi.org/10.1007/s00245-005-0852-z

    Article  MathSciNet  MATH  Google Scholar 

  19. Debrabant, K., Jakobsen, E.R.: Semi-Lagrangian schemes for linear and fully non-linear diffusion equations. Math. Comput. 82(283), 1433–1462 (2013). https://doi.org/10.1090/S0025-5718-2012-02632-9

    Article  MathSciNet  MATH  Google Scholar 

  20. Deckelnick, K., Hinze, M.: Convergence of a finite element approximation to a state-constrained elliptic control problem. SIAM J. Numer. Anal. 45(5), 1937–1953 (2007). https://doi.org/10.1137/060652361

    Article  MathSciNet  MATH  Google Scholar 

  21. Dupuis, P., Ishii, H.: SDEs with oblique reflection on nonsmooth domains. Ann. Probab. 21(1), 554–580 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  22. Dupuis, P., Ramanan, K.: Convex duality and the Skorokhod problem. I, II. Probab. Theory Relat. Fields 115(2), 153–195 (1999). https://doi.org/10.1007/s004400050269

    Article  MathSciNet  MATH  Google Scholar 

  23. Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton–Jacobi Equations. MOS-SIAM Series on Optimization (2013)

  24. Feng, X., Glowinski, R., Neilan, M.: Recent developments in numerical methods for fully nonlinear second order partial differential equations. SIAM Rev. 55(2), 205–267 (2013). https://doi.org/10.1137/110825960

    Article  MathSciNet  MATH  Google Scholar 

  25. Feng, X., Jensen, M.: Convergent semi-Lagrangian methods for the Monge–Ampère equation on unstructured grids. SIAM J. Numer. Anal. 55(2), 691–712 (2017). https://doi.org/10.1137/16M1061709

    Article  MathSciNet  MATH  Google Scholar 

  26. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, Stochastic Modelling and Applied Probability, vol. 25, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  27. Gilbarg, D., Trudinger, N.S.: Elliptic partial differential equations of second order. In: Classics in Mathematics. Springer-Verlag, Berlin (2001). Reprint of the 1998 edition

  28. Gobet, E.: Efficient schemes for the weak approximation of reflected diffusions. Monte Carlo Methods Appl. 7(1–2), 193–202 (2001). https://doi.org/10.1515/mcma.2001.7.1-2.193. (Monte Carlo and probabilistic methods for partial differential equations (Monte Carlo, 2000))

    Article  MathSciNet  MATH  Google Scholar 

  29. Hinderer, K., Rieder, U., Stieglitz, M.: Dynamic Optimization. Universitext, Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48814-1 . (Deterministic and stochastic models)

    Book  MATH  Google Scholar 

  30. Ishii, H., Sato, M.H.: Nonlinear oblique derivative problems for singular degenerate parabolic equations on a general domain. Nonlinear Anal. 57(7–8), 1077–1098 (2004). https://doi.org/10.1016/j.na.2004.04.003

    Article  MathSciNet  MATH  Google Scholar 

  31. Jaroszkowski, B., Jensen, M.: Finite element approximation of Hamilton–Jacobi–Bellman equations with nonlinear mixed boundary conditions (2021). Preprint arXiv:2105.09585

  32. Jensen, M., Smears, I.: On the notion of boundary conditions in comparison principles for viscosity solutions. In: Hamilton–Jacobi–Bellman Equations, Radon Series on Computational and Applied Mathematics, vol. 21, pp. 143–154. De Gruyter, Berlin (2018)

  33. Kruk, L.: Optimal policies for \(n\)-dimensional singular stochastic control problems. I. The Skorokhod problem. SIAM J. Control Optim. 38(5), 1603–1622 (2000). https://doi.org/10.1137/S0363012998347535

    Article  MathSciNet  MATH  Google Scholar 

  34. Kushner, H.J., Dupuis, P.: Numerical methods for stochastic control problems in continuous time. In: Applications of Mathematics (New York), vol. 24, 2nd edn. Springer-Verlag, New York (2001). https://doi.org/10.1007/978-1-4613-0007-6. (Stochastic Modelling and Applied Probability)

  35. Lions, P.L.: Generalized Solutions of Hamilton–Jacobi Equations, Research Notes in Mathematics, vol. 69. Pitman (Advanced Publishing Program), Boston (1982)

    Google Scholar 

  36. Lions, P.L.: Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. I. The dynamic programming principle and applications. Commun. Partial Differ. Equ. 8(10), 1101–1174 (1983). https://doi.org/10.1080/03605308308820297

    Article  MathSciNet  MATH  Google Scholar 

  37. Lions, P.L.: Neumann type boundary conditions for Hamilton–Jacobi equations. Duke Math. J. 52(4), 793–820 (1985). https://doi.org/10.1215/S0012-7094-85-05242-1

    Article  MathSciNet  MATH  Google Scholar 

  38. Lions, P.L., Sznitman, A.S.: Stochastic differential equations with reflecting boundary conditions. Commun. Pure Appl. Math. 37(4), 511–537 (1984). https://doi.org/10.1002/cpa.3160370408

    Article  MathSciNet  MATH  Google Scholar 

  39. Menaldi, J.L., Robin, M.: On some cheap control problems for diffusion processes. Trans. Am. Math. Soc. 278(2), 771–802 (1983). https://doi.org/10.2307/1999183

    Article  MathSciNet  Google Scholar 

  40. Milstein, G., Tretyakov, M.: Numerical solution of the Dirichlet problem for nonlinear parabolic equations by a probabilistic approach. IMA J. Numer. Anal. 21, 887–917 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  41. Milstein, G.N.: Application of the numerical integration of stochastic equations for the solution of boundary value problems with Neumann boundary conditions. Teor. Veroyatnost. i Primenen. 41(1), 210–218 (1996)

    MathSciNet  Google Scholar 

  42. Milstein, G.N., Tretyakov, M.V.: A probabilistic approach to the solution of the Neumann problem for nonlinear parabolic equations. IMA J. Numer. Anal. 22(4), 599–622 (2002). https://doi.org/10.1093/imanum/22.4.599

    Article  MathSciNet  MATH  Google Scholar 

  43. Neilan, M., Salgado, A.J., Zhang, W.: Numerical analysis of strongly nonlinear PDEs. Acta Numer. 26, 137–303 (2017). https://doi.org/10.1017/S0962492917000071

    Article  MathSciNet  MATH  Google Scholar 

  44. Rouy, E.: Numerical approximation of viscosity solutions of first-order Hamilton–Jacobi equations with Neumann type boundary conditions. Math. Models Methods Appl. Sci. 2(3), 357–374 (1992). https://doi.org/10.1142/S0218202592000223

    Article  MathSciNet  MATH  Google Scholar 

  45. Skorokhod, A.V.: Stochastic equations for diffusion processes in a bounded region. Theor. Prob. Appl. 6(3), 264–274 (1961). https://doi.org/10.1137/1106035

    Article  MathSciNet  Google Scholar 

  46. Soner, H.M., Shreve, S.E.: Regularity of the value function for a two-dimensional singular stochastic control problem. SIAM J. Control Optim. 27(4), 876–907 (1989). https://doi.org/10.1137/0327047

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The first two authors would like to thank the Italian Ministry of Instruction, University and Research (MIUR) for supporting this research with funds coming from the PRIN Project 2017 (2017KKJP4X entitled “Innovative numerical methods for evolutionary partial differential equations and applications”). Xavier Dupuis thanks the support by the EIPHI Graduate School (Contract ANR-17-EURE-0002). Elisa Calzola, Elisabetta Carlini and Francisco J. Silva were partially supported by KAUST through the subaward agreement OSR-2017-CRG6-3452.04.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Elisabetta Carlini.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Calzola, E., Carlini, E., Dupuis, X. et al. A semi-Lagrangian scheme for Hamilton–Jacobi–Bellman equations with oblique derivatives boundary conditions. Numer. Math. 153, 49–84 (2023). https://doi.org/10.1007/s00211-022-01336-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-022-01336-6

Mathematics Subject Classification

Navigation