Skip to main content

On Finite-Difference Approximations for Normalized Bellman Equations

Abstract

A class of stochastic optimal control problems involving optimal stopping is considered. Methods of Krylov (Appl. Math. Optim. 52(3):365–399, 2005) are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of convergence of finite difference approximations for the optimal reward functions.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations. M2AN Math. Model. Numer. Anal. 36(1), 33–54 (2002)

    MATH  Article  MathSciNet  Google Scholar 

  2. 2.

    Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for Hamilton-Jacobi-Bellman equations. SIAM J. Numer. Anal. 43(2), 540–558 (2005) (electronic)

    MATH  Article  MathSciNet  Google Scholar 

  3. 3.

    Biswas, I.H., Jakobsen, E.R., Karlsen, K.H.: Error estimates for finite difference-quadrature schemes for a class of nonlocal Bellman equations with variable diffusion. http://www.math.uio.no/eprint/pure_math/2006/pure_2006.html (2006)

  4. 4.

    Dong, H., Krylov, N.: The rate of convergence of finite-difference approximations for parabolic Bellman equations with Lipschitz coefficients in cylindrical domains. Appl. Math. Optim. 56(1), 37–66 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  5. 5.

    Gyöngy, I., Krylov, N.: On the rate of convergence of splitting-up approximations for SPDEs. In: Progress in Probability, vol. 56, pp. 301–321. Birkhäuser, Basel (2003)

    Google Scholar 

  6. 6.

    Gyöngy, I., Šiška, D.: On randomized stopping. Bernoulli 14(2), 352–361 (2008)

    MATH  Article  Google Scholar 

  7. 7.

    Jakobsen, E.R.: On the rate of convergence of approximation schemes for Bellman equations associated with optimal stopping time problems. Math. Models Methods Appl. Sci. 13(5), 613–644 (2003)

    MATH  Article  MathSciNet  Google Scholar 

  8. 8.

    Jakobsen, E.R., Karlsen, K.H.: Convergence rates for semi-discrete splitting approximations for degenerate parabolic equations with source terms. BIT 45(1), 37–67 (2005)

    MATH  Article  MathSciNet  Google Scholar 

  9. 9.

    Jakobsen, E.R., Karlsen, K.H., La Chioma, C.: Error estimates for approximate solutions to Bellman equations associated with controlled jump-diffusions. http://www.math.uio.no/eprint/pure_math/2005/pure_2005.html (2005)

  10. 10.

    Krylov, N.V.: Controlled Diffusion Processes. Applications of Mathematics, vol. 14. Springer, New York (1980). Translated from the Russian by A.B. Aries

    MATH  Google Scholar 

  11. 11.

    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations. Algebra Anal. 9(3), 245–256 (1997)

    Google Scholar 

  12. 12.

    Krylov, N.V.: Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies. Electron. J. Probab. 4(2), 1–19 (1999)

    MathSciNet  Google Scholar 

  13. 13.

    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000)

    MATH  Article  Google Scholar 

  14. 14.

    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman equations with Lipschitz coefficients. arXiv:math, 1(1), pp. 1–33 (2004)

  15. 15.

    Krylov, N.V.: The rate of convergence of finite-difference approximations for Bellman equations with Lipschitz coefficients. Appl. Math. Optim. 52(3), 365–399 (2005)

    MATH  Article  MathSciNet  Google Scholar 

  16. 16.

    Krylov, N.V.: On factorizations of smooth nonnegative matrix-values functions and on smooth functions with values in polyhedra. Appl. Math. Optim. 58(3), 373–392 (2008)

    MATH  Article  MathSciNet  Google Scholar 

  17. 17.

    Krylov, N.V.: A priori estimates of smoothness of solutions to difference Bellman equations with linear and quasi-linear operators. Math. Comp. 76(258), 669–698 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  18. 18.

    Kushner, H.J., Dupuis, P.: Stochastic modelling and applied probability. In: Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics, vol. 24, 2nd edn. Springer, New York (2001).

    Google Scholar 

  19. 19.

    Menaldi, J.-L.: Some estimates for finite difference approximations. SIAM J. Control Optim. 27(3), 579–607 (1989)

    MATH  Article  MathSciNet  Google Scholar 

  20. 20.

    Shiryaev, A.N.: Statisticheskii posledovatelnyi analiz. Optimalnye pravila ostanovki, 2nd revised edn. Nauka, Moscow (1976).

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to David Šiška.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gyöngy, I., Šiška, D. On Finite-Difference Approximations for Normalized Bellman Equations. Appl Math Optim 60, 297 (2009). https://doi.org/10.1007/s00245-009-9082-0

Download citation

Keywords

  • Finite-difference approximations
  • Normalized Bellman equations
  • Fully nonlinear equations
  • Optimal stopping and control