Abstract
We study a semi-discretisation scheme for stochastic optimal control problems whose dynamics are given by controlled stochastic delay (or functional) differential equations with bounded memory. Performance is measured in terms of expected costs. By discretising time in two steps, we construct a sequence of approximating finite-dimensional Markovian optimal control problems in discrete time. The corresponding value functions converge to the value function of the original problem, and we derive an upper bound on the discretisation error or, equivalently, a worst-case estimate for the rate of convergence.
Similar content being viewed by others
References
Atkinson, K., Han, W.: Theoretical Numerical Analysis. Texts in Applied Mathematics, vol. 39. Springer, New York (2001)
Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for Hamilton-Jacobi-Bellman equations. SIAM J. Numer. Anal. 43(2), 540–558 (2005)
Barles, G., Jakobsen, E.R.: Error bounds for monotone approximation schemes for parabolic Hamilton-Jacobi-Bellman equations. Math. Comput. 76(260), 1861–1893 (2007)
Bauer, H., Rieder, U.: Stochastic control problems with delay. Math. Meth. Oper. Res. 62(3), 411–427 (2005)
Buckwar, E.: Introduction to the numerical analysis of stochastic delay differential equations. J. Comput. Appl. Math. 125(1–2), 297–307 (2000)
Calzolari, A., Florchinger, P., Nappo, G.: Convergence in nonlinear filtering for stochastic delay systems. SIAM J. Control Optim. (to appear)
Chang, M.-H., Pang, T., Pemy, M.: Optimal control of stochastic functional differential equations with bounded memory. In: Zhang, X., Liu, D., Wu, L. (eds.) Operations Research and Its Applications, Papers from the Sixth International Symposium, ISORA’06, Xinjiang, China, August 8–12, pp. 82–94. World Publishing Corporation, Beijing (2006)
Elsanosi, I., Øksendal, B., Sulem, A.: Some solvable stochastic control problems with delay. Stoch. Stoch. Rep. 71(1–2), 69–89 (2000)
Falcone, M., Ferretti, R.: Discrete time high-order schemes for viscosity solutions of Hamilton-Jacobi-Bellman equations. Numer. Math. 67(3), 315–344 (1994)
Falcone, M., Rosace, R.: Discrete-time approximation of optimal control problems for delayed equations. Control Cybern. 25(3), 665–675 (1996)
Fischer, M., Reiß, M.: Discretisation of stochastic control problems for continuous time dynamics with delay. J. Comput. Appl. Math. 205(2), 969–981 (2007)
Garsia, A.M., Rodemich, E., Rumsey, H. Jr.: A real variable lemma and the continuity of paths of some Gaussian processes. Indiana Math. J. 20(6), 565–578 (1970)
Gihman, I.I., Skorohod, A.V.: Controlled Stochastic Processes. Springer, New York (1979)
Hu, Y., Mohammed, S.-E.A., Yan, F.: Discrete-time approximations of stochastic delay equations: the Milstein scheme. Ann. Probab. 32(1A), 265–314 (2004)
Karatzas, I., Shreve, S.E.: Brownian Motion and Stochastic Calculus, 2nd edn. Graduate Texts in Mathematics, vol. 113. Springer, New York (1991)
Krylov, N.V.: Controlled Diffusion Processes. Applications of Mathematics, vol. 14. Springer, New York (1980)
Krylov, N.V.: Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies. Electron. J. Probab. 4, 1–19 (1999)
Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000)
Krylov, N.V.: Mean value theorems for stochastic integrals. Ann. Probab. 29(1), 385–410 (2001)
Kushner, H.J.: Numerical approximations for nonlinear stochastic systems with delays. Stochastics 77(3), 211–240 (2005)
Kushner, H.J.: Numerical approximations for stochastic systems with delays in the state and control. Technical Report, Lefschetz Center for Dynamical systems (2005)
Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd edn. Applications of Mathematics, vol. 24. Springer, New York (2001)
Larssen, B.: Dynamic programming in stochastic control of systems with delay. Stoch. Stoch. Rep. 74(3–4), 651–673 (2002)
Larssen, B., Risebro, N.H.: When are HJB-equations for control problems with stochastic delay equations finite dimensional? Stoch. Anal. Appl. 21(3), 643–671 (2003)
Mao, X.: Numerical solutions of stochastic functional differential equations. LMS J. Comput. Math. 6, 141–161 (2003)
Mohammed, S.-E.A.: Stochastic Functional Differential Equations. Pitman, London (1984)
Øksendal, B., Sulem, A.: A maximum principle for optimal control of stochastic systems with delay, with applications to finance. In: Menaldi, J., Rofman, E., Sulem, A. (eds.) Optimal Control and Partial Differential Equations. In Honour of Professor Alain Bensoussan’s 60th Birthday. Proceedings of the Conference, Paris, December 4, 2000, pp. 64–79. IOS Press, Amsterdam (2001)
Rogers, L.C.G.: Pathwise stochastic optimal control. SIAM J. Control Optim. 46(3), 1116–1132 (2007)
Słomiński, L.: Euler’s approximations of solutions of SDEs with reflecting boundary. Stoch. Process. Appl. 94, 317–337 (2001)
Stroock, D.W., Varadhan, S.R.S.: Multidimensional Diffusion Processes. Grundlehren der Mathematischen Wissenschaften, vol. 233. Springer, Berlin (1979)
Yong, J., Zhou, X.Y.: Stochastic Controls. Hamiltonian Systems and HJB Equations. Applications of Mathematics, vol. 43. Springer, New York (1999)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fischer, M., Nappo, G. Time Discretisation and Rate of Convergence for the Optimal Control of Continuous-Time Stochastic Systems with Delay. Appl Math Optim 57, 177–206 (2008). https://doi.org/10.1007/s00245-007-9019-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-007-9019-4