Skip to main content
SpringerLink
Log in

Error Analysis of the Feedback Controls Arising in the Stochastic Linear Quadratic Control Problems

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

In this work, the author proposes a discretization for stochastic linear quadratic control problems (SLQ problems) subject to stochastic differential equations. The author firstly makes temporal discretization and obtains SLQ problems governed by stochastic difference equations. Then the author derives the convergence rates for this discretization relying on stochastic differential/difference Riccati equations. Finally an algorithm is presented. Compared with the existing results relying on stochastic Pontryagin-type maximum principle, the proposed scheme avoids solving backward stochastic differential equations and/or conditional expectations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bensoussan A, Lectures on stochastic control, Nonlinear Filtering and Stochastic Control (Cortona, 1981), Springer, Berlin-New York, 1982, 1–62.

    Google Scholar 

  2. Haussmann U G, Some examples of optimal stochastic controls or: The stochastic maximum principle at work, SIAM Rev., 1981, 23: 292–307.

    Article  MathSciNet  MATH  Google Scholar 

  3. Yong J and Zhou X Y, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.

    Book  MATH  Google Scholar 

  4. Bismut J M, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 1976, 14: 419–444.

    Article  MathSciNet  MATH  Google Scholar 

  5. Sun J and Yong J, Stochastic Linear-Quadratic Optimal Control Theory: Open-Loop and Closed-Loop Solutions, Springer, Cham, 2019.

    MATH  Google Scholar 

  6. Wonham W M, On a matrix Riccati equation of stochastic control, SIAM J. Control, 1968, 6: 681–697.

    Article  MathSciNet  MATH  Google Scholar 

  7. Beck C E W and Jentzen A, Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations, J. Nonlinear Sci., 2019, 29: 1563–1619.

    Article  MathSciNet  MATH  Google Scholar 

  8. Feng X, Glowinski R, and Neilan M, Recent developments in numerical methods for fully nonlinear second order partial differential equations, SIAM Rev., 2013, 55: 205–267.

    Article  MathSciNet  MATH  Google Scholar 

  9. Archibald R, Bao F, and Yong J, A stochastic gradient descent approach for stochastic optimal control, East Asian J. Appl. Math., 2020, 10: 635–658.

    Article  MathSciNet  MATH  Google Scholar 

  10. Archibald R, Bao F, Yong J, et al., An efficient numerical algorithm for solving data driven feedback control problems, J. Sci. Comput., 2020, 85: Paper No. 51.

  11. Dunst T and Prohl A, The forward-backward stochastic heat equation: Numerical analysis and simulation, SIAM J. Sci. Comput., 2016, 38: A2725–A2755.

    Article  MathSciNet  MATH  Google Scholar 

  12. Lü Q, Wang P, Wang Y, et al., Chapter 6 - Numerics for stochastic distributed parameter control systems: A finite transposition method, Numerical Control: Part A, (Eds. by Trélat E and Zuazua E), Elsevier, 2022, 23: 201–232.

  13. Prohl A and Wang Y, Strong error estimates for a space-time discretization of the linear-quadratic control problem with the stochastic heat equation with linear noise, IMA J. Numer. Anal., accepted, https://doi.org/10.1093/imanum/drab069.

  14. Prohl A and Wang Y, Strong rates of convergence for a space-time discretization of the backward stochastic heat equation, and of a linear-quadratic control problem for the stochastic heat equation, ESAIM Control Optim. Calc. Var., 2021, 27: Paper No. 54.

  15. Du N, Shi J, and Liu W, An effective gradient projection method for stochastic optimal control, Int. J. Numer. Anal. Model., 2013, 10: 757–774.

    MathSciNet  MATH  Google Scholar 

  16. Gong B, Liu W, Tang T, et al., An efficient gradient projection method for stochastic optimal control problems, SIAM J. Numer. Anal., 2017, 55: 2982–3005.

    Article  MathSciNet  MATH  Google Scholar 

  17. Damm T, Mena H, and Stillfjord T, Numerical solution of the finite horizon stochastic linear quadratic control problem, Numer. Linear Algebra Appl., 2017, 24: Paper No. 2091.

  18. Levajković T, Mena H, and Tuffaha A, The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach, Evol. Equ. Control Theory, 2016, 5: 105–134.

    Article  MathSciNet  MATH  Google Scholar 

  19. Stillfjord T, Adaptive high-order splitting schemes for large-scale differential Riccati equations, Numer. Algorithms, 2018, 78: 1129–1151.

    Article  MathSciNet  MATH  Google Scholar 

  20. Wang Y, Yang D, Yong J, et al., Exact controllability of linear stochastic differential equations and related problems, Math. Control Relat. Fields, 2017, 7: 305–345.

    Article  MathSciNet  MATH  Google Scholar 

  21. Wang Y and Zhang C, The norm optimal control problem for stochastic linear control systems, ESAIM Control Optim. Calc. Var., 2015, 21: 399–413.

    Article  MathSciNet  MATH  Google Scholar 

  22. Callier F M, Winkin J, and Willems J L, Convergence of the time-invariant Riccati differential equation and LQ-problem: Mechanisms of attraction, Internat. J. Control, 1994, 59: 983–1000.

    Article  MathSciNet  MATH  Google Scholar 

  23. Choi C H and Laub A J, Efficient matrix-valued algorithms for solving stiff Riccati differential equations, IEEE Trans. Automat. Control, 1990, 35: 770–776.

    Article  MathSciNet  MATH  Google Scholar 

  24. Dieci L, Numerical integration of the differential Riccati equation and some related issues, SIAM J. Numer. Anal., 1992, 29: 781–815.

    Article  MathSciNet  MATH  Google Scholar 

  25. Hansen E and Stillfjord T, Convergence analysis for splitting of the abstract differential Riccati equation, SIAM J. Numer. Anal., 2014, 52: 3128–3139.

    Article  MathSciNet  MATH  Google Scholar 

  26. Malqvist A, Persson A, and Stillfjord T, Multiscale differential Riccati equations for linear quadratic regulator problems, SIAM J. Sci. Comput., 2018, 40: A2406–A2426.

    Article  MathSciNet  MATH  Google Scholar 

  27. Wang Y, Error analysis of a discretization for stochastic linear quadratic control problems governed by SDEs, IMA J. Math. Control Inform., 2021, 38: 1148–1173.

    Article  MathSciNet  MATH  Google Scholar 

  28. Ait Rami M, Chen X, and Zhou X Y, Discrete-time indefinite LQ control with state and control dependent noises, J. Global Optim., 2003, 23: 245–265.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Southwest University, Chongqing, 400715, China

    Yanqing Wang

Authors
  1. Yanqing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanqing Wang.

Additional information

This work was supported in part by the National Natural Science Foundation of China under Grant No. 11801467, and the Chongqing Natural Science Foundation under Grant No. cstc2018jcyjAX0148.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y. Error Analysis of the Feedback Controls Arising in the Stochastic Linear Quadratic Control Problems. J Syst Sci Complex 36, 1540–1559 (2023). https://doi.org/10.1007/s11424-023-1102-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-023-1102-7

Keywords

Search

Navigation