Skip to main content
SpringerLink
Log in

A Numerical Approximation Framework for the Stochastic Linear Quadratic Regulator on Hilbert Spaces

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

We present an approximation framework for computing the solution of the stochastic linear quadratic control problem on Hilbert spaces. We focus on the finite horizon case and the related differential Riccati equations (DREs). Our approximation framework is concerned with the so-called “singular estimate control systems” (Lasiecka in Optimal control problems and Riccati equations for systems with unbounded controls and partially analytic generators: applications to boundary and point control problems, 2004) which model certain coupled systems of parabolic/hyperbolic mixed partial differential equations with boundary or point control. We prove that the solutions of the approximate finite-dimensional DREs converge to the solution of the infinite-dimensional DRE. In addition, we prove that the optimal state and control of the approximate finite-dimensional problem converge to the optimal state and control of the corresponding infinite-dimensional problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arias, E., Hernández, V., Ibanes, J., Peinado, J.: A family of BDF algorithms for solving differential matrix Riccati equations using adaptive techniques. Procedia Comput. Sci. 1(1), 2569–2577 (2010)

  2. Avalos, G., Lasiecka, I.: Differential Riccati equation for the active control of a problem in structural acoustics. J. Optim. Theory Appl. 91(3), 695–728 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  3. Banks, H.T., Kunisch, K.: The linear regulator problem for parabolic systems. SIAM J. Cont. Optim. 22, 684–698 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  4. Benner, P., Ezzatti, P., Mena, H., Quintana-Ortí, E.S., Remón, A.: Solving matrix equations on multi-core and many-core architectures. Algorithms 6(4), 857–870 (2013)

    Article  MathSciNet  Google Scholar 

  5. Benner, P., Mena, H.: Numerical solution of the infinite-dimensional LQR-problem and the associated differential Riccati equations. Technical Report MPIMD/12-13, MPI Magdeburg Preprint, 2012

  6. Benner, P., Mena, H.: Rosenbrock methods for solving differential Riccati equations. IEEE Trans Automat Contr 58(11), 2950–2957 (2013)

    Article  Google Scholar 

  7. Bensoussan, A., Da Prato, G., Delfour, M.C.: Representation and Control of Infinite Dimensional Systems. Systems & Control: Foundations & Applications, vol. I. Birkäuser, Boston (1992)

    MATH  Google Scholar 

  8. Bensoussan, A., Da Prato, G., Delfour, M.C., Mitter, S.K.: Representation and Control of Infinite Dimensional Systems. Systems & Control: Foundations & Applications, vol. II. Birkäuser, Boston (1992)

    MATH  Google Scholar 

  9. Bismut, J.-M.: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control. Optim. 14, 419–444 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bismut, J.-M.: Contrôle des systḿes linéaires quadratiques: applications de l’intégrale stochastique. In: Séminaire de Probabilités, XII. Lecture Notes in Math., vol. 649. Springer, Berlin, 1978

  11. Bucci, F., Lasiecka, I.: Singular estimates and Riccati theory for thermoelastic plate models with boundary thermal control. Dyn. Cont. Disc. Impuls. Syst. 11, 545–568 (2004)

    MathSciNet  MATH  Google Scholar 

  12. Da Prato, G.: Direct solution of a riccati equation arising in stochastic control theory. Appl. Math. Optim. 11, 191–208 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  13. Da Prato, G., Zabczyk, J.: Stochastic equations in infinite dimensions. Encyclopedia of Mathematics and its Applications, vol. 44. Cambridge University Press, Cambridge (1992)

  14. Dragan, V., Morozan, T., Stoica, A.-M.: Mathematical Methods in Robust Control of Linear Stochastic Systems, 2nd edn. Springer, New York (2013)

    Book  MATH  Google Scholar 

  15. Flandoli, F.: Direct solution of a Riccati equation arising in a stochastic control problem with control and observation on the boundary. Appl. Math. Optim. 14, 107–129 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gibson, J.S.: The Riccati integral equation for optimal control problems in Hilbert spaces. SIAM J. Cont. Optim. 17(4), 537–565 (1979)

    Article  MATH  Google Scholar 

  17. Guatteri, G., Tessitore, G.: On the backward stochastic Riccati equation in infinite dimensions. SIAM J. Control Optim. 44(1), 159–194 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  18. Guatteri, G., Tessitore, G.: Backward stochastic Riccati equations in infinite horizon l-q optimal control with infinite dimensional state space and random coefficients. Appl. Math. Optim. 57, 207–235 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hafizoglu, C.: Linear quadratic regulatory boundary/point control of stochastic partial differential equations systems with unbounded coefficients. Ph.D. Thesis, University of Virginia, US, 2006

  20. Hafizoglu, C., Lasiecka, I., Levajković, T., Mena, H., Tuffaha, A.: The stochastic linear quadratic problem with singular estimates. Preprint 2015

  21. Ichikawa, A.: Dynamic programming approach to stochastic evolution equations. SIAM J. Control. Optim. 17(1), 152–174 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kohlmann, M., Tang, S.: New developments in backward stochastic Riccati equations and their applications. In: Mathematical Finance, Konstanz, 2000. Trends Math. Birkhuser, Basel (2001)

  23. Kohlmann, M., Tang, S.: Global adapted solution of one-dimensional backward stochastic Riccati equations with application to the mean-variance hedging. Stoch. Process. Appl. 97, 1255–1288 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kohlmann, M., Tang, S.: Multidimensional backward stochastic Riccati equations and applications. SIAM J. Control. Optim. 41, 1696–1721 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  25. Kohlmann, M., Zhou, X.Y.: Relationship between backward stochastic differential equations and stochastic controls: a linear-quadratic approach. SIAM J. Control. Optim. 38, 1392–1407 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  26. Kroller, M., Kunisch, K.: Convergence rates for the feedback operators arising in the linear quadratic regulator problem governed by parabolic equations. SIAM J. Numer. Anal. 28(5), 1350–1385 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  27. Kushner, H.J.: Optimal stochastic control. IRE Trans. Auto. Control AC–7, 120–122 (1962)

    Article  Google Scholar 

  28. Lang, N., Mena, H., Saak, J.: On the benefits of the LDL factorization for large-scale differential matrix equation solvers. Linear Algebra Appl. 480, 44–71 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lasiecka, I.: Optimal Control Problems and Riccati Equations for Systems with Unbounded Controls and Partially Analytic Generators: Applications to Boundary and Point Control Problems. Lecture Notes in Mathematics, vol. 1855. Springer, Berlin (2004)

    MATH  Google Scholar 

  30. Lasiecka, I., Triggiani, R.: Control Theory for Partial Differential Equations: Continuous and Approximation Theories I. Abstract Parabolic Systems. Cambridge University Press, Cambridge, UK (2000)

    Book  MATH  Google Scholar 

  31. Lasiecka, I., Triggiani, R.: Control Theory for Partial Differential Equations: Continuous and Approximation Theories II. Abstract Hyperbolic-Like Systems Over a Finite Time Horizon. Cambridge University Press, Cambridge, UK (2000)

    Book  MATH  Google Scholar 

  32. Lasiecka, I., Triggiani, R., Optimal control and differential Riccati equations under singular estimates for \(e^{At}B \) in the absence of analyticity. Adv. Dyn. Control, Special Volume dedicated to A. V. Balakrishnan, Chapman and Hall/CRC Press, 271–309, 2004

  33. Lasiecka, I., Tuffaha, A.: Riccati theory and singular estimates for control of a generalized fluid structure interaction model. Syst. Cont. Lett. 58(7), 499–509 (2009)

    Article  MATH  Google Scholar 

  34. Levajković, T., Pilipović, S., Seleši, D., Žigić, M.: Stochastic evolution equations with multiplicative noise. Electron. J. Probab. 20(19), 1–23 (2015)

    MathSciNet  MATH  Google Scholar 

  35. Levajković, T., Mena, H.: On deterministic and stochastic linear quadratic control problem. Current Trends in Analysis and Its Applications, Trends in Mathematics, Research Perspectives, Springer International Publishing Switzerland 315–322, (2015)

  36. Levajković, T., Mena, H., Tuffaha, A., The Stochastic Linear Quadratic Control Problem: A Chaos Expansion Approach. Evolution Equations and Control Theory, accepted for publication (2016)

  37. Saak, J.: Efiziente numerische lösung eines optimalsteuerungsproblems für die abkühlung von stahlprofilen. Diplomarbeit, Fachbereich 3/Mathematik und Informatik, Universität Bremen, D-28334 Bremen, Germany, 2003

  38. Tröltzsch, F., Unger, A.: Fast solution of optimal control problems in the selective cooling of steel. Z. Angew. Math. Mech. 81, 447–456 (2001)

    MathSciNet  MATH  Google Scholar 

  39. Wonham, W.M.: On the separation theorem of stochastic control. SIAM J. Control. 6, 312–326 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  40. Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control. 6, 681–697 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  41. Yong, J., Zhou, X.Y.: Stochastic Controls - Hamiltonian Systems and HJB Equations. Applications of Mathematics, Stochastic Modelling and Applied Probability, vol. 43. Springer, New York (1999)

    MATH  Google Scholar 

Download references

Acknowledgments

The paper was partially supported by the project Solution of large-scale Lyapunov Differential Equations (P 27926) founded by the Austrian Science Foundation.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Innsbruck, Innsbruck, Austria

    Tijana Levajković & Hermann Mena

  2. Faculty of Traffic and Transport Engineering, University of Belgrade, Belgrade, Serbia

    Tijana Levajković

  3. Department of Mathematics, American University of Sharjah, Sharjah, UAE

    Amjad Tuffaha

Authors
  1. Tijana Levajković
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hermann Mena
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amjad Tuffaha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amjad Tuffaha.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Levajković, T., Mena, H. & Tuffaha, A. A Numerical Approximation Framework for the Stochastic Linear Quadratic Regulator on Hilbert Spaces. Appl Math Optim 75, 499–523 (2017). https://doi.org/10.1007/s00245-016-9339-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-016-9339-3

Keywords

Search

Navigation