Abstract
The synthesis of suboptimal feedback laws for controlling nonlinear dynamics arising from semi-discretized PDEs is studied. An approach based on the State-dependent Riccati Equation (SDRE) is presented for 2 and ∞ control problems. Depending on the nonlinearity and the dimension of the resulting problem, offline, online, and hybrid offline-online alternatives to the SDRE synthesis are proposed. The hybrid offline-online SDRE method reduces to the sequential solution of Lyapunov equations, effectively enabling the computation of suboptimal feedback controls for two-dimensional PDEs. Numerical tests for the Sine-Gordon, degenerate Zeldovich, and viscous Burgers’ PDEs are presented, providing a thorough experimental assessment of the proposed methodology.
Article PDF
Code availability
The MATLAB source code of the implementations used to compute the presented results can be downloaded from https://github.com/alessandroalla/SDRE
References
Albi, G., Bicego, S., Kalise, D.: Gradient-augmented supervised learning of optimal feedback laws using state-dependent Riccati equations. IEEE Control Syst. Lett. 6, 836–841 (2022)
Alla, A., Falcone, M., Saluzzi, L.: An efficient DP algorithm on a tree-structure for finite horizon optimal control problems. SIAM J. Sci. Comput. 41(4), A2384–A2406 (2019)
Antoulas, A.C.: Approximation of Large-scale Dynamical Systems. Advances in Design and Control. SIAM, Philadelphia (2005)
Azmi, B., Kalise, D., Kunisch, K.: Optimal feedback law recovery by gradient-augmented sparse polynomial regression. J. Machin. Learn. Res. 22(48), 1–32 (2021)
Banks, H.T., Lewis, B.M., Tran, H.T.: Nonlinear feedback controllers and compensators: A state-dependent riccati equation approach. Comput. Optim. Appl. 37(2), 177–218 (2007)
Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-JacobiBellman Equations. Birkhäuser, Boston (1997)
Beeler, S.C., Tran, H.T., Banks, H.T.: Feedback control methodologies for nonlinear systems. J. Optim. Theory Appl. 107(1), 1–33 (2000)
Benner, P., Heiland, J.: Exponential stability and stabilization of extended linearizations via continuous updates of riccati-based feedback. Int. J. Robust Nonlinear Control 28(4), 1218–1232 (2018)
Benner, P., Li, J.R., Penzl, T.: Numerical solution of large-scale Lyapunov equations, Riccati equations, and linear-quadratic optimal control problems. Numer. Linear Algebra Appl. 15(9), 755–777 (2008)
Benner, P., Mehrmann, V., Sorensen, D. (eds.): Dimension Reduction of Large-Scale Systems. Lecture Notes in Computational Science and Engineering. Springer, Berlin (2005)
Benner, P., Saak, J.: Numerical solution of large and sparse continuous time algebraic matrix Riccati and Lyapunov equations: A state of the art survey. GAMM-Mitt., 32–52 (2013)
Bini, D., Iannazzo, B., Meini, B.: Numerical Solution of Algebraic Riccati Equations. SIAM, Philadelphia (2012)
Boyd, S., Balakrishnan, V., Kabamba, P.: A bisection method for computing the \(H_{\infty }\) norm of a transfer matrix and related problems. Math. Control Signals Syst. 2(3), 207–219 (1989)
Chow, Y.T., Darbon, J., Osher, S., Yin, W.: Algorithm for overcoming the curse of dimensionality for time-dependent non-convex Hamilton-Jacobi equations arising from optimal control and differential games problems. J. Sci. Comput. 73(2-3), 617–643 (2017)
Chow, Y.T., Darbon, J., Osher, S., Yin, W.: Algorithm for overcoming the curse of dimensionality for state-dependent Hamilton-Jacobi equations. J. Comput. Phys. 387, 376–409 (2019)
Cloutier, J.R.: State-dependent Riccati equation techniques: an overview. In: Proceedings of the 1997 American Control Conference (Cat. No.97CH36041), vol. 2, pp 932–936 (1997)
Cloutier, J.R., D’Souza, C.N., Mracek, C.P.: Nonlinear regulation and nonlinear \(H_{\infty }\) control via the state-dependent Riccati equation technique. I. Theory. In: First International Conference on Nonlinear Problems in Aviation and Aerospace (Daytona Beach, FL, 1996), pp 117–130. Embry-Riddle Aeronaut. Univ. Press, Daytona Beach (1997)
Cloutier, J.R., D’Souza, C.N., Mracek, C.P.: Nonlinear regulation and nonlinear \(H_{\infty }\) control via the state-dependent Riccati equation technique. II. Examples. In: First International Conference on Nonlinear Problems in Aviation and Aerospace (Daytona Beach, FL, 1996), pp 131–141. Embry-Riddle Aeronaut. Univ. Press, Daytona Beach (1997)
Darbon, J., Langlois, G.P., Meng, T.: Overcoming the curse of dimensionality for some Hamilton-Jacobi partial differential equations via neural network architectures. Res. Math. Sci. 7(3), Paper 20, 50 (2020)
Dolgov, S., Kalise, D., Kunisch, K.: Tensor decompositions for high-dimensional Hamilton-Jacobi-Bellman equations. SIAM J. Sci. Comput. 43, A1625–A1650 (2021)
Garcke, J., Kröner, A.: Suboptimal feedback control of PDEs by solving HJB equations on adaptive sparse grids. J. Sci. Comput. 70(1), 1–28 (2017)
Gilding, B.H., Kersner, R.: Travelling Waves in Nonlinear Diffusion-convection Reaction. Progress in Nonlinear Differential Equations and their Applications, vol. 60. Basel, Birkhäuser (2004)
Gorodetsky, A., Karaman, S., Marzouk, Y.: High-dimensional stochastic optimal control using continuous tensor decompositions. Int. J. Robot. Res. 37(2-3), 340–377 (2018)
Grüne, L., Pannek, J.: Nonlinear Model Predictive Control. Communications and Control Engineering Series. Springer, London (2011). Theory and algorithms
Grüne, L., Rantzer, A.: On the infinite horizon performance of receding horizon controllers. IEEE Trans. Automat. Control 53(9), 2100–2111 (2008)
Han, J., Jentzen, A.E.W.: Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. USA 115(34), 8505–8510 (2018)
Herty, M., Kalise, D.: Suboptimal nonlinear feedback control laws for collective dynamics. In: 2018 IEEE 14th International Conference on Control and Automation (ICCA), pp 556–561 (2018)
Ito, K., Reisinger, C., Zhang, Y.: A neural network-based policy iteration algorithm with global h2-superlinear convergence for stochastic games on domains. Found. Comput. Math. 21, 331–374 (2021)
Jones, A., Astolfi, A.: On the solution of optimal control problems using parameterized state-dependent Riccati equations. In: 2020 59th IEEE Conference on Decision and Control (CDC), pp. 1098–1103 (2020)
Kalise, D., Kundu, S., Kunisch, K.: Robust feedback control of nonlinear PDEs by numerical approximation of high-dimensional Hamilton-Jacobi-Isaacs equations. SIAM J. Appl. Dyn. Syst. 19(2), 1496–1524 (2020)
Kalise, D., Kunisch, K.: Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs. SIAM J. Sci. Comput. 40(2), A629–A652 (2018)
Kang, W., Gong, Q., Nakamura-Zimmerer, T.: Algorithms of data generation for deep learning and feedback design: A survey. Physica D: Nonlin. Phenom. 425, 132955 (2021)
Kang, W., Wilcox, L.C.: Mitigating the curse of dimensionality: Sparse grid characteristics method for optimal feedback control and HJB equations. Comput. Optim. Appl. 68(2), 289–315 (2017)
Knoll, D.A., Keyes, D.E.: Jacobian-free Newton-Krylov methods: A survey of approaches and applications. J. Comput. Phys. 193(2), 357–397 (2004)
Kressner, D.: Memory-efficient Krylov subspace techniques for solving large-scale Lyapunov equations. In: IEEE International Symposium on Computer-Aided Control Systems, pp. 613–618. San Antonio (2008)
Kunisch, K., Walter, D.: Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation. ESAIM:COCV, 27 (2021)
Laub, A.: A Schur method for solving algebraic Riccati equations. IEEE Trans. Automat. Control 24(6), 913–921 (1979)
The MathWorks, Inc.: MATLAB 7 r2017b edn (2017)
Nakamura-Zimmerer, T., Gong, Q., Kang, W.: Adaptive deep learning for high-dimensional Hamilton–Jacobi–Bellman equations. SIAM J. Sci. Comput. 43(2), A1221–A1247 (2021)
Nüsken, N., Richter, L.: Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: Perspectives from the theory of controlled diffusions and measures on path space. Partial Diff. Equ. Applic. 2(4), 1–48 (2021)
Oster, M., Sallandt, L., Schneider, R.: Approximating the stationary Hamilton-Jacobi-Bellman equation by hierarchical tensor products. arXiv:1911.00279 (2019)
Palitta, D., Pozza, S., Simoncini, V.: The short-term rational Lanczos method and applications. SIAM J. Sci. Comput. 44(4), A2843–A2870 (2022). https://doi.org/10.1137/21M1403254
Palitta, D., Simoncini, V.: Computationally enhanced projection methods for symmetric Sylvester and Lyapunov matrix equations. J. Comput. Applied Math. 330, 648–659 (2018)
Petcu, M., Temam, R.: Control for the sine-gordon equation. ESAIM: Control Optimisation and Calculus of Variations 10(4), 553–573 (2004)
van der Schaft, A.J.: L2-gain analysis of nonlinear systems and nonlinear state feedback \(H_{\infty }\) control. IEEE Trans. A.tomat. Control 37(6), 770–784 (1992)
Simoncini, V.: Computational methods for linear matrix equations. SIAM Rev. 58(3), 377–441 (2016)
Simoncini, V., Szyld, D.B., Monsalve, M.: On two numerical methods for the solution of large-scale algebraic Riccati equations. IMA J. Numer. Anal. 34(3), 904–920 (2014)
Slowik, M., Benner, P., Sima, V.: Evaluation of the linear matrix equation solvers in SLICOT. J. of Numer. Anal. Industr. Appl. Math. 2(1-2), 11–34 (2007)
Soravia, P.: \({\mathscr{H}}^{\infty }\) control of nonlinear systems: differential games and viscosity solutions. SIAM J. Control Optim. 34, 1071–1097 (1996)
Soravia, P.: Equivalence between nonlinear \({\mathscr{H}}^{\infty }\) control problems and existence of viscosity solutions of Hamilton-jacobi-isaacs equations. Appl. Math. Optim 39, 17–32 (1999)
Stefansson, E., Leong, Y.P.: Sequential alternating least squares for solving high dimensional linear Hamilton-Jacobi-Bellman equation. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3757–3764 (2016)
Wang, J., Wu, G.: A multilayer recurrent neural network for solving continuous-time algebraic Riccati equations. Neural Netw. 11(5), 939–950 (1998)
Wernli, A.: Suboptimal control for the nonlinear quadratic regulator problem. Automatica—J IFAC 11, 75–84 (1975)
Funding
Part of this work was supported by the Indam-GNCS 2019 Project “Tecniche innovative e parallele per sistemi lineari e non lineari di grandi dimensioni, funzioni ed equazioni matriciali ed applicazioni”. VS is a member of the GNCS-Indam activity group. DK was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) grants EP/V04771X/1, EP/T024429/1, and EP/V025899/1. AA was supported by the CNPq research grant 3008414/2019-1 and by a research grant from PUC-Rio. AA is member of the INDAM-GNCS activity group.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Communicated by: Peter Benner
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Alla, A., Kalise, D. & Simoncini, V. State-dependent Riccati equation feedback stabilization for nonlinear PDEs. Adv Comput Math 49, 9 (2023). https://doi.org/10.1007/s10444-022-09998-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10444-022-09998-4
Keywords
- Stabilization of PDEs
- State-dependent Riccati equations
- Algebraic Riccati Equations
- Lyapunov equations
- Numerical approximation