Feedback control of parametrized PDEs via model order reduction and dynamic programming principle

Advances in Computational Mathematics volume 46, Article number: 9 (2020) Cite this article

Abstract

In this paper, we investigate infinite horizon optimal control problems for parametrized partial differential equations. We are interested in feedback control via dynamic programming equations which is well-known to suffer from the curse of dimensionality. Thus, we apply parametric model order reduction techniques to construct low-dimensional subspaces with suitable information on the control problem, where the dynamic programming equations can be approximated. To guarantee a low number of basis functions, we combine recent basis generation methods and parameter partitioning techniques. Furthermore, we present a novel technique to construct non-uniform grids in the reduced domain, which is based on statistical information. Finally, we discuss numerical examples to illustrate the effectiveness of the proposed methods for PDEs in two space dimensions.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

References

  1. 1.

    Alla, A., Falcone, M., Kalise, D.: An efficient policy iteration algorithm for dynamic programming equations. SIAM J. Sci. Comput. 37, A181–A200 (2015). https://doi.org/10.1137/130932284

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Alla, A., Falcone, M., Kalise, D.: A HJB-POD feedback synthesis approach for the wave equation. Bulletin of the Brazilian Mathematical Society New Series 47, 51–64 (2016). https://doi.org/10.1007/s00574-016-0121-6

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Alla, A., Falcone, M., Saluzzi, L.: An efficient DP algorithm on a tree-structure for finite horizon optimal control problems. SIAM J. Sci. Comput. 41, A2384–A2406 (2019). https://doi.org/10.1137/18M1203900

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Alla, A., Falcone, M., Volkwein, S.: Error analysis for POD approximations of infinite horizon problems via the dynamic programming approach. SIAM J. Control. Optim. 55, 3091–3115 (2017). https://doi.org/10.1137/15M1039596

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Alla, A., Schmidt, A., Haasdonk, B.: Model order reduction approaches for infinite horizon optimal control problems via the HJB equation. In: Benner, P., Ohlberger, M., Patera, A., Rozza, G., Urban, K. (eds.) Model Reduction of Parametrized Systems, pp 333–347. Springer International Publishing, Cham (2017), https://doi.org/10.1007/978-3-319-58786-8_21

    Google Scholar 

  6. 6.

    Antoulas, A.: Approximation of large–scale dynamical systems. SIAM Publications, Philadelphia (2005). https://doi.org/10.1137/1.9780898718713

    Google Scholar 

  7. 7.

    Atwell, J., King, B.: Proper orthogonal decomposition for reduced basis feedback controllers for parabolic equations. Math. Comput. Model. 33, 1–19 (2001). https://doi.org/10.1016/s0895-7177(00)00225-9

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Bardi, M., Capuzzo-Dolcetta, I.: Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997). https://doi.org/10.1007/978-0-8176-4755-1

    Google Scholar 

  9. 9.

    Barrault, M., Maday, Y., Nguyen, N., Patera, A.: An “empirical interpolation” method: application to efficient reduced-basis discretization of partial differential equations. Comptes Rendus de l’Académie des Sciences Series I 339, 667–672 (2004). https://doi.org/10.1016/j.crma.2004.08.006

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Bellman, R.: Dynamic Programming, 1st edn. Princeton University Press, Princeton (1957)

    Google Scholar 

  11. 11.

    Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57, 483–531 (2015). https://doi.org/10.1137/130932715

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Benner, P., Heiland, J.: LQG-balanced truncation low-order controller for stabilization of laminar flows. In: King, R. (ed.) Active Flow and Combustion Control 2014, vol. 127, pp 365–379. Springer International Publishing (2015)

  13. 13.

    Bokanowski, O., Maroso, S., Zidani, H.: Some convergence results for Howard’s algorithm. SIAM J. Numer. Anal. 47, 3001–3026 (2009). https://doi.org/10.1137/08073041X

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Breiten, T., Kunisch, K.: Riccati-based feedback control of the monodomain equations with the Fitzhugh–Nagumo model. SIAM J. Control. Optim. 52, 4057–4081 (2014). https://doi.org/10.1137/140964552

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Dedé, L.: Reduced basis method and a posteriori error estimation for parametrized linear-quadratic optimal control problems. SIAM J. Sci Comput. 32, 997–1019 (2010). https://doi.org/10.1137/090760453

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Eftang, J. L., Patera, A. T., Rønquist, E. M.: An hp certified reduced basis method for parametrized elliptic partial differential equations. SIAM J. Sci. Comput. 32, 3170–3200 (2010). https://doi.org/10.1137/090780122

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Falcone, M.: A numerical approach to the infinite horizon problem of deterministic control theory. Appl. Math. Optim. 15, 1–13 (1987). https://doi.org/10.1007/BF01442644

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Falcone, M., Ferretti, R.: Semi-lagrangian approximation schemes for linear and Hamilton-Jacobi equations, Society for industrial and applied mathematics. https://doi.org/10.1137/1.9781611973051 (2013)

  19. 19.

    Garcke, J., Kröner, A.: Suboptimal feedback control of PDEs by solving HJB equations on adaptive sparse grids. J. Sci. Comput. 70, 1–28 (2017). https://doi.org/10.1007/s10915-016-0240-7

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Grüne, L., Pannek, J.: Nonlinear model predictive control. In: Nonlinear Model Predictive Control, pp 43–66. Springer, London (2011), https://doi.org/10.1007/978-0-85729-501-9_3

    Google Scholar 

  21. 21.

    Gubisch, M., Volkwein, S.: POD for linear-quadratic optimal control. In: Benner, P., Cohen, A., Ohlberger, M., Willcox, K. (eds.) Model Reduction and Approximation: Theory and Algorithms. SIAM, Philadelphia (2017)

  22. 22.

    Haasdonk, B.: Convergence rates of the POD–Greedy method. ESAIM: Mathematical modelling and numerical Analysis 47, 859–873 (2013). https://doi.org/10.1051/m2an/2012045

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Haasdonk, B., Dihlmann, M., Ohlberger, M.: A training set and multiple basis generation approach for parametrized model reduction based on adaptive grids in parameter space. Math. Comput. Model. Dyn. Syst. 17, 423–442 (2011). https://doi.org/10.1080/13873954.2011.547674

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE Constraints. Mathematical Modelling Theory and Applications. Springer, Berlin (2009). https://doi.org/10.1007/978-1-4020-8839-1

    Google Scholar 

  25. 25.

    Howard, R.A.: Dynamic Programming and Markov processes. MIT Press, Cambridge (1960)

    Google Scholar 

  26. 26.

    Junge, O., Schreiber, A.: Dynamic programming using radial basis functions. Discrete and Continuous Dynamical Systems 35, 4439–4453 (2015). https://doi.org/10.3934/dcds.2015.35.4439

    MathSciNet  Article  MATH  Google Scholar 

  27. 27.

    Kalise, D., Kröner, A.: Reduced-order minimum time control of advection-reaction-diffusion systems via dynamic programming. In: 21st International Symposium on Mathematical Theory of Networks and Systems, Groningen, Netherlands, pp. 1196–1202 (2014). https://hal.archives-ouvertes.fr/hal-01089887

  28. 28.

    Kalise, D., Kunisch, K.: Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs. SIAM J. Sci. Comput. 40, A629–A652 (2018). https://doi.org/10.1137/17M1116635

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Kärcher, M., Grepl, M.: A posteriori error estimation for reduced order solutions of parametrized parabolic optimal control problems. ESAIM: Mathematical Modelling and Numerical Analysis 48, 1615–1638 (2014). https://doi.org/10.1051/m2an/2014012

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    Kunisch, K., Volkwein, S., Xie, L.: HJB-POD based feedback design for the optimal control of evolution problems. SIAM Journal on Applied Dynamical Systems 3, 701–722 (2004). http://epubs.siam.org/doi/abs/10.1137/030600485

    MathSciNet  Article  Google Scholar 

  31. 31.

    Kunisch, K., Xie, L.: POD-based feedback control of the Burgers equation by solving the evolutionary HJB equation. Computers & Mathematics with Applications 49, 1113–1126 (2005). https://doi.org/10.1016/j.camwa.2004.07.022

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Patera, A., Rozza, G.: Reduced basis approximation and a Posteriori Error Estimation for Parametrized Partial Differential Equations, To appear in (tentative) MIT Pappalardo Graduate Monographs in Mechanical Engineering, MIT. http://augustine.mit.edu/methodology/methodology_book.htm (2007)

  33. 33.

    Ravindran, S.S.: A reduced order approach for optimal control of fluids using proper orthogonal decomposition. Int. J. Numer. Methods Fluids 34, 435–448 (2000). https://doi.org/10.1002/1097-0363(20001115)34:5<425::AID-FLD67>3.0.CO;2-W

    MathSciNet  Article  Google Scholar 

  34. 34.

    Santos, M.S., Rust, J.: Convergence properties of policy iteration. SIAM J. Control. Optim. 42, 2094–2115 (2004). https://doi.org/10.1137/S0363012902399824

    MathSciNet  Article  MATH  Google Scholar 

  35. 35.

    Schmidt, A., Haasdonk, B.: Data-driven surrogates of value functions and applications to feedback control for dynamical systems. IFAC-PapersOnLine 51, 307–312 (2018). https://doi.org/10.1016/j.ifacol.2018.03.053

    Article  Google Scholar 

  36. 36.

    Schmidt, A., Haasdonk, B.: Reduced basis approximation of large scale parametric algebraic Riccati equations. ESAIM: Control Optimisation and Calculus of Variations 24, 129–151 (2018). https://doi.org/10.1051/cocv/2017011

    MathSciNet  Article  MATH  Google Scholar 

  37. 37.

    Simoncini, V.: Analysis of the rational Krylov subspace projection method for large-scale algebraic Riccati equations. SIAM Journal on Matrix Analysis and Applications 37, 1655–1674 (2016). https://doi.org/10.1137/16m1059382

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Tröltzsch, F.: Optimal Control of Partial Differential Equations. Theory, Methods and Applications, vol. 112 of Mathematical Modelling: Theory and Applications. American Mathematical Society, Providence (2010)

    Google Scholar 

  39. 39.

    Tröltzsch, F., Volkwein, S.: POD a-posteriori error estimates for linear-quadratic optimal control problems. Comput. Optim. Appl. 44, 83–115 (2009). https://doi.org/10.1007/s10589-008-9224-3

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Max Gunzburger for several fruitful discussions on the subject and valuable comments for improvement of the presentation.

Funding

The first author was supported by US Department of Energy grant number DE-SC0009324. The second and third authors would like to thank the German Research Foundation (DFG) for financial support of the project within the Cluster of Excellence in Simulation Technology (EXC 310/2) at the University of Stuttgart.

Author information

Affiliations

  1. Department of Mathematics, PUC-Rio, Rua Marques de São Vicente, 225, Rio de Janeiro, 22453-900, Brazil

    Alessandro Alla

  2. Institute of Applied Analysis and Numerical Simulation, University of Stuttgart, Pfaffenwaldring 57, 70569, Stuttgart, Germany

    Bernard Haasdonk & Andreas Schmidt

Authors
  1. Alessandro Alla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernard Haasdonk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Alla.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Model Reduction of Parametrized Systems

Guest Editors: Anthony Nouy, Peter Benner, Mario Ohlberger, Gianluigi Rozza, Karsten Urban and Karen Willcox

Communicated by: Anthony Nouy

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alla, A., Haasdonk, B. & Schmidt, A. Feedback control of parametrized PDEs via model order reduction and dynamic programming principle. Adv Comput Math 46, 9 (2020). https://doi.org/10.1007/s10444-020-09744-8

Download citation

Keywords

  • Dynamic programming principle
  • Semi-Lagrangian schemes
  • Hamilton-Jacobi-Bellman equations
  • Optimal control
  • Model reduction
  • Reduced basis method

Mathematics Subject Classification (2010)

  • 49L20
  • 49J20
  • 65N99
  • 78M34