Skip to main content
SpringerLink
Log in

Reinforcement Learning-Based Model Reduction for Partial Differential Equations: Application to the Burgers Equation

  • Chapter
  • First Online:
Handbook of Reinforcement Learning and Control

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 325))

Abstract

Partial differential equations (PDEs) are important mathematical models, which are used to model complex dynamic systems in applied sciences. For example, PDEs are used to model flexible beams and ropes [1, 2], crowd dynamics [3, 4], or fluid dynamics [5, 6]. However, PDEs are infinite-dimensional systems, making them hard to solve in closed form, and computationally demanding to solve numerically. For instance, when using finite element methods (FEM), one may end up with a very large discretization space, which incurs large computation times. Because of this complexity, it is often hard to use PDEs to analyze, predict, or control these systems in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Montseny, G., Audounet, J., Matignon, D.: Fractional integro-differential boundary control of the Euler–Bernoulli beam. In: IEEE, Conference on Decision and Control, San Diego, California, pp. 4973–4978 (1997)

    Google Scholar 

  2. Benosman, M.: Lyapunov-based control of the sway dynamics for elevator ropes. IEEE Trans. Control Syst. Technol. 22(5), 1855–1863 (2014)

    Google Scholar 

  3. Huges, R.: The flow of human crowds. Annu. Rev. Fluid Mech. 35, 169–182 (2003)

    Article  MathSciNet  Google Scholar 

  4. Colombo, R.M., Rosini, M.D.: Pedestrian flows and non-classical shocks. Math. Methods Appl. Sci. 28(13), 1553–1567 (2005)

    Article  MathSciNet  Google Scholar 

  5. Cordier, L., Noack, B., Tissot, G., Lehnasch, G., Delville, J., Balajewicz, M., Daviller, G., Niven, R.K.: Identification strategies for model-based control. Exp. Fluids 54(1580), 1–21 (2013)

    Google Scholar 

  6. Balajewicz, M., Dowell, E., Noack, B.: Low-dimensional modelling of high-Reynolds-number shear flows incorporating constraints from the Navier-Stokes equation. J. Fluid Mech. 729(1), 285–308 (2013)

    Article  MathSciNet  Google Scholar 

  7. Holmes, P., Lumley, J.L., Berkooz, G.: Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge University Press, Cambridge (1998)

    Google Scholar 

  8. Couplet, M., Basdevant, C., Sagaut, P.: Calibrated reduced-order POD-Galerkin system for fluid flow modelling. J. Comput. Phys. 207(1), 192–220 (2005)

    Article  MathSciNet  Google Scholar 

  9. Kalb, V.L., Deane, A.E.: An intrinsic stabilization scheme for proper orthogonal decomposition based low-dimensional models. Phys. Fluids 19(5), 054106 (2007)

    Article  Google Scholar 

  10. Bui-Thanh, T., Willcox, K., Ghattas, O., van Bloemen Waanders B.: Goal-oriented, model-constrained optimization for reduction of large-scale systems. J. Comput. Phys. 224(2), 880–896 (2007)

    Google Scholar 

  11. Ilak, M., Bagheri, S., Brandt, L., Rowley, C.W., Henningson, D.S.: Model reduction of the nonlinear complex Ginzburg-Landau equation. SIAM J. Appl. Dyn. Syst. 9(4), 1284–1302 (2010)

    Article  MathSciNet  Google Scholar 

  12. Kalashnikova, I., van Bloemen Waanders, B., Arunajatesan, S., Barone, M.: Stabilization of projection-based reduced order models for linear time-invariant systems via optimization-based eigenvalue reassignment. Comput. Methods Appl. Mech. Eng. 272, 251–270 (2014)

    Google Scholar 

  13. Wang, Z., Akhtar, I., Borggaard, J., Iliescu, T.: Proper orthogonal decomposition closure models for turbulent flows: a numerical comparison. Comput. Methods Appl. Mech. Eng. 237–240, 10–26 (2012)

    Article  MathSciNet  Google Scholar 

  14. San, O., Iliescu, T.: Proper orthogonal decomposition closure models for fluid flows: Burgers equation. Int. J. Numer. Anal. Model. 1(1), 1–18 (2013)

    MATH  Google Scholar 

  15. San, O., Borggaard, J.: Basis selection and closure for POD models of convection dominated Boussinesq flows. In: 21st International Symposium on Mathematical Theory of Networks and Systems, pp. 132–139. Groningen, The Netherlands (2014)

    Google Scholar 

  16. Benosman, M., Kramer, B., Boufounos, P.T., Grover, P.: Learning-based reduced order model stabilization for partial differential equations: application to the coupled Burgers’ equation. In: American Control Conference, pp. 1673–1678 (2016)

    Google Scholar 

  17. Benosman, M., Borggaard, J., San, O., Kramer, B.: Learning-based robust stabilization for reduced-order models of 2D and 3D Boussinesq equations. Appl. Math. Model. 49, 162–181 (2016)

    Article  MathSciNet  Google Scholar 

  18. Benosman, M.: Learning-Based Adaptive Control: An Extremum Seeking Approach - Theory and Applications. Elsevier Ed., Butterworth-Heinemann, Oxford (2016)

    Google Scholar 

  19. Balajewicz, M.: Lyapunov stable Galerkin models of post-transient incompressible flows. Technical report (2013). arXiv:1312.0284

  20. Gunzburger, M.: Finite Element Methods for Viscous Incompressible Flows. Academic, Cambridge (1989)

    Google Scholar 

  21. Kunisch, K., Volkwein, S.: Galerkin proper orthogonal decomposition methods for a general equation in fluid dynamics. SIAM J. Numer. Anal. 40(2), 492–515 (2007)

    Article  MathSciNet  Google Scholar 

  22. Kramer, B., Grover, P., Boufounos, P., Nabi, S., Benosman, M.: Sparse sensing and DMD based identification of flow regimes and bifurcations in complex flows. SIAM J. Appl. Dyn. Syst. 16(2), 1164–1196 (2016)

    Article  MathSciNet  Google Scholar 

  23. Veroy, K., Patera, A.: Certified real-time solution of the parametrized steady incompressible Navier-Stokes equations: rigorous reduced-basis a posteriori error bounds. Int. J. Numer. Methods Fluids 47(8), 773–788 (2005)

    Article  MathSciNet  Google Scholar 

  24. Chakrabarty, A., Jha, D.K., Wang, Y.: Data-driven control policies for partially known systems via kernelized Lipschitz learning. In: Proceedings of the 2019 American Control Conference (ACC), Philadelphia, PA, USA, pp. 4192–4197 (2019)

    Google Scholar 

  25. Chakrabarty, A., Jha, D.K., Wang, Y., Buzzard, G.T., Vamvoudakis, K.: Safe approximate dynamic programming via kernelized Lipschitz estimation. arXiv:1907.02151

  26. Lewis, L.F., Vrabie, D., Vamvoudakis, K.G.: Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control. Syst. Mag. 32(6), 76–105 (2012)

    Article  MathSciNet  Google Scholar 

  27. Liu, D., Wei, Q.: Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 621–634 (2014)

    Google Scholar 

  28. Sanfelice, R.G., Teel, A.R.: Dynamical properties of hybrid systems simulators. Automatica 46, 239–248 (2010)

    Article  MathSciNet  Google Scholar 

  29. Sanfelice, R.G.: Robust hybrid control systems. Ph.D. dissertation, University of California Santa Barbara (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, 02139, USA

    Mouhacine Benosman & Ankush Chakrabarty

  2. Interdisciplinary Center for Applied Mathematics, Virginia Tech, Blacksburg, VA, 24061, USA

    Jeff Borggaard

Authors
  1. Mouhacine Benosman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ankush Chakrabarty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeff Borggaard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mouhacine Benosman .

Editor information

Editors and Affiliations

  1. The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA

    Kyriakos G. Vamvoudakis

  2. Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX, USA

    Yan Wan

  3. Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX, USA

    Frank L. Lewis

  4. Army Research Office, Durham, NC, USA

    Derya Cansever

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Benosman, M., Chakrabarty, A., Borggaard, J. (2021). Reinforcement Learning-Based Model Reduction for Partial Differential Equations: Application to the Burgers Equation. In: Vamvoudakis, K.G., Wan, Y., Lewis, F.L., Cansever, D. (eds) Handbook of Reinforcement Learning and Control. Studies in Systems, Decision and Control, vol 325. Springer, Cham. https://doi.org/10.1007/978-3-030-60990-0_11

Download citation

Publish with us

Policies and ethics

Search

Navigation