Journal of Scientific Computing

, Volume 70, Issue 1, pp 1–28 | Cite as

Suboptimal Feedback Control of PDEs by Solving HJB Equations on Adaptive Sparse Grids

Article

Abstract

An approach to solve finite time horizon suboptimal feedback control problems for partial differential equations is proposed by solving dynamic programming equations on adaptive sparse grids. A semi-discrete optimal control problem is introduced and the feedback control is derived from the corresponding value function. The value function can be characterized as the solution of an evolutionary Hamilton–Jacobi Bellman (HJB) equation which is defined over a state space whose dimension is equal to the dimension of the underlying semi-discrete system. Besides a low dimensional semi-discretization it is important to solve the HJB equation efficiently to address the curse of dimensionality. We propose to apply a semi-Lagrangian scheme using spatially adaptive sparse grids. Sparse grids allow the discretization of the value functions in (higher) space dimensions since the curse of dimensionality of full grid methods arises to a much smaller extent. For additional efficiency an adaptive grid refinement procedure is explored. The approach is illustrated for the wave equation and an extension to equations of Schrödinger type is indicated. We present several numerical examples studying the effect the parameters characterizing the sparse grid have on the accuracy of the value function and the optimal trajectory.

Keywords

Closed-loop suboptimal control of PDEs HJB equations Sparse grids Curse of dimensionality 

Mathematics Subject Classification

49K20 49N35 49L20 65D15 

References

  1. 1.
    Alla, A., Falcone, M.: An adaptive POD approximation method for the control of advection-diffusion equations. Int. Ser. Numer. Math. 164, 1–17 (2013)MATHGoogle Scholar
  2. 2.
    Alla, A., Falcone, M., Kalise, D.: HJB-POD based feedback design approach for the wave equation. Bull. Braz. Math. Soc. 47(1), 51–64 (2016)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Aronna, M.S., Bonnans, J.F., Kröner, A.: Optimal Control of PDEs in a Complex Space Setting; Application to the Schrödinger Equation. Research report, INRIA, 2016, HAL report: hal-01311421. https://hal.archives-ouvertes.fr/hal-01311421/file/ABK_2016_v1.pdf
  4. 4.
    Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations. Birkhäuser, Boston (2008)MATHGoogle Scholar
  5. 5.
    Barles, G., Souganidis, P.E.: Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4(3), 271–283 (1991)MathSciNetMATHGoogle Scholar
  6. 6.
    Baur, U., Benner, P.: Modellreduktion für parametrisierte Systeme durch balanciertes Abschneiden und Interpolation (Model reduction for parametric systems using balanced truncation and interpolation). at-Automatisierungstechnik 578, 411–420 (2009)Google Scholar
  7. 7.
    Beauchard, K., Nersesyan, V.: Semi-global weak stabilization of bilinear Schrödinger equations. C. R. Math. 348(19–20), 1073–1078 (2010)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Bokanowski, O., Desilles, A., Zidani, H.: ROC-HJ-Solver. A C++ Library for Solving HJ Equations (2013). http://uma.ensta-paristech.fr/soft/ROC-HJ/
  9. 9.
    Bokanowski, O., Forcadel, N., Zidani, H.: Reachability and minimal times for state constrained nonlinear problems without any controllability assumption. SIAM J. Control Optim. 48(7), 4292–4316 (2010)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-Lagrangian scheme for first order Hamilton–Jacobi Bellman equations. J. Sci. Comput. 55(3), 575–605 (2013)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Bungartz, H.-J., Griebel, M.: Sparse grids. Acta Numer. 13, 1–123 (2004)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Buse, G.: Exploiting Many-Core Architectures for Dimensionally Adaptive Sparse Grids. Dissertation, Institut für Informatik, Technische Universität München, München (2015)Google Scholar
  13. 13.
    Carlini, E., Falcone, M., Ferretti, R.: An efficient algorithm for Hamilton–Jacobi equations in high dimension. Comput. Vis. Sci. 7(1), 15–29 (2004)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Chkifa, A., Cohen, A., Schwab, C.: High-dimensional adaptive sparse polynomial interpolation and applications to parametric PDEs. Found. Comput. Math. 14(4), 601–633 (2014)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Falcone, M., Ferretti, R.: Semi-Lagrangian approximation schemes for linear and Hamilton–Jacobi equations. In: Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia (2013)Google Scholar
  16. 16.
    Ferretti, R.: Internal approximation schemes for optimal control problems in Hilbert spaces. J. Math. Syst. Estim. Control 7(1), 1–25 (1997)MathSciNetMATHGoogle Scholar
  17. 17.
    Feuersänger, C.: Sparse Grid Methods for Higher Dimensional Approximation. Dissertation, Institut für Numerische Simulation, Universität Bonn (2010)Google Scholar
  18. 18.
    Fleming, W.H., Soner, H.M.: Controlled Markov processes and viscosity solutions. In: Stochastic Modelling and Applied Probability. Springer, New York (2006)Google Scholar
  19. 19.
    Garcke, J.: Sparse grids in a nutshell. In: Garcke, J., Griebel, M. (eds.) Sparse Arids and Applications. Lecture Notes in Computational Science and Engineering, vol. 88, pp. 57–80. Springer, Berlin (2012)Google Scholar
  20. 20.
    Garcke, J., Griebel, M.: On the computation of the eigenproblems of hydrogen and helium in strong magnetic and electric fields with the sparse grid combination technique. J. Comput. Phys. 165(2), 694–716 (2000)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Gibson, J.S.: An analysis of optimal modal regulation: convergence and stability. SIAM J. Control Optim. 19(5), 686–707 (1981)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Gombao, S.: Approximation of optimal controls for semilinear parabolic PDE by solving Hamilton–Jacobi–Bellman equations. In: Proceedings of the 15th International Symposium on the Mathematical Theory of Networks and Systems, of Notre Dame, South Bend, IN, USA (2002)Google Scholar
  23. 23.
    Griebel, M.: A parallelizable and vectorizable multi-level algorithm on sparse grids. In: Hackbusch, W. (ed.) Parallel Algorithms for Partial Differential Equations, Notes on Numerical Fluid Mechanics, vol. 31, pp. 94–100. Vieweg, Braunschweig (1991)Google Scholar
  24. 24.
    Griebel, M., Hamaekers, J.: Sparse grids for the Schrödinger equation. Math. Model. Numer. Anal. 41(2), 215–247 (2007)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Griebel, M., Knapek, S.: Optimized general sparse grid approximation spaces for operator equations. Math. Comput. 78(268), 2223–2257 (2009)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Hinze, M., Volkwein, S.: Proper orthogonal decomposition surrogate models for nonlinear dynamical systems: error estimates and suboptimal control. In: Benner, P., Mehrmann, V., Sorensen, D.C. (eds.) Dimension Reduction of Large-Scale Systems. Lecture Notes in Computer Science Engineering, vol. 45, pp. 261–306. Springer, Berlin (2005)Google Scholar
  27. 27.
    Horowitz, M.B., Damle, A., Burdick, J.W.: Linear Hamilton Jacobi Bellman equations in high dimensions. In: IEEE 53rd Annual Conference on Decision and Control (CDC), pp. 5880–5887 (2014)Google Scholar
  28. 28.
    Hu, C., Shu, C.: A discontinuous Galerkin finite element method for Hamilton–Jacobi equations. SIAM J. Sci. Comput. 21(2), 666–690 (1999)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Ishii, H.: Uniqueness of unbounded viscosity solution of Hamilton–Jacobi equations. Indiana Univ. Math. J. 33(5), 721–748 (1984)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Kalise, D., Kröner, A., Kunisch, K.: Local minimization in dynamic programming equations. SIAM J. Sci. Comput. (2016) (preprint). https://www.ricam.oeaw.ac.at/publications/reports/15/rep15-04.pdf
  31. 31.
    Kröner, A., Kunisch, K., Zidani, H.: Optimal feedback control of the undamped wave equation by solving a HJB equation. ESAIM Control Optim. Calc. Var. 21(2), 442–464 (2015)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Kunisch, K., Volkwein, S., Xie, L.: HJB-POD based feedback design for the optimal control of evolution problems. SIAM J. Appl. Dyn. Syst. 4, 701–722 (2004)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Kunisch, K., Xie, L.: POD-based feedback control of the Burgers equation by solving the evolutionary HJB equation. Comput. Math. Appl. 49, 1113–1126 (2005)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Lasiecka, I., Trigginai, R.: Control of Partial Differential Equations: Continuous and Approximation Theories, vol. I. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  35. 35.
    Lions, J.-L., Magenes, E.: Non-homogeneous Boundary Value Problems and Applications, vol. I. Springer, Berlin (1972)CrossRefMATHGoogle Scholar
  36. 36.
    Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations. Grundlehren der mathematischen Wissenschaften. Springer, Berlin (1971)Google Scholar
  37. 37.
    Mirrahimi, M., Handel, R.V.: Stabilizing feedback controls for quantum systems. SIAM J. Control Optim. 46(2), 445–467 (2007)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Mirrahimi, M., Rouchon, P., Turinici, G.: Lyapunov control of bilinear Schrödinger equations. Automatica 41(11), 1987–1994 (2005)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Morris, K.A.: Control of systems governed by partial differential equations. In: Levine, W.S. (ed.) The IEEE Control Theory Handbook. CRC Press, Boca Raton (2010)Google Scholar
  40. 40.
    Novak, E., Ritter, K.: Global optimization using hyperbolic cross points. In: Floudas, C.A. (eds.) State of the Art in Global Optimization: Computational Methods and Applications. Nonconvex Optim. Appl., vol. 7, pp. 19–33. Kluwer, Dordrecht (1996)Google Scholar
  41. 41.
    Osher, S., Shu, C.W.: High-order essentially nonoscillatory schemes for Hamilton–Jacobi equations. SIAM J. Numer. Anal. 28(4), 907–922 (1991)MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Peherstorfer, B., Zimmer, S., Bungartz, H.-J.: Model reduction with the reduced basis method and sparse grids. In: Garcke, J., Griebel, M. (eds.) Sparse Grids and Applications. LNCSE, vol. 88, pp. 223–242. Springer, Berlin (2013) (English) Google Scholar
  43. 43.
    Pflüger, D.: Spatially Adaptive Sparse Grids for High-Dimensional Problems. Dissertation, Institut für Informatik, Technische Universität München, München (2010)Google Scholar
  44. 44.
    Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk. SSSR 148, 1042–1043 (1963) (Russian), Engl. Transl.: Soviet Math. Dokl. 4:240–243 (1963)Google Scholar
  45. 45.
    Springer, R.: Lösung von Hamilton–Jacobi–Bellman–Gleichungen auf dünnen Gittern. Diplomarbeit, University Chemnitz (2013)Google Scholar
  46. 46.
    Warin, X.: Adaptive Sparse Grids for Time Dependent Hamilton–Jacobi–Bellman Equations in Stochastic Control (2014). arXiv:1408.4267
  47. 47.
    Zenger, C.: Sparse grids. In:Hackbusch, W. (ed.) Parallel Algorithms for Partial Differential Equations. Notes on Numerical Fluid Mechanics, vol. 31, Vieweg, Braunschweig, pp. 241–251 (1991)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Institut für Numerische SimulationUniversität BonnBonnGermany
  2. 2.Fraunhofer SCAISankt AugustinGermany
  3. 3.INRIA Saclay and CMAP, École PolytechniquePalaiseau CedexFrance

Personalised recommendations