Role-Oriented Code Generation in an Engine for Solving Hyperbolic PDE Systems

  • Jean-Matthieu GallardEmail author
  • Lukas Krenz
  • Leonhard Rannabauer
  • Anne Reinarz
  • Michael Bader
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1190)


The development of a high performance PDE solver requires the combined expertise of interdisciplinary teams with respect to application domain, numerical scheme and low-level optimization. In this paper, we present how the ExaHyPE engine facilitates the collaboration of such teams by isolating three roles: application, algorithms, and optimization expert. We thus support team members in letting them focus on their own area of expertise while integrating their contributions into an HPC production code.

Inspired by web application development practices, ExaHyPE relies on two custom code generation modules, the Toolkit and the Kernel Generator, which follow a Model-View-Controller architectural pattern on top of the Jinja2 template engine library. Using Jinja2’s templates to abstract the critical components of the engine and generated glue code, we isolate the application development from the engine. The template language also allows us to define and use custom template macros that isolate low-level optimizations from the numerical scheme described in the templates.

We present three use cases, each focusing on one of our user roles, showcasing how the design of the code generation modules allows to easily expand the solver schemes to support novel demands from applications, to add optimized algorithmic schemes (with reduced memory footprint, e.g.), or provide improved low-level SIMD vectorization support.


ExaHyPE Code generation High-order discontinuous Galerkin Hyperbolic PDE systems Model-View-Controller Jinja2 


Acknowledgements and Funding

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 671698. We thank the Gauss Centre for Supercomputing e.V. ( for providing computing resources on the GCS Supercomputer SuperMUC at Leibniz Supercomputing Centre (


  1. 1.
    Alnaes, M.S., Logg, A., Ølgaard, K.B., Rognes, M.E., Wells, G.N.: Unified form language: a domain-specific language for weak formulations of partial differential equations. ACM Trans. Math. Softw. 40(2) (2014)Google Scholar
  2. 2.
    Charrier, D., Hazelwood, B., Weinzierl, T.: Enclave tasking for discontinuous Galerkin methods on dynamically adaptive meshes. SIAM J. Scient. Comput. (in press). arXiv:1806.07984
  3. 3.
    Dumbser, M., Fambri, F., Tavelli, M., Bader, M., Weinzierl, T.: Efficient implementation of ADER discontinuous Galerkin schemes for a scalable hyperbolic PDE engine. Axioms 278 (2018).
  4. 4.
    Dumbser, M., Zanotti, O., Loubère, R., Diot, S.: A posteriori subcell limiting of the discontinuous Galerkin finite element method for hyperbolic conservation laws. J. Comput. Phys. 278(C), 47–75 (2013)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Duru, K., Rannabauer, L., Ling, O.K.A., Gabriel, A.A., Igel, H., Bader, M.: A stable discontinuous Galerkin method for linear elastodynamics in geometrically complex media using physics based numerical fluxes (2019). arXiv:1907.02658
  6. 6.
    Eibl, S., Rüde, U.: A modular and extensible software architecture for particle dynamics. In: 8th International Conference on Discrete Element Methods (2019). arXiv:1906.1096
  7. 7.
    Fambri, F., Dumbser, M., Köppel, S., Rezzolla, L., Zanotti, O.: ADER discontinuous Galerkin schemes for general-relativistic ideal magnetohydrodynamics. Mon. Not. R. Astron. Soc. 477, 4543–4564 (2018)Google Scholar
  8. 8.
    Gassner, G., Lörcher, F., Munz, C.D.: A discontinuous Galerkin scheme based on a space-time expansion II. Viscous flow equations in multi dimensions. J. Sci. Comput. 34(3), 260–286 (2008)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Heinecke, A., Henry, G., Hutchinson, M., Pabst, H.: LIBXSMM: accelerating small matrix multiplications by runtime code generation. In: SC 2016: International Conference for HPC, Networking, Storage and Analysis, pp. 981–991 (2016)Google Scholar
  10. 10.
    Kempf, D., Heß, R., Müthing, S., Bastian, P.: Automatic Code Generation for High-Performance Discontinuous Galerkin Methods on Modern Architectures. arXiv e-prints (2018). arXiv:1812.08075
  11. 11.
    Kirby, R.C., Mitchell, L.: Code generation for generally mapped finite elements. ACM Trans. Math. Softw. 45(4) (2019)Google Scholar
  12. 12.
    Krenz, L., Rannabauer, L., Bader, M.: A high-order discontinuous Galerkin solver with dynamic adaptive mesh refinement to simulate cloud formation processes. In: 13th International Conference on Parallel Processing and Applied Mathematics (PPAM 2019). LNCS, vol. 12043 (2020). arXiv:1905.05524
  13. 13.
    Rathgeber, F., Ham, D.A., Mitchell, L., Lange, M., Luporini, F., McRae, A.T.T., Bercea, G.T., Markall, G.R., Kelly, P.H.J.: Firedrake: automating the finite element method by composing abstractions. ACM Trans. Math. Softw. 43(3), 24 (2017)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Reinarz, A., Charrier, D.E., Bader, M., Bovard, L., Dumbser, M., Duru, K., Fambri, F., Gabriel, A.A., Gallard, J.M., Köppel, S., Krenz, L., Rannabauer, L., Rezzolla, L., Samfass, P., Tavelli, M., Weinzierl, T.: ExaHyPE: an engine for parallel dynamically adaptive simulations of wave problems. Comp. Phys. Comm. 107251 (2020)Google Scholar
  15. 15.
    Tavelli, M., Dumbser, M., Charrier, D.E., Rannabauer, L., Weinzierl, T., Bader, M.: A simple diffuse interface approach on adaptive Cartesian grids for the linear elastic wave equations with complex topography. J. Comp. Phys. 386, 158–189 (2019)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Uphoff, C., Bader, M.: Yet another tensor toolbox for discontinuous Galerkin methods and other applications. ACM Trans. Math. Softw. (under review). arXiv:1903.11521
  17. 17.
    Weinzierl, T.: The Peano software-parallel, automaton-based, dynamically adaptive grid traversals. ACM Trans. Math. Softw. 45(2), 14:1–14:41 (2019)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Zanotti, O., Fambri, F., Dumbser, M., Hidalgo, A.: Space-time adaptive ADER discontinuous Galerkin finite element schemes with a posteriori sub-cell finite volume limiting. Comput. Fluids 118, 204–224 (2015)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Jean-Matthieu Gallard
    • 1
    Email author
  • Lukas Krenz
    • 1
  • Leonhard Rannabauer
    • 1
  • Anne Reinarz
    • 1
  • Michael Bader
    • 1
  1. 1.Department of InformaticsTechnical University of MunichMunichGermany

Personalised recommendations