Skip to main content

Enhancing OpenFOAM’s Performance on HPC Systems

  • 851 Accesses


OpenFOAM is one of the most popular open source tools for CFD simulations of engineering applications. It is therefore also often used on supercomputers to perform large eddy simulations or even direct numerical simulations of complex cases. In this work, general guidelines for improving OpenFOAM’s performance on HPC clusters are given. A comparison of the serial performance for different compilers shows that the Intel compiler generally generates the fastest executables for different standard applications. More aggressive compiler optimization options beyond O3 yield performance increases of about 5 % for simple cases and can lead to improvements of up to 25 % for reactive flow cases. Link-time optimization does not lead to a performance gain. The parallel scaling behavior of reactive flow solvers shows an optimum at 5000 cells per MPI rank in the tested cases, where caching effects counterbalance communication overhead, leading to super linear scaling. In addition, two self-developed means of improving performance are presented: the first one targets OpenFOAM’s most accurate discretization scheme “cubic”. In this scheme, some polynomials are unnecessarily reevaluated during the simulation. A simple change in the code can reuse the results and achieve performance gains of about 5 %. Secondly, the performance of reactive flow solvers is investigated with Score-P/Vampir and load imbalances due to the computation of the chemical reaction rates are identified. A dynamic-adaptive load balancing approach has been implemented for OpenFOAM’s reacting flow solvers which can decrease computation times by 40 % and increases the utilization of the HPC hardware. This load balancing approach utilizes the special feature of the reaction rate computation, that no information of neighboring cells are required, allowing to implement the load balancing efficiently.


  • OpenFOAM
  • Load balancing
  • Reactive flows
  • Performance optimization
  • Combustion

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-66792-4_16
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   189.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-66792-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   249.99
Price excludes VAT (USA)
Hardcover Book
USD   249.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12


  1. OpenCFD, OpenFOAM: The Open Source CFD Toolbox. User Guide Version 1.4, OpenCFD Limited. Reading UK (2007)

    Google Scholar 

  2. T. Poinsot, D. Veynante, Theoretical and Numerical Combustion (R.T, Edwards, 2001)

    Google Scholar 

  3. R. Kee, M. Coltrin, P. Glarborg, Chemically reacting flow: theory and practice (John Wiley & Sons, 2005)

    Google Scholar 

  4. G.I. Taylor, A.E. Green, Mechanism of the production of small eddies from large ones, Proceedings of the Royal Society of London Series A-Mathematical and Physical Sciences, vol. 158, no. 895, pp. 499–521 (1937)

    Google Scholar 

  5. G. Smith, D. Golden, M. Frenklach, N. Moriarty, B. Eiteneer, M. Goldenberg et al., Gri 3.0 reaction mechanism

    Google Scholar 

  6. Karlsruhe institute of technology (2018),

  7. High performance computing center stuttgart (2018)

  8. T. Zirwes, F. Zhang, J. Denev, P. Habisreuther, H. Bockhorn, Automated code generation for maximizing performance of detailed chemistry calculations in OpenFOAM, in High Performance Computing in Science and Engineering ’17, ed. by W. Nagel, D. Kröner, M. Resch (Springer, 2017) pp. 189–204

    Google Scholar 

  9. T. Zirwes, F. Zhang, P. Habisreuther, J. Denev, H. Bockhorn, D. Trimis, Optimizing load balancing of reacting flow solvers in openfoam for high performance computing. ESI (2018)

    Google Scholar 

  10. Suite of nonlinear and differential/algebraic equation solvers

  11. Score-p tracing tool,

  12. Vampir visualization tool,

  13. T. Zirwes, F. Zhang, P. Habisreuther, M. Hansinger, H. Bockhorn, M. Pfitzner, D. Trimis, Quasi-DNS dataset of a piloted flame with inhomogeneous inlet conditions (Turb. and Combust, Flow, 2019)

    Google Scholar 

  14. H. Zhou, J. You, S. Xiong, Y. Yang, D. Thévenin, S. Chen, Interactions between the premixed flame front and the three-dimensional taylor-green vortex. Proc. Combust. Instit. 37(2), 2461–2468 (2019)

    CrossRef  Google Scholar 

  15. P. Boivin, Reduced-kinetic mechanisms for hydrogen and syngas combustion including autoignition (Universidad Carlos III, Madrid, Spain, Disseration, 2011)

    Google Scholar 

Download references


This work was supported by the Helmholtz Association of German Research Centres (HGF) through the Research Unit EMR, Topic 4 Gasification (34.14.02). This work was performed on the national supercomputer Cray XC40 Hazel Hen at the High Performance Computing Center Stuttgart (HLRS) and on the computational resource ForHLR II with the acronym DNSbomb funded by the Ministry of Science, Research and the Arts Baden-Württemberg and DFG (“Deutsche Forschungsgemeinschaft”).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thorsten Zirwes .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Zirwes, T., Zhang, F., Denev, J.A., Habisreuther, P., Bockhorn, H., Trimis, D. (2021). Enhancing OpenFOAM’s Performance on HPC Systems. In: Nagel, W.E., Kröner, D.H., Resch, M.M. (eds) High Performance Computing in Science and Engineering '19. Springer, Cham.

Download citation