Enhancing OpenFOAM’s Performance on HPC Systems

Abstract

OpenFOAM is one of the most popular open source tools for CFD simulations of engineering applications. It is therefore also often used on supercomputers to perform large eddy simulations or even direct numerical simulations of complex cases. In this work, general guidelines for improving OpenFOAM’s performance on HPC clusters are given. A comparison of the serial performance for different compilers shows that the Intel compiler generally generates the fastest executables for different standard applications. More aggressive compiler optimization options beyond O3 yield performance increases of about 5 % for simple cases and can lead to improvements of up to 25 % for reactive flow cases. Link-time optimization does not lead to a performance gain. The parallel scaling behavior of reactive flow solvers shows an optimum at 5000 cells per MPI rank in the tested cases, where caching effects counterbalance communication overhead, leading to super linear scaling. In addition, two self-developed means of improving performance are presented: the first one targets OpenFOAM’s most accurate discretization scheme “cubic”. In this scheme, some polynomials are unnecessarily reevaluated during the simulation. A simple change in the code can reuse the results and achieve performance gains of about 5 %. Secondly, the performance of reactive flow solvers is investigated with Score-P/Vampir and load imbalances due to the computation of the chemical reaction rates are identified. A dynamic-adaptive load balancing approach has been implemented for OpenFOAM’s reacting flow solvers which can decrease computation times by 40 % and increases the utilization of the HPC hardware. This load balancing approach utilizes the special feature of the reaction rate computation, that no information of neighboring cells are required, allowing to implement the load balancing efficiently.