Exploiting Hybrid Parallelism in the LBM Implementation Musubi on Hawk

Abstract

In this contribution we look into the efficiency and scalability of our Lattice Boltzmann implementation Musubi when using OpenMP threads within an MPI parallel computation on Hawk. The Lattice Boltzmann method enables explicit computation of incompressible flows and the mesh discretization can be automatically generated, even for complex geometries. The basic Lattice Boltzmann kernel is fairly simple and involves only few floating point operations for each lattice node. A simple loop over all lattice nodes in each partition of the MPI parallel setup lends to a straight forward loop parallelization with OpenMP. With increased core counts per compute node, the use of threads on the shared memory nodes is gaining importance, as it avoids overly small partitions with many outbound communications to neighboring partitions. We briefly discuss the hybrid parallelization of Musubi and investigate how the usage of OpenMP threads affects the performance when running simulations on the Hawk supercomputer at HLRS.