On-the-Fly Calculation of Model Factors for Multi-paradigm Applications

Abstract

Model factors provide initial insight into fundamental issues of parallel applications. These metrics elaborate beyond conventional HPC metrics to indicate whether an application suffers from systemic or local load imbalances, or high cost for synchronization or data transfer. The metrics are also useful to compare the parallel characteristics of different versions of the same application. This work proposes a model of separating the impact factors of different layered parallelism paradigms. In contrast to previous work in this area, we successfully separate all factors and can prove all efficiency values to be between 0 and 1. While we use MPI + OpenMP as an example in this work, the general concepts also apply to layering other parallel programming paradigms. As a proof of concept, we present a tool that collects the necessary performance data and determines different critical paths in the execution without measurable runtime overhead. We evaluate the methodology with synthetic source code examples but also with a real-world application and an application using the latest or future MPI and OpenMP features in order to evaluate the methodology on applications with an overlap of asynchronous computation and communication at the highest possible concurrency.