OpenMP Code Offloading: Splitting GPU Kernels, Pipelining Communication and Computation, and Selecting Better Grid Geometries

Abstract

This paper presents three ideas that focus on improving the execution of high-level parallel code in GPUs. The first addresses programs that include multiple parallel blocks within a single region of GPU code. A proposed compiler transformation can split such regions into multiple, leading to the launching of multiple kernels, one for each parallel region. Advantages include the opportunity to tailor grid geometry of each kernel to the parallel region that it executes and the elimination of the overheads imposed by a code-generation scheme meant to handle multiple nested parallel regions. Second, is a code transformation that sets up a pipeline of kernel execution and asynchronous data transfer. This transformation enables the overlap of communication and computation. Intricate technical details that are required for this transformation are described. The third idea is that the selection of a grid geometry for the execution of a parallel region must balance the GPU occupancy with the potential saturation of the memory throughput in the GPU. Adding this additional parameter to the geometry selection heuristic can often yield better performance at lower occupancy levels.

A Artifact Description Appendix: OpenMP Target Offloading: Splitting GPU Kernels, Pipelining Communication and Computation, and Selecting Better Grid Geometries

1.1 A.1 Abstract

This artifact contains the code for our experimental evaluations of the kernel splitting and kernel pipelining methods with instructions to run the benchmark versions to replicate all experimental results from Sect. 7.

1.2 A.2 Description

Check-List

• Program: C code, Python3 code

• Compilation: Prototype of Clang-YKT compiler used

• Transformations: Kernel-Splitting, Kernel-Pipelining

• Hardware: Intel i7-4770 with Nvidia Titan X Pascal, IBM POWER8 (8335-GTB) with Nvidia P100 GPU

• Software: x86: Ubuntu 18.04 LTS, Cuda V9.1.85; POWER: RHEL Server 7.3, Cuda V9.2.88

• Experiment workflow: Install Clang-YKT prototype then run the provided benchmarks with the given script

• Publicly available?: Yes

How Software Can Be Obtained. Our prototype of the Clang-YKT compiler is available on Github, with our benchmark versions used for all experiments included.

The original Clang-YKT compiler can be found at:

https://github.com/clang-ykt/clang

With the commit hash: 49d8020e03f898ea31212f6c565001e067f67d4f

Hardware Dependencies. An Intel i7-4770 machine with an Nvidia Titan X Pascal GPU was used for almost all experimentation and for similar results an equivalent machine must be utilized. This is especially true for the experiments on occupancy as our optimal occupancies are tied to the Nvidia Titan X Pascal GPU. An additional IBM POWER8 (8335-GTB) host with an Nvidia P100 GPU was used with kernel-pipelining for experimenting with different page sizes and to replicate those results a similar machine must be utilized.

Software Dependencies. A prototype of the Clang-YKT compiler from Github was utilized for compilation of OpenMP code though any compiler that supports OpenMP 4 can be used to run the kernel-splitting and kernel-pipelining benchmark versions.

Datasets. Experiments for each benchmark require only inputting the given tripcount desired for each, with only SRAD requiring an additional pgm image.

1.3 A.3 Installation

Clone the Clang-YKT prototype repository (includes all testing files):

$ git clone https://github.com/uasys/openmp-split

Then install the compiler with the following commands:

$ mkdir -p $build

# 60 stands for GPU compute capability

$ cmake DCMAKE_BUILD_TYPE=RELEASE DCMAKE_INSTALL_PREFIX=$CLANGYKT_DIR DLLVM_ENABLE_BACKTRACES=ON DLLVM_ENABLE_WERROR=OFF DBUILD_SHARED_LIBS=OFF DLLVM_ENABLE_RTTI=ON DOPENMP_ENABLE_LIBOMPTARGET=ON DCMAKE_C_FLAGS=’- DOPENMP_NVPTX_COMPUTE_CAPABILITY=60’ DCMAKE_CXX_FLAGS=’DOPENMP_NVPTX_COMPUTE_CAPABILITY=60’ DLIBOMPTARGET_NVPTX_COMPUTE_CAPABILITY=60 DCLANG_OPENMP_NVPTX_DEFAULT_ARCH=sm_60 DLIBOMPTARGET_NVPTX_ENABLE_BCLIB=true -G Ninja $LLVM_BASE

$ ninja -j4; ninja install

After installation the GPU clock rate must be locked at 80% of the nominal clock rate of the GPU to prevent any variation in performance due to frequency scaling when performing the experiments.

To lock the Nvidia Titan X Pascal input:

nvidia-smi -pm 1

nvidia-smi -application-clocks=4513,1240

1.4 A.4 Experiment Workflow

Experimentation is performed by executing the runTest.py file in the given transformations folder with the chosen benchmark’s name and the tripcount to run it at. Benchmarks for kernel-splitting and those for kernel-pipelining are held in separate folders.

1.5 A.5 Evaluation and Expected Results

The script above will produce a printout once all runs are complete that contains the average run time of each version with the percentage variance and the speedup ratio relative to the baseline.

1.6 A.6 Experiment Customization

Adjusting grid geometry can be done by editing the BLOCKS macro values in the benchmark files with a postfix including G which indicate versions with custom grid geometry.

1.7 A.7 Notes

None.