OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools

Volume 8356 of the series Lecture Notes in Computer Science pp 74-89

Hybrid Programming Using OpenSHMEM and OpenACC

  • Matthew BakerAffiliated withOak Ridge National Laboratory
  • , Swaroop PophaleAffiliated withUniversity of Houston
  • , Jean-Charles VasnierAffiliated withCAPS Entreprise
  • , Haoqiang JinAffiliated withNASA Ames
  • , Oscar HernandezAffiliated withOak Ridge National Laboratory

* Final gross prices may vary according to local VAT.

Get Access


With high performance systems exploiting multicore and accelerator-based architectures on a distributed shared memory system, heterogenous hybrid programming models are the natural choice to exploit all the hardware made available on these systems. Previous efforts looking into hybrid models have primarily focused on using OpenMP directives (for shared memory programming) with MPI (for inter-node programming on a cluster), using OpenMP to spawn threads on a node and communication libraries like MPI to communicate across nodes. As accelerators get added into the mix, and there is better hardware support for PGAS languages/APIs, this means that new and unexplored heterogenous hybrid models will be needed to effectively leverage the new hardware. In this paper we explore the use of OpenACC directives to program GPUs and the use of OpenSHMEM, a PGAS library for onesided communication between nodes. We use the NAS-BT Multi-zone benchmark that was converted to use the OpenSHMEM library API for network communication between nodes and OpenACC to exploit accelerators that are present within a node. We evaluate the performance of the benchmark and discuss our experiences during the development of the OpenSHMEM+OpenACC hybrid program.