OpenSHMEM and Related Technologies. Experiences, Implementations, and Tools

Volume 8356 of the series Lecture Notes in Computer Science pp 163-177

Reducing Synchronization Overhead Through Bundled Communication

  • James DinanAffiliated withIntel Corp.
  • , Clement ColeAffiliated withIntel Corp.
  • , Gabriele JostAffiliated withIntel Corp.
  • , Stan SmithAffiliated withIntel Corp.
  • , Keith UnderwoodAffiliated withIntel Corp.
  • , Robert W. WisniewskiAffiliated withIntel Corp.

* Final gross prices may vary according to local VAT.

Get Access


OpenSHMEM provides a one-sided communication interface that allows for asynchronous, one-sided communication operations on data stored in a partitioned global address space. While communication in this model is efficient, synchronizations must currently be achieved through collective barriers or one-sided updates of sentinel locations in the global address space. These synchronization mechanisms can over-synchronize, or require additional communication operations, respectively, leading to high overheads. We propose a SHMEM extension that utilizes capabilities present in most high performance interconnects (e.g. communication events) to bundle synchronization information together with communication operations. Using this approach, we improve ping-pong latency for small messages by a factor of two, and demonstrate significant improvement to synchronization-heavy communication patterns, including all-to-all and pipelined parallel stencil communication.