Evaluating Contexts in OpenSHMEM-X Reference Implementation

  • Aurelien Bouteiller
  • Swaroop PophaleEmail author
  • Swen Boehm
  • Matthew B. Baker
  • Manjunath Gorentla Venkata
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10679)


Many-core processors are now ubiquitous in supercomputing. This evolution pushes toward the adoption of mixed models in which cores are exploited with threading models (and related programming abstractions, such as OpenMP), while communication between distributed memory domains employ a communication Application Programming Interface (API). OpenSHMEM is a partitioned global address space communication specification that exposes one-sided and synchronization operations. As the threaded semantics of OpenSHMEM are being fleshed out by its standardization committee, it is important to assess the soundness of the proposed concepts. This paper implements and evaluate the “context” extension in relation to threaded operations. We discuss the implementation challenges of the context and the associated API in OpenSHMEM-X. We then evaluate its performance in threaded situations on the Infiniband network using micro-benchmarks and the Random Access benchmark and see that adding communication contexts significantly improves message rate achievable by the executing multi-threaded PEs.


PGAS Shared memory Interoperability 



This work is supported by the United States Department of Defense and used resources of the Extreme Scale Systems Center located at the Oak Ridge National Laboratory.


  1. 1.
    Baker, M., Aderholdt, F., Venkata, M.G., Shamis, P.: OpenSHMEM-UCX: evaluation of UCX for implementing OpenSHMEM programming model. In: Gorentla Venkata, M., Imam, N., Pophale, S., Mintz, T.M. (eds.) OpenSHMEM 2016. LNCS, vol. 10007, pp. 114–130. Springer, Cham (2016). CrossRefGoogle Scholar
  2. 2.
    Baker, M., Pophale, S., Vasnier, J.-C., Jin, H., Hernandez, O.: Hybrid programming using OpenSHMEM and OpenACC. In: Poole, S., Hernandez, O., Shamis, P. (eds.) OpenSHMEM 2014. LNCS, vol. 8356, pp. 74–89. Springer, Cham (2014). CrossRefGoogle Scholar
  3. 3.
    Bonachea, D.: Gasnet specification, v1.1. Technical report, Berkeley, CA, USA (2002)Google Scholar
  4. 4.
    ten Bruggencate, M., Roweth, D., Oyanagi, S.: Thread-safe SHMEM extensions. In: Poole, S., Hernandez, O., Shamis, P. (eds.) OpenSHMEM 2014. LNCS, vol. 8356, pp. 178–185. Springer, Cham (2014). CrossRefGoogle Scholar
  5. 5.
    Chapman, B., Curtis, T., Pophale, S., Poole, S., Kuehn, J., Koelbel, C., Smith, L.: Introducing OpenSHMEM: SHMEM for the PGAS community. In: Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model, PGAS 2010, pp. 2:1–2:3. ACM, New York (2010).
  6. 6.
    Dinan, J., Balaji, P., Goodell, D., Miller, D., Snir, M., Thakur, R.: Enabling MPI interoperability through flexible communication endpoints. In: EuroMPI 2013, Madrid, Spain (2013)Google Scholar
  7. 7.
    Dinan, J., Flajslik, M.: Contexts: a mechanism for high throughput communication in OpenSHMEM. In: Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models, PGAS 2014, pp. 10:1–10:9. ACM, New York (2014).
  8. 8.
    Lawry, W., Wilson, C., Maccabe, A.B., Brightwell, R.: COMB: a portable benchmark suite for assessing MPI overlap. In: 2002 IEEE International Conference on Cluster Computing (CLUSTER 2002), 23–26 September 2002, Chicago, IL, USA, pp. 472–475 (2002).
  9. 9.
    Namashivayam, N., Knaak, D., Cernohous, B., Radcliffe, N., Pagel, M.: An evaluation of thread-safe and contexts-domains features in cray SHMEM. In: Gorentla Venkata, M., Imam, N., Pophale, S., Mintz, T.M. (eds.) OpenSHMEM 2016. LNCS, vol. 10007, pp. 163–180. Springer, Cham (2016). CrossRefGoogle Scholar
  10. 10.
    Nieplocha, J., Carpenter, B.: ARMCI: a portable remote memory copy library for distributed array libraries and compiler run-time systems. In: Rolim, J., et al. (eds.) IPPS 1999. LNCS, vol. 1586, pp. 533–546. Springer, Heidelberg (1999). CrossRefGoogle Scholar
  11. 11.
    Shamis, P., Venkata, M.G., Lopez, M.G., Baker, M.B., Hernandez, O., Itigin, Y., Dubman, M., Shainer, G., Graham, R.L., Liss, L., Shahar, Y., Potluri, S., Rossetti, D., Becker, D., Poole, D., Lamb, C., Kumar, S., Stunkel, C., Bosilca, G., Bouteiller, A.: UCX: an open source framework for HPC network APIs and beyond. In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, pp. 40–43, August 2015Google Scholar
  12. 12.
    Sridharan, S., Dinan, J., Kalamkar, D.D.: Enabling efficient multithreaded MPI communication through a library-based implementation of MPI endpoints. In: SC14: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 487–498 (2014)Google Scholar
  13. 13.
    The Ohio State University: OSU micro-benchmarks (2016).
  14. 14.
    Gorentla Venkata, M., Imam, N., Pophale, S., Mintz, T.M. (eds.): OpenSHMEM 2016. LNCS, vol. 10007. Springer, Cham (2016). Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Aurelien Bouteiller
    • 2
  • Swaroop Pophale
    • 1
    Email author
  • Swen Boehm
    • 1
  • Matthew B. Baker
    • 1
  • Manjunath Gorentla Venkata
    • 1
  1. 1.Oak Ridge National Laboratory, Computer Science and Mathematics DivisionOak RidgeUSA
  2. 2.Innovative Computing LaboratoryThe University of TennesseeKnoxvilleUSA

Personalised recommendations