Skip to main content
View expanded cover

Workshop on OpenSHMEM and Related Technologies

OpenSHMEM 2018: OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity pp 70–89Cite as

The OpenFAM API: A Programming Model for Disaggregated Persistent Memory

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 11283)

Abstract

Recent technology advances in high-density, byte-addressable non-volatile memory (NVM) and low-latency interconnects have enabled building large-scale systems with a large disaggregated fabric-attached memory (FAM) pool shared across decentralized compute nodes. The OpenFAM API is an API for programming with persistent FAM that is inspired by the OpenSHMEM partitioned global address space (PGAS) model. Unlike OpenSHMEM, where each node contributes local memory toward a logically shared global address space, FAM isn’t associated with a particular node and can be addressed directly from any node without the cooperation or involvement of another node. The OpenFAM API enables programmers to manage FAM allocations, access FAM-resident data structures, and order FAM operations. Because state in FAM can survive program termination, the API also provides interfaces for naming and managing data beyond the lifetime of a single program invocation.

Keywords

  • Disaggregated memory
  • Fabric-attached memory
  • Persistent memory
  • Non-volatile memory
  • Gen-Z

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-04918-8_5
  • Chapter length: 20 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-04918-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.

References

  1. Asanovic, K.: FireBox: a hardware building block for 2020 warehouse-scale computers. Keynote at USENIX Conference on File and Storage Technologies (FAST) 2014

    Google Scholar 

  2. Binnig, C., Crotty, A., Galakatos, A., Kraska, T., Zamanian, E.: The end of slow networks: it’s time for a redesign. Proc. VLDB Endow. 9(7), 528–539 (2016)

    CrossRef  Google Scholar 

  3. Cavium, ThunderX2 ARM Processors. https://www.cavium.com/product-thunderx2-arm-processors.html. Accessed 22 July 2018

  4. Cray: Cray XC Series GNI and DMAPP API User Guide (CLE 6.0.UP05) S-2446. https://pubs.cray.com/pdf-attachments/attachment?pubId=00478935-DA&attachmentId=pub_00478935-DA.pdf. Accessed 28 July 2018

  5. Dulloor, S.R., et al.: Data tiering in heterogeneous memory systems. In: Proceedings of 11th ACM European Conference on Computer Systems (EuroSys) (2016)

    Google Scholar 

  6. Faraboschi, P., Keeton, K., Marsland, T., Milojicic, D.: Beyond processor-centric operating systems. In: Proceedings of Workshop on Hot Topics in Operating Systems (HotOS) (2015)

    Google Scholar 

  7. Funk, M.: Drilling down into the machine from HPE. The Next Platform, 4 January 2016. https://www.nextplatform.com/2016/01/04/drilling-down-into-the-machine-from-hpe/. Accessed 22 July 2018

  8. Funk, M.: Drilling into the CCIX coherence standard. The Next Platform, 13 July 2016. https://www.nextplatform.com/2016/01/04/drilling-down-into-the-machine-from-hpe/. Accessed 22 July 2018

  9. Gen-Z Consortium: Gen-Z Atomics, October 2017. https://genzconsortium.org/wp-content/uploads/2018/05/1711_Gen-Z-Atomics.pdf. Accessed 25 July 2018

  10. Gen-Z Consortium: Gen-Z Coherency, October 2017. http://genzconsortium.org/wp-content/uploads/2017/08/Gen-Z-Coherency.pdf. Accessed 25 July 2018

  11. Gen-Z Consortium: Gen-Z Core Specification. https://genzconsortium.org/specification/core-specification-1-0/. Accessed 22 July 2018

  12. Gen-Z Consortium: Gen-Z ZMMU and Memory Interleave, June 2018. https://genzconsortium.org/wp-content/uploads/2018/06/Gen-Z-MMU-and-Memory-Interleave-1.pdf. Accessed 25 July 2018

  13. Hewlett Packard Enterprise: HPE Persistent Memory: The Performance of Memory with the Persistence of Storage. https://www.hpe.com/us/en/servers/persistent-memory.html. Accessed 25 July 2018

  14. Hewlett Packard Enterprise: New HPE Pointnext Capabilities Accelerate Transition to Memory-Driven Computing. https://news.hpe.com/new-hpe-pointnext-capabilities-accelerate-transition-to-memory-driven-computing/. Accessed 22 July 2018

  15. Huawei: High Throughput Computing Data Center Architecture: Thinking of Data Center 3.0. http://www.huawei.com/ilink/en/download/HW_349607. Accessed 25 July 2018

  16. Intel: Facebook Collaborate on Future Data Center Rack Technologies, Intel Newsroom, 16 January 2013. http://goo.gl/6h2Ut. Accessed 25 July 2018

  17. Intel: Intel Omni-Path Architecture (Intel OPA) Driving Exascale Computing and HPC. http://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-architecture-fabric-overview.html. Accessed 25 July 2018

  18. Intel Rack Scale Design: Data Center Agility at Scale. http://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html. Accessed 25 July 2018

  19. High Bandwidth Memory (HBM) DRAM, JEDEC Standard JESD235A, November 2015. http://www.jedec.org/standards-documents/results/jesd235. Accessed 25 July 2018

  20. Keeton, K.: Memory-driven computing. Keynote at 33rd International Conference on Massive Storage Systems and Technology (MSST), May 2017. http://storageconference.us/2017/Presentations/Keeton.pdf

  21. Kerrisk, M.: Linux/UNIX System Programming: POSIX Shared Memory, February 2015. http://man7.org/training/download/posix_shm_slides.pdf. Accessed 30 July 2018

  22. Lim, K., et al.: System-level implications of disaggregated memory. In: Proceedings of International Symposium on High Performance Computer Architecture (HPCA) (2012)

    Google Scholar 

  23. Lockwood, G.: What’s so bad about POSIX I/O?. The Next Platform, 11 September 2017. https://www.nextplatform.com/2017/09/11/whats-bad-posix-io/. Accessed 29 July 2018

  24. Loesing, S., Pilman, M., Etter, T., Kossmann, D.: On the design and scalability of distributed shared-data databases. In: Proceedings of 15th ACM SIGMOD International Conference on Management of Data (SIGMOD), pp. 663–676 (2015)

    Google Scholar 

  25. Novakovic, S., Daglis, A., Bugnion, E., Falsafi, B., Grot, B.: Scale-out NUMA. In: Proceedings of Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (2014)

    Google Scholar 

  26. OpenCAPI Overview, 14 October 2016. http://opencapi.org/wp-content/uploads/2016/09/OpenCAPI-Overview.10.14.16.pdf. Accessed 25 July 2018

  27. OpenSHMEM API Version 1.4. http://openshmem.org/site/sites/default/site_files/OpenSHMEM-1.4.pdf. Accessed 25 July 2018

  28. The OpenFAM API. https://github.com/OpenFAM/API. Accessed 27 June 2018

  29. pmem.io: Persistent Memory Programming. http://pmem.io. Accessed 29 July 2018

  30. Qureshi, M.K., Srinivasan, V., Rivers, J.A.: Scalable high performance main memory system using phase-change memory technology. In: Proceedings of 36th International Symposium on Computer Architecture (ISCA), pp. 24–33 (2009)

    Google Scholar 

  31. RDMA Consortium: http://www.rdmaconsortium.org. Accessed 25 July 2018

  32. Seyedi, M.A., Fiorentino, M.: Silicon photonics; ring modulator transmitters. In: Guenther, R., Steel, D. (eds.) Encyclopedia of Modern Optics, 2nd edn, vol. 4, pp. 216–223 (2018)

    Google Scholar 

  33. Shah, H., Marti, F., Noureddine, W., Eiriksson, A., Sharp, R.: Remote Direct Memory Access (RDMA) Protocol Extensions, IETF RFC 7306, June 2014. https://tools.ietf.org/html/rfc7306. Accessed 25 July 2018

  34. Spelman, L.: Reimagining the Data Center Memory and Storage Hierarchy, Intel Newsroom, 30 May 2018. https://newsroom.intel.com/editorials/re-architecting-data-center-memory-storage-hierarchy/. Accessed 25 July 2018

  35. Storage Networking Industry Association (SNIA): NVM Programming Model (NPM), Version 1.2. https://www.snia.org/sites/default/files/technical_work/final/NVMProgrammingModel_v1.2.pdf. Accessed 29 July 2018

  36. Strukov, D.B., Snider, G.S., Steward, D.R., Williams, R.S.: The missing memristor found. Nature 453, 80–83 (2008)

    CrossRef  Google Scholar 

  37. Xie, Y.: Modeling, architecture and applications for emerging memory technologies. IEEE Design and Test of Computers 28(1), 44–51 (2011)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hewlett Packard Enterprise, Palo Alto, CA, 94304, USA

    Kimberly Keeton, Sharad Singhal & Michael Raymond

Authors
  1. Kimberly Keeton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sharad Singhal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Raymond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimberly Keeton .

Editor information

Editors and Affiliations

  1. Oak Ridge National Laboratory, Oak Ridge, TN, USA

    Swaroop Pophale

  2. Oak Ridge National Laboratory, Oak Ridge, TN, USA

    Neena Imam

  3. Oak Ridge National Laboratory, Oak Ridge, TN, USA

    Ferrol Aderholdt

  4. Oak Ridge National Laboratory, Oak Ridge, TN, USA

    Manjunath Gorentla Venkata

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Keeton, K., Singhal, S., Raymond, M. (2019). The OpenFAM API: A Programming Model for Disaggregated Persistent Memory. In: Pophale, S., Imam, N., Aderholdt, F., Gorentla Venkata, M. (eds) OpenSHMEM and Related Technologies. OpenSHMEM in the Era of Extreme Heterogeneity. OpenSHMEM 2018. Lecture Notes in Computer Science(), vol 11283. Springer, Cham. https://doi.org/10.1007/978-3-030-04918-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04918-8_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04917-1

  • Online ISBN: 978-3-030-04918-8

  • eBook Packages: Computer ScienceComputer Science (R0)