A Distributed Hash Table for Shared Memory

  • Wytse OortwijnEmail author
  • Tom van Dijk
  • Jaco van de Pol
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9574)


Distributed algorithms for graph searching require a high-performance CPU-efficient hash table that supports find-or-put. This operation either inserts data or indicates that it has already been added before. This paper focuses on the design and evaluation of such a hash table, targeting supercomputers. The latency of find-or-put is minimized by using one-sided RDMA operations. These operations are overlapped as much as possible to reduce waiting times for roundtrips. In contrast to existing work, we use linear probing and argue that this requires less roundtrips. The hash table is implemented in UPC. A peak-throughput of 114.9 million op/s is reached on an Infiniband cluster. With a load-factor of 0.9, find-or-put can be performed in \(4.5\,\upmu \mathrm{s}\) on average. The hash table performance remains very high, even under high loads.


Distributed hash table High-performance computing Partitioned global address space Remote direct memory access 


  1. 1.
    Chapman, B., Curtis, T., Pophale, S., Poole, S., Kuehn, J., Koelbel, C., Smith, L.: Introducing OpenSHMEM: SHMEM for the PGAS community. In: Fourth Conference on Partitioned Global Address Space Programming Model. ACM (2010)Google Scholar
  2. 2.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. MIT press, Cambridge (2009)zbMATHGoogle Scholar
  3. 3.
    Dragojevi, A., Narayanan, D., Hodson, O., Castro, M.: FaRM: Fast remote memory. In: 11th USENIX Conference on Networked Systems Design and Implementation, NSDI, vol. 14 (2014)Google Scholar
  4. 4.
    El-Ghazawi, T., Smith, L.: UPC: Unified Parallel C. In: ACM/IEEE Conference on Supercomputing. ACM (2006)Google Scholar
  5. 5.
    Farreras, M., Almasi, G., Cascaval, C., Cortes, T.: Scalable RDMA performance in PGAS languages. In: Parallel and Distributed Processing, pp. 1–12. IEEE (2009)Google Scholar
  6. 6.
    Herlihy, M.P., Shavit, N.N., Tzafrir, M.: Hopscotch hashing. In: Taubenfeld, G. (ed.) DISC 2008. LNCS, vol. 5218, pp. 350–364. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    InfiniBand Trade Association: Accessed 9 May 2015.
  8. 8.
    The Distributed ASCI Supercomputer 5 (2015).
  9. 9.
    Kalia, A., Kaminsky, M., Andersen, D.G.: Using RDMA efficiently for key-value services. In: ACM Conference on SIGCOMM, pp. 295–306. ACM (2014)Google Scholar
  10. 10.
    Mitchell, C., Geng, Y., Li, J.: Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In: USENIX Annual Technical Conference, pp. 103–114 (2013)Google Scholar
  11. 11.
    Pagh, R., Rodler, F.F.: Cuckoo hashing. J. Algorithms 51(2), 122–144 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Laarman, A., van de Pol, J., Weber, M.: Boosting multi-core reachability performance with shared hash tables. In: Conference on Formal Methods in Computer-Aided Design, FMCAD, pp. 247–256 (2010)Google Scholar
  13. 13.
    Ross, K.A.: Efficient hash probes on modern processors. In: IEEE 23rd International Conference on Data Engineering, pp. 1297–1301. IEEE (2007)Google Scholar
  14. 14.
    Rumble, S.M., Ongaro, D., Stutsman, R., Rosenblum, M., Ousterhout, J.K.: Its time for low latency. In: HotOS (2011)Google Scholar
  15. 15.
    Szepesi, T., Wong, B., Cassell, B., Brecht, T.: Designing a low-latency cuckoo hash table for write-intensive workloads using RDMA. In: First International Workshop on Rack-scale Computing (2014)Google Scholar
  16. 16.
    van Dijk, T., van de Pol, J.: Sylvan: Multi-core decision diagrams. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 677–691. Springer, Heidelberg (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Wytse Oortwijn
    • 1
    Email author
  • Tom van Dijk
    • 1
  • Jaco van de Pol
    • 1
  1. 1.Formal Methods and Tools, Department of EEMCSUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations