Abstract
Driven by the two main hardware trends increasing main memory and massively parallel multi-core processing in the past few years, there has been much research effort in parallelizing well-known join algorithms. However, the non-uniform memory access (NUMA) of these architectures to main memory has only gained limited attention in the design of these algorithms. We study recent proposals of main memory hash join implementations and identify their major performance problems on NUMA architectures. We then develop a NUMA-aware hash join for massively parallel environments, and show how the specific implementation details affect the performance on a NUMA system. Our experimental evaluation shows that a carefully engineered hash join implementation outperforms previous high performance hash joins by a factor of more than two, resulting in an unprecedented throughput of 3/4 billion join argument quintuples per second.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Throughout the paper we refer to \(M\) as \(2^{20}\) and to the overall performance as \((|R| + |S|) {/} runtime\).
- 2.
hashFunction sets the most significant bit of the hash value to 1 and thus ensures no hash value equals 0. This limits the hash domain to \(2^{63}\), but does not increase the number of collisions, since the least significant bits determine the hash table position.
- 3.
When a page is first written to, it is assigned to the memory node of the writing thread, which usually results in pseudo-random assignment.
References
Blanas, S., Li, Y., Patel, J.M.: Design and evaluation of main memory hash join algorithms for multi-core CPUs. In: SIGMOD (2011)
Kim, C., Sedlar, E., Chhugani, J., Kaldewey, T., Nguyen, A.D., Blas, A.D., Lee, V.W., Satish, N., Dubey, P.: Sort vs. hash revisited: fast join implementation on modern multi-core CPUs. PVLDB 2, 1378–1389 (2009)
Albutiu, M.C., Kemper, A., Neumann, T.: Massively parallel sort-merge joins in main memory multi-core database systems. PVLDB 5, 1064–1075 (2012)
Balkesen, C., Teubner, J., Alonso, G., Özsu, T.: Main-memory hash joins on multi-core CPUs: tuning to the underlying hardware. In: ICDE (2013)
Balkesen, C., Teubner, J., Alonso, G., Özsu, T.: Source code. (http://www.systems.ethz.ch/sites/default/files/multicore-hashjoins-0_1_tar.gz)
Manegold, S., Boncz, P.A., Kersten, M.L.: Optimizing main-memory join on modern hardware. IEEE Trans. Knowl. Data Eng. 14, 709–730 (2002)
Boncz, P.A., Manegold, S., Kersten, M.L.: Database architecture optimized for the new bottleneck: memory access. In: VLDB (1999)
Chen, S., Ailamaki, A., Gibbons, P.B., Mowry, T.C.: Improving hash join performance through prefetching. ACM Trans. Database Syst. 32 (2007)
Blanas, S., Patel, J.M.: How efficient is our radix join implementation? (2011). http://pages.cs.wisc.edu/sblanas/files/comparison.pdf
Li, Y., Pandis, I., Mueller, R., Raman, V., Lohman, G.: NUMA-aware algorithms: the case of data shuffling. In: CIDR (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Lang, H., Leis, V., Albutiu, MC., Neumann, T., Kemper, A. (2015). Massively Parallel NUMA-Aware Hash Joins. In: Jagatheesan, A., Levandoski, J., Neumann, T., Pavlo, A. (eds) In Memory Data Management and Analysis. IMDM IMDM 2013 2014. Lecture Notes in Computer Science(), vol 8921. Springer, Cham. https://doi.org/10.1007/978-3-319-13960-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-13960-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13959-3
Online ISBN: 978-3-319-13960-9
eBook Packages: Computer ScienceComputer Science (R0)