Abstract
We present two algorithms for set containment joins based on inverted lists. The first algorithm scans the left relation and determines for each tuple all the qualifying tuples by querying the inverted file for the right relation. The second algorithm employs the common inverted file for both relations. We focus on improving performance of algorithms in main memory by reducing number of L2 cache misses which is achieved by applying such techniques as partitioning and compression. We study algorithms analytically and experimentally and determine which one is better depending on parameters of the input relations. We also demonstrate that both algorithms are superior to some other known methods for set containment joins.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Boncz, P.A., Manegold, S., Kersten, M.L.: Database Architecture Optimized for the New Bottleneck: Memory Access. In: Proceedings of the 25th VLDB Conference, pp. 54–65 (1999)
Helmer, S., Moerkotte, G.: Evaluation of main memory join algorithms for joins with set comparison join predicates. In: Proceedings of the 23rd VLDB Conference, pp. 386–395 (1997)
Helmer, S., Moerkotte, G.: Compiling away set containment and intersection joins, (technical report) (2002)
Mamoulis, N.: Efficient processing of joins on set-valued attributes. In: Proceedings of the SIGMOD 2003 Conference, pp. 157–168 (2003)
Manegold, S.: The Calibrator, a Cache-Memory and TLB Calibration Tool, http://homepages.cwi.nl/~manegold/Calibrator/
Manegold, S., Boncz, P., Nes, N., Kersten, M.: Cache-conscious radix-decluster projections. In: Proceeding of the SIGMOD 2004 Conference (2004)
Melnik, S., Garcia-Molina, H.: Adaptive Algorithms for Set Containment Joins. ACM Transactions on Database Systems 28, 56–99 (2003)
Ramasamy, K., et al.: Set containment joins: The good, the bad and the ugly. In: Proceedings of the 26th VLDB Conference, pp. 351–362 (2000)
Rao, J., Ross, K.A.: Making B+-Trees Cache-Conscious in Main Memory. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 475–486 (2000)
Shaporenkov, D.: Multi-indices - a tool for optimizing join processing in main memory. In: Proceedings of the Baltic DBIS 2004 Conference (2004)
Shaporenkov, D.: Performance comparison of main-memory algorithms for set containment joins. In: Proceedings of the SYRCoDIS 2004 (2004)
Shatdal, A., Kant, C., Naughton, J.F.: Cache Conscious Algorithms for Relational Query Processing. In: Proceedings of the 20th VLDB Conference, pp. 510–521 (1994)
Valduriez, P.: Join Indices. ACM Transactions on Database Systems 12, 218–246 (1987)
Witten, I., Moffat, A., Bell, T.: Managing Gigabytes: Compressing and Indexing Documents and Images, 2nd edn. Morgan Kaufmann publishers, San Francisco (1999)
Zhou, J., Ross, K.A.: Buffering Accesses to Memory-Resident Index Structures. In: Proceedings of the 29th VLDB Conference (2003)
C# Language Specification. ECMA-334 International Standard (2001)
Intel VTune Performance Analyzer, http://www.intel.com/software/products/vtune
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shaporenkov, D. (2005). Efficient Main-Memory Algorithms for Set Containment Join Using Inverted Lists. In: Eder, J., Haav, HM., Kalja, A., Penjam, J. (eds) Advances in Databases and Information Systems. ADBIS 2005. Lecture Notes in Computer Science, vol 3631. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11547686_11
Download citation
DOI: https://doi.org/10.1007/11547686_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28585-4
Online ISBN: 978-3-540-31895-8
eBook Packages: Computer ScienceComputer Science (R0)