Efficient Main-Memory Algorithms for Set Containment Join Using Inverted Lists

  • Dmitry Shaporenkov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3631)

Abstract

We present two algorithms for set containment joins based on inverted lists. The first algorithm scans the left relation and determines for each tuple all the qualifying tuples by querying the inverted file for the right relation. The second algorithm employs the common inverted file for both relations. We focus on improving performance of algorithms in main memory by reducing number of L2 cache misses which is achieved by applying such techniques as partitioning and compression. We study algorithms analytically and experimentally and determine which one is better depending on parameters of the input relations. We also demonstrate that both algorithms are superior to some other known methods for set containment joins.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Boncz, P.A., Manegold, S., Kersten, M.L.: Database Architecture Optimized for the New Bottleneck: Memory Access. In: Proceedings of the 25th VLDB Conference, pp. 54–65 (1999)Google Scholar
  2. 2.
    Helmer, S., Moerkotte, G.: Evaluation of main memory join algorithms for joins with set comparison join predicates. In: Proceedings of the 23rd VLDB Conference, pp. 386–395 (1997)Google Scholar
  3. 3.
    Helmer, S., Moerkotte, G.: Compiling away set containment and intersection joins, (technical report) (2002)Google Scholar
  4. 4.
    Mamoulis, N.: Efficient processing of joins on set-valued attributes. In: Proceedings of the SIGMOD 2003 Conference, pp. 157–168 (2003)Google Scholar
  5. 5.
    Manegold, S.: The Calibrator, a Cache-Memory and TLB Calibration Tool, http://homepages.cwi.nl/~manegold/Calibrator/
  6. 6.
    Manegold, S., Boncz, P., Nes, N., Kersten, M.: Cache-conscious radix-decluster projections. In: Proceeding of the SIGMOD 2004 Conference (2004)Google Scholar
  7. 7.
    Melnik, S., Garcia-Molina, H.: Adaptive Algorithms for Set Containment Joins. ACM Transactions on Database Systems 28, 56–99 (2003)CrossRefGoogle Scholar
  8. 8.
    Ramasamy, K., et al.: Set containment joins: The good, the bad and the ugly. In: Proceedings of the 26th VLDB Conference, pp. 351–362 (2000)Google Scholar
  9. 9.
    Rao, J., Ross, K.A.: Making B+-Trees Cache-Conscious in Main Memory. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 475–486 (2000)Google Scholar
  10. 10.
    Shaporenkov, D.: Multi-indices - a tool for optimizing join processing in main memory. In: Proceedings of the Baltic DBIS 2004 Conference (2004)Google Scholar
  11. 11.
    Shaporenkov, D.: Performance comparison of main-memory algorithms for set containment joins. In: Proceedings of the SYRCoDIS 2004 (2004)Google Scholar
  12. 12.
    Shatdal, A., Kant, C., Naughton, J.F.: Cache Conscious Algorithms for Relational Query Processing. In: Proceedings of the 20th VLDB Conference, pp. 510–521 (1994)Google Scholar
  13. 13.
    Valduriez, P.: Join Indices. ACM Transactions on Database Systems 12, 218–246 (1987)CrossRefGoogle Scholar
  14. 14.
    Witten, I., Moffat, A., Bell, T.: Managing Gigabytes: Compressing and Indexing Documents and Images, 2nd edn. Morgan Kaufmann publishers, San Francisco (1999)Google Scholar
  15. 15.
    Zhou, J., Ross, K.A.: Buffering Accesses to Memory-Resident Index Structures. In: Proceedings of the 29th VLDB Conference (2003)Google Scholar
  16. 16.
    C# Language Specification. ECMA-334 International Standard (2001)Google Scholar
  17. 17.
    Intel VTune Performance Analyzer, http://www.intel.com/software/products/vtune

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Dmitry Shaporenkov
    • 1
  1. 1.University of Saint-PetersburgRussia

Personalised recommendations