Improved and optimized partitioning techniques in database query processing
In this paper we present two improvements to the partitioning process: 1) A new dynamic buffer management strategy is employed to increase the average block size of I/O-transfers to temporary files, and 2) An optimal switching between three different variants of the partitioning methods that ensures minimal partitioning cost. The expected performance gain resulting from the new management strategy is about 30% for a reasonable resource configuration. The performance gain decreases with increasing available buffer space. The different partitioning strategies (partial partitioning or hybrid hashing, one pass partitioning, and multipass partitioning) are analyzed, and we present the optimal working range for these, as a function of operand volume and available memory.
KeywordsRelational algebra partitioning methods buffer management query processing
Unable to display preview. Download preview PDF.
- 1.K. Bratbergsengen. Hashing Methods and Relational Algebra Operations. In Proceedings of the 10th International Conference on VLDB, 1984.Google Scholar
- 2.K. Bratbergsengen, R. Larsen, O. Risnes, and T. Aandalen. A Neighbor Connected Processor Network for Performing Relational Algebra Operations. In Fifth Workshop on Computer Architecture for Non-Numeric Processing, March 11–14, 1980 (SIGIR Vol. XV No. 2, SIGMOD Vol.X No. 4), 1980.Google Scholar
- 3.D. L. Davidson and G. Graefe. Memory-Contention Responsive Hash Joins. In Proceedings of the 20th International Conference on VLDB, 1994.Google Scholar
- 4.D. DeWitt, R. Katz, F. Olken, L. Shapiro, M. Stonebraker, and D. Wood. Implementation Techniques for Main Memory Database Systems. In Proc. ACM SIGMOD Conf., 1984.Google Scholar
- 5.G. Graefe. Query Evaluation Techniques for Large Databases. ACM Computing Surveys, 25(2), 1993.Google Scholar
- 6.G. Graefe. Volcano — An Extensible and Parallel Query Evaluation System. IEEE Transactions on Knowledge and Data Engineering, 6(1), 1994.Google Scholar
- 7.M. Kitsuregawa, H. Tanaka, and T. Motooka. Application of Hash to Data Base Machine and its Architecture. New Generation Computing, 1(1), 1983.Google Scholar
- 8.D. Knuth. The Art of Computer Programming. Sorting and Searching. Addison-Wesley Publishing Company Inc., 1973.Google Scholar
- 9.M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-Partitioned Join Method Using Dynamic Destaging Strategy. In Proceedings of the 14th International Conference on VLDB, 1988.Google Scholar
- 10.L. D. Shapiro. Join Processing in Database Systems with Large Main Memories. ACM Transactions on Database Systems, 11(3), 1986.Google Scholar
- 11.H. Zeller and J. Gray. An Adaptive Hash Join Algorithm for Multiuser Environments. In Proceedings of the 16th International Conference on VLDB, 1990.Google Scholar