The VLDB Journal

, Volume 9, Issue 3, pp 231–246

Optimizing database architecture for the new bottleneck: memory access

Authors

  • Stefan Manegold
    • CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands; E-mail: {S.Manegold,M.L.Kersten}@cwi.nl
  • Peter A. Boncz
    • Data Distilleries B.V., Kruislaan 402, 1098 SM Amsterdam, The Netherlands; E-mail: P.Boncz@ddi.nl
  • Martin L. Kersten
    • CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands; E-mail: {S.Manegold,M.L.Kersten}@cwi.nl
Regular contribution

DOI: 10.1007/s007780000031

Cite this article as:
Manegold, S., Boncz, P. & Kersten, M. The VLDB Journal (2000) 9: 231. doi:10.1007/s007780000031

Abstract.

In the past decade, advances in the speed of commodity CPUs have far out-paced advances in memory latency. Main-memory access is therefore increasingly a performance bottleneck for many computer applications, including database systems. In this article, we use a simple scan test to show the severe impact of this bottleneck. The insights gained are translated into guidelines for database architecture, in terms of both data structures and algorithms. We discuss how vertically fragmented data structures optimize cache performance on sequential data access. We then focus on equi-join, typically a random-access operation, and introduce radix algorithms for partitioned hash-join. The performance of these algorithms is quantified using a detailed analytical model that incorporates memory access cost. Experiments that validate this model were performed on the Monet database system. We obtained exact statistics on events such as TLB misses and L1 and L2 cache misses by using hardware performance counters found in modern CPUs. Using our cost model, we show how the carefully tuned memory access pattern of our radix algorithms makes them perform well, which is confirmed by experimental results.

Key words: Main-memory databases – Query processing – Memory access optimization – Decomposed storage model – Join algorithms – Implementation techniques
Download to read the full article text

Copyright information

© Springer-Verlag Berlin Heidelberg 2000