Fast Loads and Queries

  • Goetz Graefe
  • Harumi Kuno
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6380)


For efficient query processing, a relational table should be indexed in multiple ways; for efficient database loading, indexes should be omitted. This research introduces new techniques called zones filters, zone indexes, adaptive merging, and partition filters. The new data structures can be created as side effects of the load process, with all required analyses accomplished while a moderate amount of new data still remains in the buffer pool. Traditional sorting and indexing are not required. Nonetheless, query performance matches that of Netezza’s zone maps where those apply, exceeds it for the many predicates for which zone maps are ineffective, and can be comparable to query processing with traditional indexing, as demonstrated in our simulations.


Query Processing Range Query Query Execution Query Performance Load Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ailamaki, A., De Witt, D.J., Hill, M.D.: Marios Sk- ounakis: Weaving relations for cache performance. In: VLDB 2001, pp. 169–180 (2001)Google Scholar
  2. Bloom, B.H.: Space time trade-offs in hash coding with allowable errors. ACM Commun. 13(7), 422–426 (1970)zbMATHCrossRefGoogle Scholar
  3. Boral, H., DeWitt, D.J.: Database machines: An idea whose time passed? In: A critique of the future of database machines. In: IWDM 1983, pp. 166–187 (1983)Google Scholar
  4. Bayer, R., Unterauer, K.: Prefix B-trees. ACM TODS 2(1), 11–26 (1977)CrossRefGoogle Scholar
  5. Cao, Y., Das, G.C., Chan, C.Y., Tan, K.-L.: Optimizing complex queries with multiple relation instances. In: SIGMOD 2008, pp. 525–538 (2008)Google Scholar
  6. Fernandez, P.M.: Red Brick Warehouse: A read-mostly RDBMS for Open SMP platforms. In: SIGMOD 1994, p. 492 (1994)Google Scholar
  7. Graefe, G.: Sorting and indexing with partitioned B-trees. In: CIDR 2003 (2003)Google Scholar
  8. Graefe, G.: Hierarchical locking in B-tree indexes. In: BTW 2007, pp. 18–42 (2007)Google Scholar
  9. Graefe, G.: The five-minute rule twenty years later, and how flash memory changes the rules. In: DaMoN 2007, p. 6 (2007)Google Scholar
  10. Graefe, G.: Integrating PAX and NSM page formats. Hewlett-Packard Laboratories (2008) (unpublished manuscript)Google Scholar
  11. Graefe, G., Kuno, H.: Self-selecting, self-tuning, incrementally optimized indexes. To appear in EDBT (2010)Google Scholar
  12. Graefe, G., Kuno, H.: Two adaptive indexing techniques: improvements and performance evaluation (submitted)Google Scholar
  13. Graefe, G., Kuno, H.: Adaptive indexing for relational keys. To appear in SMDB (2010)Google Scholar
  14. Grtner, A., Kemper, A., Kossmann, D., Zeller, B.: Efficient bulk deletes in relational databases. In: ICDE 2001, pp. 183–192 (2001)Google Scholar
  15. Graefe, G., Larson, P.-k.: B-Tree indexes and CPU caches. In: ICDE 2001, pp. 349–358 (2001)Google Scholar
  16. Idreos, S., Kersten, M.L., Manegold, S.: Database cracking. In: CIDR 2007, pp. 68–78 (2007)Google Scholar
  17. Idreos, S., Kersten, M.L., Manegold, S.: Updating a cracked database. In: SIGMOD 2007, pp. 413–424 (2007)Google Scholar
  18. Idreos, S., Kersten, M., Manegold, S.: Self-organizing tu-ple reconstruction in column stores. In: SIGMOD 2009, pp. 297–308 (2009)Google Scholar
  19. Idreos, S., Manegold, S., Graefe, G., Kuno, H.: Adaptive indexing. Submitted for publication (2010)Google Scholar
  20. Jermaine, C., Datta, A., Omiecinski, E.: A novel index supporting high volume data warehouse insertion. In: VLDB 1999, pp. 235–246 (1999)Google Scholar
  21. Kersten, M.L., Manegold, S.: Cracking the database store. In: CIDR 2005 (2005)Google Scholar
  22. Lomet, D.B.: The evolution of effective B-tree page organization and techniques: a personal account. SIGMOD Record 30(3), 64–69 (2001)CrossRefGoogle Scholar
  23. Lang, C.A., Bhattacharjee, B., Malkemus, T., Wong, K.: Increasing buffer-locality for multiple index based scans through intelligent placement and index scan speed control. In: VLDB 2007, pp. 1298–1309 (2007)Google Scholar
  24. Lehman, T.J., Carey, M.J.: A study of index structures for main memory database management systems. In: VLDB 1986, pp. 294–303 (1986)Google Scholar
  25. Leslie, H., Jain, R., Birdsall, D., Yaghmai, H.: Efficient search of multi-dimensional B-trees. In: VLDB 1995, pp. 710–719 (1995)Google Scholar
  26. Monash, C.: Kognito WX2 overview (January 2008),
  27. Moerkotte, G.: Small materialized aggregates: A light weight index structure for data warehousing. In: VLDB 1998, pp. 476–487 (1998)Google Scholar
  28. Merrett, T.H., Kambayashi, Y., Yasuura, H.: Scheduling of Page- Fetches in Join Operations. In: VLDB 1981, pp.488–498 (1981)Google Scholar
  29. Muth, P., O’Neil, P., Pick, A., Weikum, G.: The LHAM log-structured history data access method. VLDB J. 8(3-4), 199–221 (2000)CrossRefGoogle Scholar
  30. Murphy, M.C., Rotem, D.: Multiprocessor Join Scheduling. IEEE TKDE 5(2), 322–338 (1993)Google Scholar
  31. Nyberg, C., Barclay, T., Cvetanovic, Z., Gray, J., Lomet, D.B.: AlphaSort: A Cache-Sensitive Parallel External Sort. VLDB J. 4(4), 603–627 (1995)CrossRefGoogle Scholar
  32. Rao, J., Ross, K.A.: Making B+-trees cache conscious in main memory. In: SIGMOD 2000, pp. 475–486 (2000)Google Scholar
  33. Slezak, D., Wroblewski, J., Eastwood, V., Synak, P.: Brighthouse: an analytic data warehouse for ad-hoc queries. PVLDB 1(2), 1337–1345 (2008)Google Scholar
  34. Valduriez, P.: Join Indices. ACM TODS 12(2), 218–246 (1987)CrossRefGoogle Scholar
  35. Zukowski, M., Hman, S., Nes, N., Boncz, P.A.: Cooperative scans: Dynamic bandwidth sharing in a DBMS. In: VLDB 2007, pp. 723–734 (2007)Google Scholar
  36. Zhou, J., Larson, P.-k., Freytag, J.C.: Wolfgang Lehner: Efficient exploitation of similar subexpressions for query processing. In: SIGMOD 2007, pp. 533–544 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Goetz Graefe
    • 1
  • Harumi Kuno
    • 1
  1. 1.HP LabsPalo AltoUSA

Personalised recommendations