Compression-Aware In-Memory Query Processing: Vision, System Design and Beyond

Juliana Hildebrandt; Dirk Habich; Patrick Damme; Wolfgang Lehner

doi:10.1007/978-3-319-56111-0_3

International Workshop on In-Memory Data Management and Analytics

International Workshop on Accelerating Data Analysis and Data Management Systems

IMDM 2016, ADMS 2016: Data Management on New Hardware pp 40-56

Compression-Aware In-Memory Query Processing: Vision, System Design and Beyond

Authors
Authors and affiliations

Juliana Hildebrandt
Dirk HabichEmail author
Patrick Damme
Wolfgang Lehner

Conference paper

First Online:: 23 March 2017

DOI: 10.1007/978-3-319-56111-0_3

Part of the Lecture Notes in Computer Science book series (LNCS, volume 10195)

Cite this paper as:: Hildebrandt J., Habich D., Damme P., Lehner W. (2017) Compression-Aware In-Memory Query Processing: Vision, System Design and Beyond. In: Blanas S., Bordawekar R., Lahiri T., Levandoski J., Pavlo A. (eds) Data Management on New Hardware. IMDM 2016, ADMS 2016. Lecture Notes in Computer Science, vol 10195. Springer, Cham

Abstract

In-memory database systems have to keep base data as well as intermediate results generated during query processing in main memory. In addition, the effort to access intermediate results is equivalent to the effort to access the base data. Therefore, the optimization of intermediate results is interesting and has a high impact on the performance of the query execution. For this domain, we propose the continuous use of lightweight compression methods for intermediate results and have the aim of developing a balanced query processing approach based on compressed intermediate results. To minimize the overall query execution time, it is important to find a balance between the reduced transfer times and the increased computational effort. This paper provides an overview and presents a system design for our vision. Our system design addresses the challenge of integrating a large and evolving corpus of lightweight data compression algorithms in an in-memory column store. In detail, we present our model-driven approach and describe ongoing research topics to realize our compression-aware query processing vision.

References

1.
Abadi, D., Boncz, P.A., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5(3), 197–280 (2013)CrossRefGoogle Scholar
2.
Abadi, D.J., Madden, S.R., Ferreira, M.C.: Integrating compression and execution in column-oriented database systems. In: SIGMOD, pp. 671–682 (2006)
3.
Anh, V.N., Moffat, A.: Inverted index compression using word-aligned binary codes. Inf. Retr. 8(1), 151–166 (2005)CrossRefGoogle Scholar
4.
Arroyuelo, D., González, S., Oyarzún, M., Sepulveda, V.: Document identifier reassignment and run-length-compressed inverted indexes for improved search performance. In: SIGIR, pp. 173–182 (2013)
5.
Boncz, P.A., Kersten, M.L., Manegold, S.: Breaking the memory wall in MonetDB. Commun. ACM 51(12), 77–85 (2008)CrossRefGoogle Scholar
6.
Chen, Z., Gehrke, J., Korn, F.: Query optimization in compressed database systems. SIGMOD Rec. 30(2), 271–282 (2001)CrossRefGoogle Scholar
7.
Copeland, G.P., Khoshafian, S.N.: A decomposition storage model. SIGMOD Rec. 14(4), 268–279 (1985)CrossRefGoogle Scholar
8.
Damme, P., Habich, D., Lehner, W.: A benchmark framework for data compression techniques. In: Nambiar, R., Poess, M. (eds.) TPCTC 2015. LNCS, vol. 9508, pp. 77–93. Springer, Cham (2016). doi:10.1007/978-3-319-31409-9_6 CrossRefGoogle Scholar
9.
Damme, P., Habich, D., Lehner, W.: Direct transformation techniques for compressed data: general approach and application scenarios. In: Morzy, T., Valduriez, P., Bellatreche, L. (eds.) ADBIS 2015. LNCS, vol. 9282, pp. 151–165. Springer, Cham (2015). doi:10.1007/978-3-319-23135-8_11 CrossRefGoogle Scholar
10.
Delbru, R., Campinas, S., Samp, K., Tummarello, G., Dangan, L., Delbru, R., Campinas, S., Samp, K., Tummarello, G.: Adaptive frame of reference for compressing inverted lists (2010)
11.
Goldstein, J., Ramakrishnan, R., Shaft, U.: Compressing relations and indexes. In: ICDE, pp. 370–379 (1998)
12.
Habich, D., Richly, S., Lehner, W.: GignoMDA - exploiting cross-layer optimization for complex database applications. In: VLDB (2006)
13.
Iyer, B.R., Wilhite, D.: Data compression support in databases. In: VLDB Conference, pp. 695–704 (1994)
14.
Kissinger, T., Schlegel, B., Habich, D., Lehner, W.: KISS-Tree: smart latch-free in-memory indexing on modern architectures. In: DaMoN, pp. 16–23 (2012)
15.
Kissinger, T., Schlegel, B., Habich, D., Lehner, W.: QPPT: query processing on prefix trees. In: CIDR 2013 (2013)
16.
Kleppe, A., Warmer, J., Bast, W.: MDA Explained. The Model Driven Architecture: Practice and Promise. Addison-Wesley, Massachusetts (2003)Google Scholar
17.
Leis, V., Kemper, A., Neumann, T.: The adaptive radix tree: artful indexing for main-memory databases. In: ICDE, pp. 38–49 (2013)
18.
Lemire, D., Boytsov, L.: Decoding billions of integers per second through vectorization. Softw. Pract. Exper. 45(1), 1–29 (2015)CrossRefGoogle Scholar
19.
Neumann, T.: Efficiently compiling efficient query plans for modern hardware. PVLDB 4(9), 539–550 (2011)Google Scholar
20.
Qiao, L., Raman, V., Reiss, F., Haas, P.J., Lohman, G.M.: Main-memory scan sharing for multi-core cpus. PVLDB 1, 610–621 (2008)Google Scholar
21.
Roth, M.A., Van Horn, S.J.: Database compression. SIGMOD Rec. 22(3), 31–39 (1993)CrossRefGoogle Scholar
22.
Schlegel, B., Gemulla, R., Lehner, W.: Fast integer compression using SIMD instructions. In: DaMoN (2010)
23.
Silvestri, F., Venturini, R.: Vsencoding: efficient coding and fast decoding of integer lists via dynamic programming. In: CIKM, pp. 1219–1228 (2010)
24.
Stepanov, A.A., Gangolli, A.R., Rose, D.E., Ernst, R.J., Oberoi, P.S.: SIMD-based decoding of posting lists. In: CIKM, pp. 317–326 (2011)
25.
Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: SIMD-scan: ultra fast in-memory table scan using on-chip vector processing units. PVLDB 2(1), 385–394 (2009)Google Scholar
26.
Williams, R.: Adaptive Data Compression. Kluwer International Series in Engineering and Computer Science: Communications and Information Theory. Springer, US (1991)CrossRefMATHGoogle Scholar
27.
Zukowski, M., Heman, S., Nes, N., Boncz, P.: Super-scalar RAM-CPU cache compression. In: ICDE, p. 59 (2006)

Copyright information

Authors and Affiliations

Juliana Hildebrandt
- 1
Dirk Habich
- 1
Email author
Patrick Damme
- 1
Wolfgang Lehner
- 1

1.Database Systems GroupTechnische Universität DresdenDresdenGermany

Personalised recommendations

Actions

Buy eBook

USD 44.99

Instant download
Readable on all devices
Own it forever
Local sales tax included if applicable

Compression-Aware In-Memory Query Processing: Vision, System Design and Beyond

Abstract

Copyright information

Authors and Affiliations

Personalised recommendations

Cookies