Abstract
In this paper we present a novel disk-based distributed column-store, describe its architecture and discuss a number of technical solutions. Our system is essentially a query engine which was written completely from scratch. It is aimed for shared-nothing environments and supports different forms of parallel query processing.
Query processing in PosDB is organized according to the classic Volcano pull-based model which is adapted for the column-store case. Currently, we support late materialization only, and therefore employ a join index data structure to represent positional information. In our system query plan can consist of both positional and value operators. PosDB has about a dozen of core operators among which several variants of selections and joins, aggregation. We also have several operators that ensure intra-query parallelism and operators for network interoperability. In its current state the system is fully capable of processing the Star Schema Benchmark in a local and distributed environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
O’Neil, P.E., O’Neil, E.J., Chen, X.: The Star Schema Benchmark (SSB) (2009). http://www.cs.umb.edu/~poneil/StarSchemaB.PDF. Accessed 20 July 2012
Google Supersonic Library (2017). https://code.google.com/archive/p/supersonic/. Accessed 12 February 2017
Abadi, D., Boncz, P., Harizopoulos, S.: The Design and Implementation of Modern Column-Oriented Database Systems. Now Publishers Inc., Hanover, massachusetts (2013)
Abadi, D.J., Boncz, P.A., Harizopoulos, S.: Column-oriented database systems. Proc. VLDB Endow. 2(2), 1664–1665 (2009)
Arulraj, J., Pavlo, A., Menon, P.: Bridging the Archipelago between row-stores and column-stores for hybrid workloads. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 583–598 (2016)
Chernishev, G.: Physical design approaches for column-stores. SPIIRAS Proceedings 30, 204–222 (2013)
Chernishev, G.: Towards self-management in a distributed column-store system. In: Morzy, T., Valduriez, P., Bellatreche, L. (eds.) ADBIS 2015. CCIS, vol. 539, pp. 97–107. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23201-0_12
Chernishev, G.: The design of an adaptive column-store system. J. Big Data 4(1), 21 (2017)
Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K.: A study of PosDB performance in a distributed environment. In: Proceedings of the 2017 Software Engineering and Information Management, SEIM 2017 (2017)
Graefe, G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–169 (1993)
Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: Two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 35(1), 40–45 (2012)
Liu, Y., et al.: DCODE: A distributed column-oriented database engine for big data analytics. In: Khalil, I., Neuhold, E., Tjoa, A.M., Da Xu, L., You, I. (eds.) CONFENIS/ICT-EurAsia -2015. LNCS, vol. 9357, pp. 289–299. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24315-3_30
Stonebraker, M., et al.: C-Store: A column-oriented DBMS. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 553–564. VLDB Endowment (2005)
Valduriez, P.: Join indices. ACM Trans. Database Syst. 12(2), 218–246 (1987)
Zhang, Y., Xiao, Y., Wang, Z., Ji, X., Huang, Y., Wang, S.: ScaMMDB: Facing challenge of mass data processing with MMDB. In: Chen, L., et al. (eds.) APWeb/WAIM -2009. LNCS, vol. 5731, pp. 1–12. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03996-6_1
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K. (2018). PosDB: A Distributed Column-Store Engine. In: Petrenko, A., Voronkov, A. (eds) Perspectives of System Informatics. PSI 2017. Lecture Notes in Computer Science(), vol 10742. Springer, Cham. https://doi.org/10.1007/978-3-319-74313-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-74313-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74312-7
Online ISBN: 978-3-319-74313-4
eBook Packages: Computer ScienceComputer Science (R0)