Advertisement

PosDB: A Distributed Column-Store Engine

  • George ChernishevEmail author
  • Viacheslav Galaktionov
  • Valentin Grigorev
  • Evgeniy Klyuchikov
  • Kirill Smirnov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10742)

Abstract

In this paper we present a novel disk-based distributed column-store, describe its architecture and discuss a number of technical solutions. Our system is essentially a query engine which was written completely from scratch. It is aimed for shared-nothing environments and supports different forms of parallel query processing.

Query processing in PosDB is organized according to the classic Volcano pull-based model which is adapted for the column-store case. Currently, we support late materialization only, and therefore employ a join index data structure to represent positional information. In our system query plan can consist of both positional and value operators. PosDB has about a dozen of core operators among which several variants of selections and joins, aggregation. We also have several operators that ensure intra-query parallelism and operators for network interoperability. In its current state the system is fully capable of processing the Star Schema Benchmark in a local and distributed environment.

Keywords

Column Stores Star Schema Benchmark Query Plan Shared-nothing Environment Tuple Reconstruction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    O’Neil, P.E., O’Neil, E.J., Chen, X.: The Star Schema Benchmark (SSB) (2009). http://www.cs.umb.edu/~poneil/StarSchemaB.PDF. Accessed 20 July 2012
  2. 2.
    Google Supersonic Library (2017). https://code.google.com/archive/p/supersonic/. Accessed 12 February 2017
  3. 3.
    Abadi, D., Boncz, P., Harizopoulos, S.: The Design and Implementation of Modern Column-Oriented Database Systems. Now Publishers Inc., Hanover, massachusetts (2013)Google Scholar
  4. 4.
    Abadi, D.J., Boncz, P.A., Harizopoulos, S.: Column-oriented database systems. Proc. VLDB Endow. 2(2), 1664–1665 (2009)CrossRefGoogle Scholar
  5. 5.
    Arulraj, J., Pavlo, A., Menon, P.: Bridging the Archipelago between row-stores and column-stores for hybrid workloads. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 583–598 (2016)Google Scholar
  6. 6.
    Chernishev, G.: Physical design approaches for column-stores. SPIIRAS Proceedings 30, 204–222 (2013)Google Scholar
  7. 7.
    Chernishev, G.: Towards self-management in a distributed column-store system. In: Morzy, T., Valduriez, P., Bellatreche, L. (eds.) ADBIS 2015. CCIS, vol. 539, pp. 97–107. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23201-0_12 CrossRefGoogle Scholar
  8. 8.
    Chernishev, G.: The design of an adaptive column-store system. J. Big Data 4(1), 21 (2017)CrossRefGoogle Scholar
  9. 9.
    Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K.: A study of PosDB performance in a distributed environment. In: Proceedings of the 2017 Software Engineering and Information Management, SEIM 2017 (2017)Google Scholar
  10. 10.
    Graefe, G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–169 (1993)CrossRefGoogle Scholar
  11. 11.
    Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: Two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 35(1), 40–45 (2012)Google Scholar
  12. 12.
    Liu, Y., et al.: DCODE: A distributed column-oriented database engine for big data analytics. In: Khalil, I., Neuhold, E., Tjoa, A.M., Da Xu, L., You, I. (eds.) CONFENIS/ICT-EurAsia -2015. LNCS, vol. 9357, pp. 289–299. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24315-3_30 CrossRefGoogle Scholar
  13. 13.
    Stonebraker, M., et al.: C-Store: A column-oriented DBMS. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 553–564. VLDB Endowment (2005)Google Scholar
  14. 14.
    Valduriez, P.: Join indices. ACM Trans. Database Syst. 12(2), 218–246 (1987)CrossRefGoogle Scholar
  15. 15.
    Zhang, Y., Xiao, Y., Wang, Z., Ji, X., Huang, Y., Wang, S.: ScaMMDB: Facing challenge of mass data processing with MMDB. In: Chen, L., et al. (eds.) APWeb/WAIM -2009. LNCS, vol. 5731, pp. 1–12. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-03996-6_1 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • George Chernishev
    • 1
    • 2
    Email author
  • Viacheslav Galaktionov
    • 1
  • Valentin Grigorev
    • 1
  • Evgeniy Klyuchikov
    • 1
  • Kirill Smirnov
    • 1
  1. 1.Saint-Petersburg UniversitySaint-PetersburgRussia
  2. 2.JetBrains ResearchSaint-PetersburgRussia

Personalised recommendations