Advertisement

PosDB: A Distributed Column-Store Engine

  • George Chernishev
  • Viacheslav Galaktionov
  • Valentin Grigorev
  • Evgeniy Klyuchikov
  • Kirill Smirnov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10742)

Abstract

In this paper we present a novel disk-based distributed column-store, describe its architecture and discuss a number of technical solutions. Our system is essentially a query engine which was written completely from scratch. It is aimed for shared-nothing environments and supports different forms of parallel query processing.

Query processing in PosDB is organized according to the classic Volcano pull-based model which is adapted for the column-store case. Currently, we support late materialization only, and therefore employ a join index data structure to represent positional information. In our system query plan can consist of both positional and value operators. PosDB has about a dozen of core operators among which several variants of selections and joins, aggregation. We also have several operators that ensure intra-query parallelism and operators for network interoperability. In its current state the system is fully capable of processing the Star Schema Benchmark in a local and distributed environment.

References

  1. 1.
    O’Neil, P.E., O’Neil, E.J., Chen, X.: The Star Schema Benchmark (SSB) (2009). http://www.cs.umb.edu/~poneil/StarSchemaB.PDF. Accessed 20 July 2012
  2. 2.
    Google Supersonic Library (2017). https://code.google.com/archive/p/supersonic/. Accessed 12 February 2017
  3. 3.
    Abadi, D., Boncz, P., Harizopoulos, S.: The Design and Implementation of Modern Column-Oriented Database Systems. Now Publishers Inc., Hanover, massachusetts (2013)Google Scholar
  4. 4.
    Abadi, D.J., Boncz, P.A., Harizopoulos, S.: Column-oriented database systems. Proc. VLDB Endow. 2(2), 1664–1665 (2009)CrossRefGoogle Scholar
  5. 5.
    Arulraj, J., Pavlo, A., Menon, P.: Bridging the Archipelago between row-stores and column-stores for hybrid workloads. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 583–598 (2016)Google Scholar
  6. 6.
    Chernishev, G.: Physical design approaches for column-stores. SPIIRAS Proceedings 30, 204–222 (2013)Google Scholar
  7. 7.
    Chernishev, G.: Towards self-management in a distributed column-store system. In: Morzy, T., Valduriez, P., Bellatreche, L. (eds.) ADBIS 2015. CCIS, vol. 539, pp. 97–107. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23201-0_12 CrossRefGoogle Scholar
  8. 8.
    Chernishev, G.: The design of an adaptive column-store system. J. Big Data 4(1), 21 (2017)CrossRefGoogle Scholar
  9. 9.
    Chernishev, G., Galaktionov, V., Grigorev, V., Klyuchikov, E., Smirnov, K.: A study of PosDB performance in a distributed environment. In: Proceedings of the 2017 Software Engineering and Information Management, SEIM 2017 (2017)Google Scholar
  10. 10.
    Graefe, G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–169 (1993)CrossRefGoogle Scholar
  11. 11.
    Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: Two decades of research in column-oriented database architectures. IEEE Data Eng. Bull. 35(1), 40–45 (2012)Google Scholar
  12. 12.
    Liu, Y., et al.: DCODE: A distributed column-oriented database engine for big data analytics. In: Khalil, I., Neuhold, E., Tjoa, A.M., Da Xu, L., You, I. (eds.) CONFENIS/ICT-EurAsia -2015. LNCS, vol. 9357, pp. 289–299. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24315-3_30 CrossRefGoogle Scholar
  13. 13.
    Stonebraker, M., et al.: C-Store: A column-oriented DBMS. In: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005, pp. 553–564. VLDB Endowment (2005)Google Scholar
  14. 14.
    Valduriez, P.: Join indices. ACM Trans. Database Syst. 12(2), 218–246 (1987)CrossRefGoogle Scholar
  15. 15.
    Zhang, Y., Xiao, Y., Wang, Z., Ji, X., Huang, Y., Wang, S.: ScaMMDB: Facing challenge of mass data processing with MMDB. In: Chen, L., et al. (eds.) APWeb/WAIM -2009. LNCS, vol. 5731, pp. 1–12. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-03996-6_1 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • George Chernishev
    • 1
    • 2
  • Viacheslav Galaktionov
    • 1
  • Valentin Grigorev
    • 1
  • Evgeniy Klyuchikov
    • 1
  • Kirill Smirnov
    • 1
  1. 1.Saint-Petersburg UniversitySaint-PetersburgRussia
  2. 2.JetBrains ResearchSaint-PetersburgRussia

Personalised recommendations