Advertisement

Efficient Transaction Processing for Hyrise in Mixed Workload Environments

  • David Schwalb
  • Martin Faust
  • Johannes Wust
  • Martin Grund
  • Hasso Plattner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8921)

Abstract

Hyrise is an in-memory storage engine designed for mixed enterprise workloads that originally started as a research prototype for hybrid table layouts and basic transaction processing capabilities. This paper presents our incremental improvements and learnings to better support transactional consistency in mixed workloads.

In particular, the paper addresses a multi-version concurrency control mechanism with lock-free commit steps, tree-based multi-column indices, in-memory optimized logging and recovery mechanisms. Additionally, a mixed workload scheduling mechanism is presented, addressing partition-able transactional workloads in combination with analytical queries.

References

  1. 1.
    Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley, Reading (1987)Google Scholar
  2. 2.
    Diaconu, C., Freedman, C., Ismert, E., Larson, P.-A., Mittal, P., Stonecipher, R., Verma, N., Zwilling, M.: Hekaton: SQL server’s memory-optimized OLTP engine. In: SIGMOD (2013)Google Scholar
  3. 3.
    Faust, M., Grund, M., Berning, T., Schwalb, D., Plattner, H.: Vertical bit-packing: optimizing operations on bit-packed vectors leveraging SIMD instructions. In: Han, W.-S., Lee, M.L., Muliantara, A., Sanjaya, N.A., Thalheim, B., Zhou, S. (eds.) DASFAA 2014. LNCS, vol. 8505, pp. 132–145. Springer, Heidelberg (2014)Google Scholar
  4. 4.
    Faust, M., Schwalb, D., Krueger, J., Plattner, H.: Fast lookups for in-memory column stores: group-key indices, lookup and maintenance. In: ADMS in Conjunction with VLDB (2012)Google Scholar
  5. 5.
    Grund, M., Krueger, J., Plattner, H., Zeier, A., Cudre-Mauroux, P., Madden, S.: HYRISE–a main memory hybrid storage engine. In: VLDB (2010)Google Scholar
  6. 6.
    Johnson, R., Pandis, I., Stoica, R., Athanassoulis, M., Ailamaki, A.: Aether: a scalable approach to logging. In: VLDB (2010)Google Scholar
  7. 7.
    Kallman, R., Kimura, H., Natkins, J., Pavlo, A., Rasin, A., Zdonik, S., Jones, E., Madden, S., Stonebraker, M., Zhang, Y.: H-store: a high-performance, distributed main memory transaction processing system. In: VLDB (2008)Google Scholar
  8. 8.
    Kaufmann, M., Vagenas, P., Fischer, P.M., Kossmann, D., Färber, F.: Comprehensive and interactive temporal query processing with SAP HANA. In: VLDB (2013)Google Scholar
  9. 9.
    Kemper, A., Neumann, T.: HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE (2011)Google Scholar
  10. 10.
    Krüger, J., Kim, C., Grund, M., Satish, N., Schwalb, D., Chhugani, J., Plattner, H., Dubey, P., Zeier, A.: Fast updates on read-optimized databases using multi-core CPUs. In: VLDB (2011)Google Scholar
  11. 11.
    Larson, P.-A., Blanas, S., Diaconu, C., Freedman, C., Patel, J.M., Zwilling, M.: High-performance concurrency control mechanisms for main-memory databases. In: VLDB (2011)Google Scholar
  12. 12.
    Larson, P.-A., Clinciu, C., Hanson, E.N., Oks, A., Price, S.L., Rangarajan, S., Surna, A., Zhou, Q.: SQL server column store indexes. In: SIGMOD (2011)Google Scholar
  13. 13.
    Leis, V., Kemper, A., Neumann, T.: The adaptive radix tree: ARTful indexing for main-memory databases. In: ICDE (2013)Google Scholar
  14. 14.
    Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. In: TODS (1998)Google Scholar
  15. 15.
    Mühe, H., Kemper, A., Neumann, T.: Executing long-running transactions in synchronization-free main memory database systems. In: CIDR (2013)Google Scholar
  16. 16.
    Mühlbauer, T., Rödiger, W., Reiser, A., Kemper, A., Neumann, T.: ScyPer: a hybrid OLTP&OLAP distributed main memory database system for scalable real-time analytics. In: BTW (2013)Google Scholar
  17. 17.
    Pandis, I., Johnson, R., Hardavellas, N., Ailamaki, A.: Data-oriented transaction execution. In: VLDB (2010)Google Scholar
  18. 18.
    Plattner, H.: A common database approach for OLTP and OLAP using an in-memory column database. In: SIGMOD (2009)Google Scholar
  19. 19.
    Raman, V., Attaluri, G., Barber, R., Chainani, N., Kalmuk, D., KulandaiSamy, V., Leenstra, J., Lightstone, S., Liu, S., Lohman, G.M., Malkemus, T., Mueller, R., Pandis, I., Schiefer, B., Sharpe, D., Sidle, R., Storm, A., Zhang, L.: DB2 with BLU acceleration: so much more than just a column store. In: VLDB (2013)Google Scholar
  20. 20.
    Schwalb, D., Faust, M., Krueger, J., Plattner, H.: Physical column organization in in-memory column stores. In: Meng, W., Feng, L., Bressan, S., Winiwarter, W., Song, W. (eds.) DASFAA 2013, Part II. LNCS, vol. 7826, pp. 48–63. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  21. 21.
    Sikka, V., Färber, F., Lehner, W., Cha, S.K., Peh, T., Bornhövd, C.: Efficient transaction processing in SAP HANA database - the end of a column store myth. In: SIGMOD (2012)Google Scholar
  22. 22.
    Stonebraker, M., Abadi, D., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E.: C-store: a column-oriented DBMS. In: VLDB (2005)Google Scholar
  23. 23.
    Stonebraker, M., Weisberg, A.: The VoltDB main memory DBMS. IEEE Data Eng. Bull. 36(2), 21–27 (2013)Google Scholar
  24. 24.
    Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: SOSP (2013)Google Scholar
  25. 25.
    Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: SIMD-Scan: ultra fast in-memory table scan using on-chip vector processing units. In: VLDB (2009)Google Scholar
  26. 26.
    Wust, J., Boese, J.-H., Renkes, F., Blessing, S., Krueger, J., Plattner, H.: Efficient logging for enterprise workloads on column-oriented in-memory databases. In: CIKM (2012)Google Scholar
  27. 27.
    Wust, J., Grund, M., Hoewelmeyer, K., Schwalb, D., Plattner, H.: Concurrent execution of mixed enterprise workloads on in-memory databases. In: Bhowmick, S.S., Dyreson, C.E., Jensen, C.S., Lee, M.L., Muliantara, A., Thalheim, B. (eds.) DASFAA 2014, Part I. LNCS, vol. 8421, pp. 126–140. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  28. 28.
    Wust, J., Grund, M., Plattner, H.: Dynamic query prioritization for in-memory databases. In: IMDM in Conjunction with VLDB (2013)Google Scholar
  29. 29.
    Wust, J., Grund, M., Plattner, H.: Tamex: a task-based query execution framework for mixed enterprise workloads on in-memory databases. In: GI-Jahrestagung (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • David Schwalb
    • 1
  • Martin Faust
    • 1
  • Johannes Wust
    • 1
  • Martin Grund
    • 2
  • Hasso Plattner
    • 1
  1. 1.Hasso Plattner InstitutePotsdamGermany
  2. 2.eXascale InfolabUniversity of FribourgFribourgSwitzerland

Personalised recommendations