, Volume 14, Issue 3, pp 173–181 | Cite as

HyPer Beyond Software: Exploiting Modern Hardware for Main-Memory Database Systems

  • Florian Funke
  • Alfons Kemper
  • Tobias Mühlbauer
  • Thomas Neumann
  • Viktor Leis


In this paper, we survey the use of advanced hardware features for optimizing main-memory database systems in the context of our HyPer project. We exploit the virtual memory management for snapshotting the transactional data in order to separate OLAP queries from parallel OLTP transactions. The access behavior of database objects from simultaneous OLTP transactions is monitored using the virtual memory management component in order to compact the database into hot and cold partitions. Utilizing many-core NUMA-organized database servers is facilitated by the morsel-driven adaptive parallelization and partitioning that guarantees data locality w.r.t. the processing core. The most recent Hardware Transactional Memory support of, e.g., Intel’s Haswell processor, can be used as the basis for a lock-free concurrency control scheme for OLTP transactions. Finally, we show how heterogeneous processors of “wimpy” devices such as tablets can be utilized for high-performance and energy-efficient query processing.


Main-memory database system Hardware transactional memory Energy efficiency Snapshot OLTP OLAP 



This work has been supported by the German Research Foundation DFG and various collaborations and donations by industry partners (Google, IBM, Oracle, SAP).


  1. 1.
    Albutiu MC, Kemper A, Neumann T (2012) AQ2Massively parallel sort-merge joins in main memory multi-core database systems. Proceedings VDLB Endowment 5(10):1064–1075CrossRefGoogle Scholar
  2. 2.
    Alonso G (2013) AQ3Hardware killed the software star. In: ICDE, Brisbane, Queensland, 8–12 April 2013Google Scholar
  3. 3.
    Alonso G, Kossmann D, Roscoe T (2011) AQ4SWissBox: an architecture for data processing appliances. In: CIDR 2011, Asilomar, CA, USA, January 9–12, 2011Google Scholar
  4. 4.
    Apple. AQ5Data Management in iOS., accessed 1 Sep 2014
  5. 5.
    Balakrishnan S, Rajwar R, Upton M, Lai K (2005) The impact of performance asymmetry in emerging multicore architectures. ACM SIGARCH Comp Archit News 33(2):506–517CrossRefGoogle Scholar
  6. 6.
    Balkesen C, Alonso G, Teubner J, Özsu MT (2013) Multi-core, main-memory joins: sort vs. hash revisited. Proceedings VDLB Endowment 7(1):85–96Google Scholar
  7. 7.
    Bernstein PA, Hadzilacos V, Goodman N (1987) AQ6Concurrency control and recovery in database systems. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USAGoogle Scholar
  8. 8.
    Carey MJ (1983) Modeling and evaluation of database concurrency control algorithms. PhD thesis . EECS Department, University of California, Berkeley, CA, USAGoogle Scholar
  9. 9.
    Diaconu C, Freedman C, Ismert E, Larson P-Å, Mittal P, Stonecipher R, Verma N, Zwilling M (2013) Hekaton: SQL server’s memory-optimized OLTP engine. In: SIGMOD 2013, New York, NY, USA, June 22–27, 2013Google Scholar
  10. 10.
    Esmaeilzadeh H, Blem E, St. Amant R, Sankaralingam K, Burger D (2011) Dark silicon and the end of multicore scaling. In: ISCA 2011, June 4–8, 2011, San Jose, CA, USAGoogle Scholar
  11. 11.
    Färber F, Cha SK, Primsch J, Bornhövd C, Sigg S, Lehner W (2011) SAP HANA database: data management for modern business applications. SIGMOD Rec 40(4):45–51CrossRefGoogle Scholar
  12. 12.
    Funke F, Kemper A, Neumann T (2012) Compacting transactional data in hybrid OLTP&OLAP databases. Proceedings VDLB Endowment 5(11):1424–1435CrossRefGoogle Scholar
  13. 13.
    Google. Storage options., accessed 1 Sep 2014
  14. 14.
    Gorman M (2004) Understanding the Linux virtual memory manager. Prentice Hall , Upper Saddle River, New Jersey, USAGoogle Scholar
  15. 15.
    Greenhalgh P (2011) big. LITTLE processing with ARM Cortex-A15 & Cortex-A7. Whitepaper by ARM, URL:, September 2011, accessed 1 Sep 2014
  16. 16.
    Hardavellas N, Ferdman M, Falsafi B, Ailamaki A (2011) Toward dark silicon in servers. IEEE Micro 31(4):6–15CrossRefGoogle Scholar
  17. 17.
    Harizopoulos S, Abadi DJ, Madden S, Stonebraker M (2008) OLTP through the looking glass, and what we found there. In: SIGMOD 2008, Vancouver, BC, Canada, June 10–12, 2008Google Scholar
  18. 18.
    Herlihy M, Moss JEB (1993) Transactional memory: architectural support for lock-free data structures. In: ISCA May 1993, San Diego, CA, USAGoogle Scholar
  19. 19.
    IHS. Processor market set for strong growth in 2013, Courtesy of smartphones and tablets., accessed 1 Sep 2014
  20. 20.
    Kemper A, Neumann T (2011) HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE, Hannover, 11–16 April 2011Google Scholar
  21. 21.
    Leis V, Kemper A, Neumann T (2013) The adaptive radix tree: ARTful indexing for main-memory databases. In: ICDE, Brisbane, Queensland, 8–12 April 2013Google Scholar
  22. 22.
    Leis V, Kemper A, Neumann T (2014) Exploiting hardware transactional memory in main-memory databases. In: ICDE, Chicago, IL, 31 March to 4 April 2014Google Scholar
  23. 23.
    Leis V, Boncz P, Kemper A, Neumann T (2014) Morsel-driven parallelism: a NUMA-aware query evaluation framework for the many-core age. In: Proceedings of the 2014 ACM SIGMOD international conference on management of data, Snowbird, UT, 22–27 June 2014Google Scholar
  24. 24.
    Li Y, Pandis I, Müller R, Raman V, Lohman GM (2013) NUMA-aware algorithms: the case of data shuffling. In: CIDR 2013, Asilomar, CA, USA, January 6–9, 2013Google Scholar
  25. 25.
    Mühlbauer T, Rödiger W, Seilbeck R, Kemper A, Neumann T (2014) Heterogeneity-conscious parallel query execution: getting a better mileage while driving faster! In: DaMoN 2014, Snowbird, Utah, USA, June 23, 2014Google Scholar
  26. 26.
    Mühlbauer T, Rödiger W, Seilbeck R, Reiser A, Kemper A, Neumann T (2014) One DBMS for all: the brawny few and the wimpy crowd. In: SIGMOD 2014, Snowbird, UT, USA, June 22–27, 2014Google Scholar
  27. 27.
    Neumann T (2011) Efficiently compiling efficient query plans for modern hardware. Proceedings VLDB Endowment 4(9):539–550CrossRefGoogle Scholar
  28. 28.
    Schall D, Härder T (2013) Energy-proportional query execution using a cluster of wimpy nodes. In: DaMoN 2013, New York, NY, USA, June 24, 2013Google Scholar
  29. 29.
    IBM solidDB (2012), accessed 1 Sep 2014
  30. 30.
    Tsirogiannis D, Harizopoulos S, Shah MA (2010) Analyzing the energy efficiency of a database server. In: SIGMOD 2010, Indianapolis, Indiana, USA, June 6–10, 2010Google Scholar
  31. 31.
    Van Craeynest K, Eeckhout L (2013) Understanding fundamental design choices in single-ISA heterogeneous multicore architectures. ACM Trans Archit Code Optim 9(4):Art. 32CrossRefGoogle Scholar
  32. 32.
    VoltDB (2010) Technical overview., accessed 1 Sep 2014

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Florian Funke
    • 1
  • Alfons Kemper
    • 1
  • Tobias Mühlbauer
    • 1
  • Thomas Neumann
    • 1
  • Viktor Leis
    • 1
  1. 1.Fakultät für InformatikTU MünchenMünchenDeutschland

Personalised recommendations