Skip to main content
SpringerLink
Log in

HyPer Beyond Software: Exploiting Modern Hardware for Main-Memory Database Systems

  • Schwerpunktbeitrag
  • Published:
Datenbank-Spektrum Aims and scope Submit manuscript

Abstract

In this paper, we survey the use of advanced hardware features for optimizing main-memory database systems in the context of our HyPer project. We exploit the virtual memory management for snapshotting the transactional data in order to separate OLAP queries from parallel OLTP transactions. The access behavior of database objects from simultaneous OLTP transactions is monitored using the virtual memory management component in order to compact the database into hot and cold partitions. Utilizing many-core NUMA-organized database servers is facilitated by the morsel-driven adaptive parallelization and partitioning that guarantees data locality w.r.t. the processing core. The most recent Hardware Transactional Memory support of, e.g., Intel’s Haswell processor, can be used as the basis for a lock-free concurrency control scheme for OLTP transactions. Finally, we show how heterogeneous processors of “wimpy” devices such as tablets can be utilized for high-performance and energy-efficient query processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Low Level Virtual Machine (LLVM, llvm.org) is an open-source compiler infrastructure project that provides a machine-independent assembly language.

References

  1. Albutiu MC, Kemper A, Neumann T (2012) AQ2Massively parallel sort-merge joins in main memory multi-core database systems. Proceedings VDLB Endowment 5(10):1064–1075

    Article  Google Scholar 

  2. Alonso G (2013) AQ3Hardware killed the software star. In: ICDE, Brisbane, Queensland, 8–12 April 2013

  3. Alonso G, Kossmann D, Roscoe T (2011) AQ4SWissBox: an architecture for data processing appliances. In: CIDR 2011, Asilomar, CA, USA, January 9–12, 2011

  4. Apple. AQ5Data Management in iOS. https://developer.apple.com/technologies/ios/data-management.html, accessed 1 Sep 2014

  5. Balakrishnan S, Rajwar R, Upton M, Lai K (2005) The impact of performance asymmetry in emerging multicore architectures. ACM SIGARCH Comp Archit News 33(2):506–517

    Article  Google Scholar 

  6. Balkesen C, Alonso G, Teubner J, Özsu MT (2013) Multi-core, main-memory joins: sort vs. hash revisited. Proceedings VDLB Endowment 7(1):85–96

    Google Scholar 

  7. Bernstein PA, Hadzilacos V, Goodman N (1987) AQ6Concurrency control and recovery in database systems. Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA

  8. Carey MJ (1983) Modeling and evaluation of database concurrency control algorithms. PhD thesis . EECS Department, University of California, Berkeley, CA, USA

  9. Diaconu C, Freedman C, Ismert E, Larson P-Å, Mittal P, Stonecipher R, Verma N, Zwilling M (2013) Hekaton: SQL server’s memory-optimized OLTP engine. In: SIGMOD 2013, New York, NY, USA, June 22–27, 2013

  10. Esmaeilzadeh H, Blem E, St. Amant R, Sankaralingam K, Burger D (2011) Dark silicon and the end of multicore scaling. In: ISCA 2011, June 4–8, 2011, San Jose, CA, USA

  11. Färber F, Cha SK, Primsch J, Bornhövd C, Sigg S, Lehner W (2011) SAP HANA database: data management for modern business applications. SIGMOD Rec 40(4):45–51

    Article  Google Scholar 

  12. Funke F, Kemper A, Neumann T (2012) Compacting transactional data in hybrid OLTP&OLAP databases. Proceedings VDLB Endowment 5(11):1424–1435

    Article  Google Scholar 

  13. Google. Storage options. http://developer.android.com/guide/topics/data/data-storage.html#db, accessed 1 Sep 2014

  14. Gorman M (2004) Understanding the Linux virtual memory manager. Prentice Hall , Upper Saddle River, New Jersey, USA

  15. Greenhalgh P (2011) big. LITTLE processing with ARM Cortex-A15 & Cortex-A7. Whitepaper by ARM, URL: http://www.arm.com/files/download/big_LITTLE_Final_Final.pdf, September 2011, accessed 1 Sep 2014

  16. Hardavellas N, Ferdman M, Falsafi B, Ailamaki A (2011) Toward dark silicon in servers. IEEE Micro 31(4):6–15

    Article  Google Scholar 

  17. Harizopoulos S, Abadi DJ, Madden S, Stonebraker M (2008) OLTP through the looking glass, and what we found there. In: SIGMOD 2008, Vancouver, BC, Canada, June 10–12, 2008

  18. Herlihy M, Moss JEB (1993) Transactional memory: architectural support for lock-free data structures. In: ISCA May 1993, San Diego, CA, USA

  19. IHS. Processor market set for strong growth in 2013, Courtesy of smartphones and tablets. http://press.ihs.com/printpdf/18632, accessed 1 Sep 2014

  20. Kemper A, Neumann T (2011) HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE, Hannover, 11–16 April 2011

  21. Leis V, Kemper A, Neumann T (2013) The adaptive radix tree: ARTful indexing for main-memory databases. In: ICDE, Brisbane, Queensland, 8–12 April 2013

  22. Leis V, Kemper A, Neumann T (2014) Exploiting hardware transactional memory in main-memory databases. In: ICDE, Chicago, IL, 31 March to 4 April 2014

  23. Leis V, Boncz P, Kemper A, Neumann T (2014) Morsel-driven parallelism: a NUMA-aware query evaluation framework for the many-core age. In: Proceedings of the 2014 ACM SIGMOD international conference on management of data, Snowbird, UT, 22–27 June 2014

  24. Li Y, Pandis I, Müller R, Raman V, Lohman GM (2013) NUMA-aware algorithms: the case of data shuffling. In: CIDR 2013, Asilomar, CA, USA, January 6–9, 2013

  25. Mühlbauer T, Rödiger W, Seilbeck R, Kemper A, Neumann T (2014) Heterogeneity-conscious parallel query execution: getting a better mileage while driving faster! In: DaMoN 2014, Snowbird, Utah, USA, June 23, 2014

  26. Mühlbauer T, Rödiger W, Seilbeck R, Reiser A, Kemper A, Neumann T (2014) One DBMS for all: the brawny few and the wimpy crowd. In: SIGMOD 2014, Snowbird, UT, USA, June 22–27, 2014

  27. Neumann T (2011) Efficiently compiling efficient query plans for modern hardware. Proceedings VLDB Endowment 4(9):539–550

    Article  Google Scholar 

  28. Schall D, Härder T (2013) Energy-proportional query execution using a cluster of wimpy nodes. In: DaMoN 2013, New York, NY, USA, June 24, 2013

  29. IBM solidDB (2012) http://www.ibm.com/software/data/soliddb/, accessed 1 Sep 2014

  30. Tsirogiannis D, Harizopoulos S, Shah MA (2010) Analyzing the energy efficiency of a database server. In: SIGMOD 2010, Indianapolis, Indiana, USA, June 6–10, 2010

  31. Van Craeynest K, Eeckhout L (2013) Understanding fundamental design choices in single-ISA heterogeneous multicore architectures. ACM Trans Archit Code Optim 9(4):Art. 32

    Article  Google Scholar 

  32. VoltDB (2010) Technical overview.http://voltdb.com/downloads/datasheets_collateral/technical_overview.pdf, accessed 1 Sep 2014

Download references

Acknowledgement

This work has been supported by the German Research Foundation DFG and various collaborations and donations by industry partners (Google, IBM, Oracle, SAP).

Author information

Authors and Affiliations

  1. Fakultät für Informatik, TU München, München, Deutschland

    Florian Funke, Alfons Kemper, Tobias Mühlbauer, Thomas Neumann & Viktor Leis

Authors
  1. Florian Funke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfons Kemper
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tobias Mühlbauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Viktor Leis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alfons Kemper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Funke, F., Kemper, A., Mühlbauer, T. et al. HyPer Beyond Software: Exploiting Modern Hardware for Main-Memory Database Systems. Datenbank Spektrum 14, 173–181 (2014). https://doi.org/10.1007/s13222-014-0165-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13222-014-0165-y

Keywords

Search

Navigation