Advertisement

Hyrise-NV: Instant Recovery for In-Memory Databases Using Non-Volatile Memory

  • David Schwalb
  • Girish Kumar B.K.
  • Markus Dreseler
  • Anusha S.
  • Martin Faust
  • Adolf Hohl
  • Tim Berning
  • Gaurav Makkar
  • Hasso Plattner
  • Parag Deshmukh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9643)

Abstract

Emerging non-volatile memory technologies (NVM) offer fast and byte-addressable access, allowing to rethink the durability mechanisms of in-memory databases. In this paper, we present Hyrise-NV, a database storage engine that maintains table and index structures on NVM. Our architecture updates the database state and index structures transactionally consistent on NVM using multi-version data structures, allowing to instantly recover databases independent of their size. For index structures, we present nvBTree using multi-versioning to provide failure-atomic tree updates on NVM. We evaluate Hyrise-NV both on DRAM and with hardware-based emulation of NVM using the TPC-C benchmark. Hyrise-NV recovers databases independent of their size, allowing the recovery of a table with 10 million rows in less than 100 ms.

Keywords

Index Structure Cache Line Software Transactional Memory Parallel User Storage Engine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

We thank Konrad Büker, Jürgen Schrage and Ahmad Waizy from Fujitsu and Rami Akkad, Bernhard Höppner and Jürgen Müller from the SAP Innovation Center Potsdam for their support and hardware access. We also thank Subramanya Dulloor from Intel Labs for access to the PMEP emulator.

References

  1. 1.
    Chatzistergiou, A., et al.: REWIND: recovery write-ahead system for in-memory non-volatile data-structures. In: VLDB (2015)Google Scholar
  2. 2.
    Chen, S., Jin, Q.: Persistent B+-trees in non-volatile main memory. In: VLDB (2015)Google Scholar
  3. 3.
    Chen, S., et al.: Rethinking database algorithms for phase change memory. In: CIDR (2011)Google Scholar
  4. 4.
    Coburn, J., et al.: NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories. SIGPLAN 46, 105–118 (2011)CrossRefGoogle Scholar
  5. 5.
    Condit, J., et al.: Better I/O through byte-addressable, persistent memory. In: SOSP (2009)Google Scholar
  6. 6.
    DeBrabant, J., et al.: A prolegomenon on OLTP database systems for non-volatile memory. In: VLDB (2014)Google Scholar
  7. 7.
    Dulloor, S.R., et al.: System software for persistent memory. In: EuroSys (2014)Google Scholar
  8. 8.
    Fang, R., et al.: High performance database logging using storage class memory. In: ICDE (2011)Google Scholar
  9. 9.
    Faust, M., et al.: Fast lookups for in-memory column stores: group-key indices, lookup and maintenance. In: ADMS (2012)Google Scholar
  10. 10.
    Grund, M., et al.: HYRISE—a main memory hybrid storage engine. In: VLDB (2010)Google Scholar
  11. 11.
    Höppner, B., et al.: An approach for hybrid-memory scaling columnar in-memory databases. In: ADMS (2014)Google Scholar
  12. 12.
    Kallman, R., et al.: H-store: a high-performance, distributed main memory transaction processing system. In: VLDB (2008)Google Scholar
  13. 13.
    Kemper, A., Neumann, T.: HyPer: a hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE (2011)Google Scholar
  14. 14.
    Krüger, J., et al.: Fast updates on read-optimized databases using multi-core CPUs. VLDB 5, 61–72 (2011)Google Scholar
  15. 15.
    Malviya, N., et al.: Rethinking main memory OLTP recovery. In: ICDE (2014)Google Scholar
  16. 16.
    Mohan, C., et al.: ARIES: a transaction recovery method supporting fine- granularity locking and partial rollbacks using write-ahead logging. In: TODS (1998)Google Scholar
  17. 17.
    Narayanan, D., Hodson, O.: Whole-system persistence. In: ASPLOS (2012)Google Scholar
  18. 18.
    Ongaro, D., et al.: Fast crash recovery in RAMCloud. In: SOSP (2011)Google Scholar
  19. 19.
    Oukid, I., et al.: Instant recovery for main-memory databases. In: CIDR (2015)Google Scholar
  20. 20.
    Pelley, S., et al.: Storage management in the NVRAM era. In: VLDB (2013)Google Scholar
  21. 21.
    Plattner, H., et al.: The impact of columnar in-memory databases on enterprise systems. In: VLDB (2014)Google Scholar
  22. 22.
    Raman, V., et al.: DB2 with BLU acceleration: so much more than just a column store. In: VLDB (2013)Google Scholar
  23. 23.
    Schwalb, D., et al.: Efficient transaction processing for hyrise in mixed workload environments. In: IMDM (2014)Google Scholar
  24. 24.
    Schwalb, D., et al.: NVC-hashmap: a persistent and concurrent hashmap for non-volatile memories. In: IMDM (2015)Google Scholar
  25. 25.
    Schwalb, D., et al.: nvm malloc: memory allocation for NVRAM. In: ADMS (2015)Google Scholar
  26. 26.
    Stonebraker, M., et al.: C-Store: a column-oriented DBMS. In: VLDB (2005)Google Scholar
  27. 27.
    Thatte, S.M.: Persistent memory: a storage system for object-oriented databases. In: OODS (1991)Google Scholar
  28. 28.
    Tianzheng, W., Ryan, J.: Scalable logging through emerging non-volatile memory. In: VLDB (2014)Google Scholar
  29. 29.
    Venkataraman, S., et al.: Consistent and durable data structures for non-volatile byte-addressable memory. In: FAST (2011)Google Scholar
  30. 30.
    Volos, H., et al.: Mnemosyne: lightweight persistent memory. In: ASPLOS (2011)Google Scholar
  31. 31.
    Wang, C., et al.: NVMalloc: exposing an aggregate SSD store as a memory partition in extreme-scale machines. In: IPDPS (2012)Google Scholar
  32. 32.
    Wu, M., Zwaenepoel, W.: eNVy: a non-volatile, main memory storage system. In: ASPLOS (1994)Google Scholar
  33. 33.
    Wust, J., et al.: Efficient logging for enterprise workloads on column-oriented in-memory databases. In: CIKM (2012)Google Scholar
  34. 34.
    Yang, J., et al.: NV-Tree: reducing consistency cost for NVM-based single level systems. In: FAST (2015)Google Scholar
  35. 35.
    Zhang, Y., et al.: Mojim: a reliable and highly-available non-volatile memory system. In: ASPLOS (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • David Schwalb
    • 1
  • Girish Kumar B.K.
    • 2
  • Markus Dreseler
    • 1
  • Anusha S.
    • 2
  • Martin Faust
    • 1
  • Adolf Hohl
    • 2
  • Tim Berning
    • 1
  • Gaurav Makkar
    • 2
  • Hasso Plattner
    • 1
  • Parag Deshmukh
    • 2
  1. 1.Hasso Plattner InstitutePotsdamGermany
  2. 2.NetAppSunnyvaleUSA

Personalised recommendations