Skip to main content
SpringerLink
Log in

MM-DIRECT

Main memory database instant recovery with tuple consistent checkpoint

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Main memory databases (MMDBs) technology handles the primary database in Random Access Memory (RAM) to provide high throughput and low latency. However, volatile memory makes MMDBs much more sensitive to system failures. The contents of the database are lost in these failures, and, as a result, systems may be unavailable for a long time until the database recovery process has been finished. Therefore, novel recovery techniques are needed to repair crashed MMDBs as quickly as possible. This paper presents MM-DIRECT (Main Memory Database Instant RECovery with Tuple consistent checkpoint), a recovery technique that enables MMDBs to schedule transactions simultaneously with the database recovery process at system startup. Thus, it gives the impression that the database is instantly restored. The approach implements a tuple-level consistent checkpoint to reduce the recovery time. To validate the proposed approach, experiments were performed in a prototype implemented on the Redis database. The results show that the instant recovery technique effectively provides high transaction throughput rates even during the recovery process and normal database processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33

Similar content being viewed by others

Notes

  1. https://github.com/ArlinoMagalhaes/MM-DIRECT.

References

  1. Ailamaki, A., DeWitt, D.J., Hill, M.D., Wood, D.A.: Dbmss on a modern processor: Where does time go? In: Atkinson, M.P., Orlowska, M.E., Valduriez, P., Zdonik, S.B., Brodie, M.L. (eds.) VLDB’99, Proceedings of 25th International Conference on Very Large Data Bases, September 7–10, 1999, Edinburgh, Scotland, UK, pp. 266–277. Morgan Kaufmann (1999). http://www.vldb.org/conf/1999/P28.pdf

  2. Arulraj, J., Pavlo, A.: How to build a non-volatile memory database management system. In: Salihoglu, S., Zhou, W., Chirkova, R., Yang, J., Suciu, D. (eds.) Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14–19, 2017, pp. 1753–1758. ACM (2017). https://doi.org/10.1145/3035918.3054780

  3. Behravesh, R., Coronado, E., Riggio, R.: Performance evaluation on virtualization technologies for NFV deployment in 5g networks. In: Jacquenet, C., Turck, F.D., Chemouil, P., Esposito, F., Festor, O., Cerroni, W., Secci, S. (eds.) 5th IEEE Conference on Network Softwarization, NetSoft 2019, Paris, France, June 24–28, 2019, pp. 24–29. IEEE (2019). https://doi.org/10.1109/NETSOFT.2019.8806664

  4. Berkeley DB Documentation: Berkeley db programmer’s reference guide (2020). https://docs.oracle.com/cd/E17276_01/html/programmer_reference

  5. Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley (1987). http://research.microsoft.com/en-us/people/philbe/ccontrol.aspx

  6. Binna, R., Zangerle, E., Pichl, M., Specht, G., Leis, V.: HOT: A height optimized trie index for main-memory database systems. In: Das, G., Jermaine, C.M., Bernstein, P.A. (eds.) Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10–15, 2018, pp. 521–534. ACM (2018). https://doi.org/10.1145/3183713.3196896

  7. Cao, W., Sahin, S., Liu, L., Bao, X.: Evaluation and analysis of in-memory key-value systems. In: Pu, C., Fox, G.C., Damiani, E. (eds.) 2016 IEEE International Congress on Big Data, San Francisco, CA, USA, June 27–July 2, 2016, pp. 26–33. IEEE Computer Society (2016). https://doi.org/10.1109/BigDataCongress.2016.13

  8. Chen, S., Gibbons, P.B., Mowry, T.C.: Improving index performance through prefetching. In: Mehrotra, S., Sellis, T.K. (eds.) Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data, Santa Barbara, CA, USA, May 21–24, 2001, pp. 235–246. ACM (2001). https://doi.org/10.1145/375663.375688

  9. Choi, J., Lian, R., Li, Z., Canis, A., Anderson, J.H.: Accelerating memcached on AWS cloud FPGAS. In: Proceedings of the 9th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2018, Toronto, ON, Canada, June 20–22, 2018, pp. 2:1–2:8. ACM (2018). https://doi.org/10.1145/3241793.3241795

  10. Comer, D.: The ubiquitous b-tree. ACM Comput. Surv. 11(2), 121–137 (1979). https://doi.org/10.1145/356770.356776

    Article  MathSciNet  Google Scholar 

  11. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Hellerstein, J.M., Chaudhuri, S., Rosenblum, M. (eds.) Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC 2010, Indianapolis, Indiana, USA, June 10–11, 2010, pp. 143–154. ACM (2010). https://doi.org/10.1145/1807128.1807152

  12. DeWitt, D.J., Katz, R.H., Olken, F., Shapiro, L.D., Stonebraker, M., Wood, D.A.: Implementation techniques for main memory database systems. In: Yormark, B. (ed.) SIGMOD’84, Proceedings of Annual Meeting, Boston, MA, USA, June 18–21, 1984, pp. 1–8. ACM Press (1984). https://doi.org/10.1145/602259.602261

  13. Diaconu, C., Freedman, C., Ismert, E., Larson, P., Mittal, P., Stonecipher, R., Verma, N., Zwilling, M.: Hekaton: SQL server’s memory-optimized OLTP engine. In: Ross, K.A., Srivastava, D., Papadias, D. (eds.) Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2013, New York, NY, USA, June 22–27, 2013, pp. 1243–1254. ACM (2013). https://doi.org/10.1145/2463676.2463710

  14. Eich, M.H.: Main memory database recovery. In: Proceedings of the Fall Joint Computer Conference, November 2–6, 1986, Dallas, TX, USA, pp. 1226–1232. IEEE Computer Society (1986)

  15. Faerber, F., Kemper, A., Larson, P., Levandoski, J.J., Neumann, T., Pavlo, A.: Main memory database systems. Found. Trends Databases 8(1–2), 1–130 (2017). https://doi.org/10.1561/1900000058

    Article  Google Scholar 

  16. Fan, B., Andersen, D.G., Kaminsky, M.: Memc3: Compact and concurrent memcache with dumber caching and smarter hashing. In: Feamster, N., Mogul, J.C. (eds.) Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2013, Lombard, IL, USA, April 2–5, 2013, pp. 371–384. USENIX Association (2013). https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/fan

  17. Färber, F., Cha, S.K., Primsch, J., Bornhövd, C., Sigg, S., Lehner, W.: SAP HANA database: data management for modern business applications. ACM Sigmod Record 40(4), 45–51 (2011). https://doi.org/10.1145/2094114.2094126

    Article  Google Scholar 

  18. Färber, F., May, N., Lehner, W., Große, P., Müller, I., Rauhe, H., Dees, J.: The SAP HANA database—an architecture overview. IEEE Database Eng. Bull. 35(1), 28–33 (2012). http://sites.computer.org/debull/A12mar/hana.pdf

  19. Freedman, C., Ismert, E., Larson, P.: Compilation in the microsoft SQL server hekaton engine. IEEE Database Eng. Bull. 37(1), 22–30 (2014). http://sites.computer.org/debull/A14mar/p22.pdf

  20. Funke, F., Kemper, A., Mühlbauer, T., Neumann, T., Leis, V.: Hyper beyond software: exploiting modern hardware for main-memory database systems. Datenbank-Spektrum 14(3), 173–181 (2014). https://doi.org/10.1007/s13222-014-0165-y

    Article  Google Scholar 

  21. Garcia-Molina, H., Salem, K.: Main memory database systems: an overview. IEEE Trans. Knowl. Data 4(6), 509–516 (1992). https://doi.org/10.1109/69.180602

    Article  Google Scholar 

  22. Gatla, O.R., Zheng, M., Hameed, M., Dubeyko, V., Manzanares, A., Blagojevic, F., Guyot, C., Mateescu, R.: Towards robust file system checkers. ACM Trans. Storage 14(4), 35:1-35:25 (2018). https://doi.org/10.1145/3281031

    Article  Google Scholar 

  23. Genosa, W.: Monitoring performance with IOSTAT and VMSTAT. Sys Admin. 3(2), 6–16 (1994)

  24. Graefe, G.: Sorting and indexing with partitioned b-trees. In: 1st biennial conference on innovative data systems research, CIDR 2003, Asilomar, CA, USA, January 5–8, 2003, Online Proceedings (2003). http://www.cidrdb.org. http://www-db.cs.wisc.edu/cidr/cidr2003/program/p1.pdf

  25. Graefe, G., McKenna, W.J.: The volcano optimizer generator: extensibility and efficient search. In: Proceedings of the 9th International Conference on Data Engineering, April 19–23, 1993, Vienna, Austria, pp. 209–218. IEEE Computer Society (1993). https://doi.org/10.1109/ICDE.1993.344061

  26. Gruenwald, L., Huang, J., Dunham, M.H., Lin, J.L., Peltier, A.C.: Recovery in main memory databases (1996). https://cs.ou.edu/~database/documents/gdhlp96.pdf

  27. Härder, T., Reuter, A.: Principles of transaction-oriented database recovery. ACM Comput. Surv. (CSUR) 15(4), 287–317 (1983). https://doi.org/10.1145/289.291

    Article  MathSciNet  Google Scholar 

  28. Hazenberg, W., Hemminga, S.: Main memory database systems: opportunities and pitfalls. In: SC@ RUG 2011 Proceedings, p. 113 (2011)

  29. Ibrahim, A.A.Z.A., Varrette, S., Bouvry, P.: PRESENCE: toward a novel approach for performance evaluation of mobile cloud SAAS web services. In: 2018 International Conference on Information Networking, ICOIN 2018, Chiang Mai, Thailand, January 10–12, 2018, pp. 50–55. IEEE (2018). https://doi.org/10.1109/ICOIN.2018.8343082

  30. Kallman, R., Kimura, H., Natkins, J., Pavlo, A., Rasin, A., Zdonik, S.B., Jones, E.P.C., Madden, S., Stonebraker, M., Zhang, Y., Hugg, J., Abadi, D.J.: H-store: a high-performance, distributed main memory transaction processing system. Proc. VLDB Endow. 1(2), 1496–1499 (2008). https://doi.org/10.14778/1454159.1454211. http://www.vldb.org/pvldb/vol1/1454211.pdf

  31. Kemper, A., Neumann, T.: Hyper: a hybrid oltp &olap main memory database system based on virtual memory snapshots. In: Abiteboul, S., Böhm, K., Koch, C., Tan, K. (eds.) Proceedings of the 27th International Conference on Data Engineering, ICDE 2011, April 11–16, 2011, Hannover, Germany, pp. 195–206. IEEE Computer Society (2011). https://doi.org/10.1109/ICDE.2011.5767867

  32. Kim, C., Chhugani, J., Satish, N., Sedlar, E., Nguyen, A.D., Kaldewey, T., Lee, V.W., Brandt, S.A., Dubey, P.: FAST: fast architecture sensitive tree search on modern CPUS and GPUS. In: Elmagarmid, A.K., Agrawal, D. (eds.) Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2010, Indianapolis, Indiana, USA, June 6–10, 2010, pp. 339–350. ACM (2010). https://doi.org/10.1145/1807167.1807206

  33. Kim, K., Wang, T., Johnson, R., Pandis, I.: ERMIA: fast memory-optimized database system for heterogeneous workloads. In: Özcan, F., Koutrika, G., Madden, S. (eds.) Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26–July 01, 2016, pp. 1675–1687. ACM (2016). https://doi.org/10.1145/2882903.2882905

  34. Kim, L., Lee, E.: ROVN: replica placement for distributed data system with heterogeneous memory devices. IEICE Electron. Express 18(23), 20210,379 (2021). https://doi.org/10.1587/elex.18.20210379

    Article  Google Scholar 

  35. Kimura, H.: FOEDUS: OLTP engine for a thousand cores and NVRAM. In: T.K. Sellis, S.B. Davidson, Z.G. Ives (eds.) Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, VIC, Australia, May 31–June 4, 2015, pp. 691–706. ACM (2015). https://doi.org/10.1145/2723372.2746480

  36. Larson, P., Levandoski, J.J.: Modern main-memory database systems. Proc. VLDB Endow. 9(13), 1609–1610 (2016). https://doi.org/10.14778/3007263.3007321. http://www.vldb.org/pvldb/vol9/p1609-larson.pdf

  37. Larson, P., Zwilling, M., Farlee, K.: The hekaton memory-optimized OLTP engine. IEEE Database Eng. Bull. 36(2), 34–40 (2013). http://sites.computer.org/debull/A13june/Hekaton1.pdf

  38. Lee, J., Kwon, Y.S., Färber, F., Muehle, M., Lee, C., Bensberg, C., Lee, J., Lee, A.H., Lehner, W.: SAP HANA distributed in-memory database system: Transaction, session, and metadata management. In: Jensen, C.S., Jermaine, C.M., Zhou, X. (eds.) 29th IEEE International Conference on Data Engineering, ICDE 2013, Brisbane, Australia, April 8–12, 2013, pp. 1165–1173. IEEE Computer Society (2013). https://doi.org/10.1109/ICDE.2013.6544906

  39. Lee, L., Xie, S., Ma, Y., Chen, S.: Index checkpoints for instant recovery in in-memory database systems. Proc. VLDB Endow. 15(8), 1671–1683 (2022). https://www.vldb.org/pvldb/vol15/p1671-lee.pdf

  40. Lehman, T.J., Carey, M.J.: A study of index structures for main memory database management systems. In: Chu, W.W., Gardarin, G., Ohsuga, S., Kambayashi, Y. (eds.) VLDB’86 12th International Conference on Very Large Data Bases, August 25–28, 1986, Kyoto, Japan, Proceedings, pp. 294–303. Morgan Kaufmann (1986). http://www.vldb.org/conf/1986/P294.PDF

  41. Leis, V., Boncz, P.A., Kemper, A., Neumann, T.: Morsel-driven parallelism: a numa-aware query evaluation framework for the many-core age. In: C.E. Dyreson, F. Li, M.T. Özsu (eds.) International Conference on Management of Data, SIGMOD 2014, Snowbird, UT, USA, June 22–27, 2014, pp. 743–754. ACM (2014). https://doi.org/10.1145/2588555.2610507

  42. Leis, V., Kemper, A., Neumann, T.: The adaptive radix tree: artful indexing for main-memory databases. In: Jensen, C.S., Jermaine, C.M., Zhou, X. (eds.) 29th IEEE International Conference on Data Engineering, ICDE 2013, Brisbane, Australia, April 8–12, 2013, pp. 38–49. IEEE Computer Society (2013). https://doi.org/10.1109/ICDE.2013.6544812

  43. Leis, V., Kemper, A., Neumann, T.: Exploiting hardware transactional memory in main-memory databases. In: Cruz, I.F., Ferrari, E., Tao, Y., Bertino, E., Trajcevski, G. (eds.) IEEE 30th International Conference on Data Engineering, Chicago, ICDE 2014, IL, USA, March 31–April 4, 2014, pp. 580–591. IEEE Computer Society (2014). https://doi.org/10.1109/ICDE.2014.6816683

  44. Levandoski, J.J., Lomet, D.B., Sengupta, S.: The bw-tree: A b-tree for new hardware platforms. In: Jensen, C.S., Jermaine, C.M., Zhou, X. (eds.) 29th IEEE International Conference on Data Engineering, ICDE 2013, Brisbane, Australia, April 8–12, 2013, pp. 302–313. IEEE Computer Society (2013). https://doi.org/10.1109/ICDE.2013.6544834

  45. Liedes, A., Wolski, A.: SIREN: A memory-conserving, snapshot-consistent checkpoint algorithm for in-memory databases. In: Liu, L., Reuter, A., Whang, K. Zhang, J. (eds.) Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006, 3–8 April 2006, Atlanta, GA, USA, p. 99. IEEE Computer Society (2006). https://doi.org/10.1109/ICDE.2006.140

  46. Litwin, W.: Linear hashing: a new tool for file and table addressing. In: 6th International Conference on Very Large Data Bases, October 1–3, 1980, Montreal, QC, Canada, Proceedings, pp. 212–223. IEEE Computer Society (1980)

  47. Lomet, D.B., Fekete, A.D., Wang, R., Ward, P.: Multi-version concurrency via timestamp range conflict management. In: Kementsietsidis, A., Salles, M.A.V. (eds.) IEEE 28th International Conference on Data Engineering (ICDE 2012), Washington, DC, USA (Arlington, Virginia), 1–5 April, 2012, pp. 714–725. IEEE Computer Society (2012). https://doi.org/10.1109/ICDE.2012.10

  48. Ma, Y., Xie, S., Zhong, H., Lee, L., Lv, K.: Hiengine: How to architect a cloud-native memory-optimized database engine. In: Ives, Z., Bonifati, A., Abbadi, A.E. (eds.) SIGMOD ’22: International Conference on Management of Data, Philadelphia, PA, USA, June 12–17, 2022, pp. 2177–2190. ACM (2022). https://doi.org/10.1145/3514221.3526043

  49. Magalhães, A.: Main memory databases instant recovery. In: Bernstein, P.A., Rabl, T. (eds.) Proceedings of the VLDB 2021 PhD Workshop co-located with the 47th International Conference on Very Large Databases (VLDB 2021), Copenhagen, Denmark, August 16, 2021, CEUR Workshop Proceedings, vol. 2971. CEUR-WS.org (2021). http://ceur-ws.org/Vol-2971/paper10.pdf

  50. Magalhães, A., Brayner, A., Monteiro, J.M., Moraes, G.: Indexed log file: towards main memory database instant recovery. In: Velegrakis, Y., Zeinalipour-Yazti, D., Chrysanthis, P.K., Guerra, F. (eds.) Proceedings of the 24th International Conference on Extending Database Technology, EDBT 2021, Nicosia, Cyprus, March 23–26, 2021, pp. 355–360. OpenProceedings.org (2021). https://doi.org/10.5441/002/edbt.2021.34

  51. Magalhães, A., Monteiro, J.M., Brayner, A.: Main memory database recovery: a survey. ACM Comput. Surv. 54(2), 46:1-46:36 (2021). https://doi.org/10.1145/3442197

    Article  Google Scholar 

  52. Malviya, N., Weisberg, A., Madden, S., Stonebraker, M.: Rethinking main memory OLTP recovery. In: Cruz, I.F., Ferrari, E., Tao, Y., Bertino, E., Trajcevski, G. (eds.) IEEE 30th International Conference on Data Engineering, Chicago, ICDE 2014, IL, USA, March 31–April 4, 2014, pp. 604–615. IEEE Computer Society (2014). https://doi.org/10.1109/ICDE.2014.6816685

  53. Mao, Y., Kohler, E., Morris, R.T.: Cache craftiness for fast multicore key-value storage. In: P. Felber, F. Bellosa, H. Bos (eds.) European Conference on Computer Systems, Proceedings of the 7th EuroSys Conference 2012, EuroSys ’12, Bern, Switzerland, April 10–13, 2012, pp. 183–196. ACM (2012). https://doi.org/10.1145/2168836.2168855

  54. Memtier Benchmark: Github—redislabs/memtier_benchmark: Nosql redis and memcache traffic generation and benchmarking tool (2020). https://github.com/RedisLabs/memtier_benchmark

  55. Menon, P., Pavlo, A., Mowry, T.C.: Relaxed operator fusion for in-memory databases: Making compilation, vectorization, and prefetching work together at last. Proc. VLDB Endow. 11(1), 1–13 (2017). https://doi.org/10.14778/3151113.3151114. http://www.vldb.org/pvldb/vol11/p1-menon.pdf

  56. Mitchell, C., Geng, Y., Li, J.: Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In: Birrell, A., Sirer, E.G. (eds.) 2013 USENIX Annual Technical Conference, San Jose, CA, USA, June 26–28, 2013, pp. 103–114. USENIX Association (2013). https://www.usenix.org/conference/atc13/technical-sessions/presentation/mitchell

  57. Mohan, C., Haderle, D., Lindsay, B.G., Pirahesh, H., Schwarz, P.M.: ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst. (TODS) 17(1), 94–162 (1992). https://doi.org/10.1145/128765.128770

    Article  Google Scholar 

  58. Mohan, C., Levine, F.E.: ARIES/IM: an efficient and high concurrency index management method using write-ahead logging. In: Stonebraker, M. (ed.) Proceedings of the 1992 ACM SIGMOD International Conference on Management of Data, San Diego, CA, USA, June 2–5, 1992, pp. 371–380. ACM Press (1992). https://doi.org/10.1145/130283.130338

  59. Mühe, H., Kemper, A., Neumann, T.: How to efficiently snapshot transactional data: hardware or software controlled? In: Harizopoulos, S., Luo, Q. (eds.) Proceedings of the 7th International Workshop on Data Management on New Hardware, DaMoN 2011, Athens, Greece, June 13, 2011, pp. 17–26. ACM (2011). https://doi.org/10.1145/1995441.1995444

  60. Neumann, T., Mühlbauer, T., Kemper, A.: Fast serializable multi-version concurrency control for main-memory database systems. In: Sellis, T.K., Davidson, S.B., Ives, Z.G. (eds.) Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, VIC, Australia, May 31–June 4, 2015, pp. 677–689. ACM (2015). https://doi.org/10.1145/2723372.2749436

  61. Nordby, M., Krzenski, S., Al Saadi, F.: Application load simulation and the potential for denial-of-service when the linux top program is misused (2020). https://www.micsymposium.org/mics_2007/Nordby.pdf

  62. Olson, M.A., Bostic, K., Seltzer, M.I.: Berkeley db. In: USENIX Annual Technical Conference, FREENIX Track, pp. 183–191 (1999)

  63. Ouaknine, K., Agra, O., Guz, Z.: Optimization of rocksdb for redis on flash. In: Proceedings of the International Conference on Compute and Data Analysis, ICCDA 2017, Lakeland, FL, USA, May 19–23, 2017, pp. 155–161. ACM (2017). https://doi.org/10.1145/3093241.3093278

  64. Palakollu, S.M.: Introduction to the linux environment. In: Practical System Programming with C, pp. 1–36. Springer (2021)

  65. Rao, J., Ross, K.A.: Making b\({}^{{+}}\)-trees cache conscious in main memory. In: Chen, W., Naughton, J.F., Bernstein, P.A. (eds.) Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, May 16–18, 2000, Dallas, TX, USA, pp. 475–486. ACM (2000). https://doi.org/10.1145/342009.335449

  66. Redis: Redis (2020). https://redis.io

  67. Redis: Documentation—redis (2023). https://redis.io/docs/

  68. Redis Labs: memtier_benchmark: A high-throughput benchmarking tool for redis & memcached (2020). https://redislabs.com/blog/memtier_benchmark-a-high-throughput-benchmarking-tool-for-redis-memcached

  69. Redis Labs: Redis labs—the best redis experience (2020). https://redislabs.com

  70. Ren, K., Diamond, T., Abadi, D.J., Thomson, A.: Low-overhead asynchronous checkpointing in main-memory database systems. In: Özcan, F., Koutrika, G., Madden, S. (eds.) Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26–July 01, 2016, pp. 1539–1551. ACM (2016). https://doi.org/10.1145/2882903.2915966

  71. Ren, K., Thomson, A., Abadi, D.J.: Lightweight locking for main memory database systems. Proc. VLDB Endow. 6(2), 145–156 (2012). https://doi.org/10.14778/2535568.2448947. http://www.vldb.org/pvldb/vol6/p145-ren.pdf

  72. Salem, K., Garcia-Molina, H.: Checkpointing memory-resident databases. In: Proceedings of the 5th International Conference on Data Engineering, February 6–10, 1989, Los Angeles, CA, USA, pp. 452–462. IEEE Computer Society (1989). https://doi.org/10.1109/ICDE.1989.47249

  73. Sauer, C.: Modern techniques for transaction-oriented database recovery. Ph.D. thesis, Kaiserslautern University of Technology, Germany (2017). http://www.dr.hut-verlag.de/978-3-8439-3297-4.html

  74. Sauer, C.: Modern techniques for transaction-oriented database recovery. In: Grust, T., Naumann, F., Böhm, A., Lehner, W., Härder, T., Rahm, E., Heuer, A., Klettke, M., Meyer, H. (eds.) Datenbanksysteme für Business, Technologie und Web (BTW 2019), 18. Fachtagung des GI-Fachbereichs “Datenbanken und Informationssysteme” (DBIS), 4.-8. März 2019, Rostock, Germany, Proceedings, LNI, vol. P-289, pp. 487–496. Gesellschaft für Informatik, Bonn (2019). https://doi.org/10.18420/btw2019-30

  75. Sauer, C., Graefe, G., Härder, T.: Instant restore after a media failure. In: Kirikova, M., Nørvåg, K., Papadopoulos, G.A. (eds.) Advances in Databases and Information Systems—21st European Conference, ADBIS 2017, Nicosia, Cyprus, September 24–27, 2017, Proceedings, Lecture Notes in Computer Science, vol. 10509, pp. 311–325. Springer (2017). https://doi.org/10.1007/978-3-319-66917-5_21

  76. Sauer, C., Graefe, G., Härder, T.: Fineline: log-structured transactional storage and recovery. Proc. VLDB Endow. 11(13), 2249–2262 (2018). https://doi.org/10.14778/3275366.3275373. http://www.vldb.org/pvldb/vol11/p2249-sauer.pdf

  77. Schuster, A.: Introducing the microsoft vista event log file format. Digit. Investig. 4(Supplement), 65–72 (2007). https://doi.org/10.1016/j.diin.2007.06.015

    Article  Google Scholar 

  78. Sikka, V., Färber, F., Lehner, W., Cha, S.K., Peh, T., Bornhövd, C.: Efficient transaction processing in SAP HANA database: the end of a column store myth. In: Candan, K.S., Chen, Y., Snodgrass, R.T., Gravano, L., Fuxman, A. (eds.) Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2012, Scottsdale, AZ, USA, May 20–24, 2012, pp. 731–742. ACM (2012). https://doi.org/10.1145/2213836.2213946

  79. Stonebraker, M., Weisberg, A.: The voltdb main memory DBMS. IEEE Database Eng. Bull. 36(2), 21–27 (2013). http://sites.computer.org/debull/A13june/VoltDB1.pdf

  80. Strickland, J.P., Uhrowczik, P.P., Watts, V.L.: IMS/VS: an evolving system. IBM Syst. J. 21(3), 490–510 (1982). https://doi.org/10.1147/sj.214.0490

    Article  Google Scholar 

  81. Tan, K., Cai, Q., Ooi, B.C., Wong, W., Yao, C., Zhang, H.: In-memory databases: challenges and opportunities from software and hardware perspectives. ACM Sigmod Record 44(2), 35–40 (2015). https://doi.org/10.1145/2814710.2814717

    Article  Google Scholar 

  82. Thomson, A., Diamond, T., Weng, S., Ren, K., Shao, P., Abadi, D.J.: Calvin: fast distributed transactions for partitioned database systems. In: Candan, K.S., Chen, Y., Snodgrass, R.T., Gravano, L., Fuxman, A., (eds.) Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2012, Scottsdale, AZ, USA, May 20–24, 2012, pp. 1–12. ACM (2012). https://doi.org/10.1145/2213836.2213838

  83. Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: Kaminsky, M., Dahlin, M. (eds.) ACM SIGOPS 24th Symposium on Operating Systems Principles, SOSP ’13, Farmington, PA, USA, November 3–6, 2013, pp. 18–32. ACM (2013). https://doi.org/10.1145/2517349.2522713

  84. Weikum, G., Vossen, G.: Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery. Morgan Kaufmann (2002)

  85. Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., Schaffner, J.: Simd-scan: ultra fast in-memory table scan using on-chip vector processing units. Proc. VLDB Endow. 2(1), 385–394 (2009). https://doi.org/10.14778/1687627.1687671. http://www.vldb.org/pvldb/vol2/vldb09-327.pdf

  86. Wu, Y., Guo, W., Chan, C., Tan, K.: Fast failure recovery for main-memory dbmss on multicores. In: Salihoglu, S., Zhou, W., Chirkova, R., Yang, J., Suciu, D. (eds.) Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14–19, 2017, pp. 267–281. ACM (2017). https://doi.org/10.1145/3035918.3064011

  87. Yadava, H.: The Berkeley DB Book. Apress (2007)

  88. Yahoo Labs: Home—research.yahoo.com (2023). https://research.yahoo.com

  89. Yao, C., Agrawal, D., Chen, G., Ooi, B.C., Wu, S.: Adaptive logging: optimizing logging and recovery costs in distributed in-memory databases. In: Özcan, F., Koutrika, G., Madden, S. (eds.) Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26–July 01, 2016, pp. 1119–1134. ACM (2016). https://doi.org/10.1145/2882903.2915208

  90. YCSB GitHub: Home. brianfrankcooper/ycsb wiki. github (2023). https://github.com/brianfrankcooper/YCSB/wiki

  91. Zhang, H., Chen, G., Ooi, B.C., Tan, K., Zhang, M.: In-memory big data management and processing: a survey. IEEE Trans. Knowl. Data 27(7), 1920–1948 (2015). https://doi.org/10.1109/TKDE.2015.2427795

  92. Zhang, J., Yao, Z., Feng, J.: Ncredis: an nvm-optimized redis with memory caching. In: Strauss, C., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) Database and Expert Systems Applications—32nd International Conference, DEXA 2021, Virtual Event, September 27–30, 2021, Proceedings, Part II, Lecture Notes in Computer Science, vol. 12924, pp. 70–76. Springer (2021). https://doi.org/10.1007/978-3-030-86475-0_7

  93. Zhang, Y., Swanson, S.: A study of application performance with non-volatile main memory. In: IEEE 31st Symposium on Mass Storage Systems and Technologies, MSST 2015, Santa Clara, CA, USA, May 30–June 5, 2015, pp. 1–10. IEEE Computer Society (2015). https://doi.org/10.1109/MSST.2015.7208275

  94. Zheng, W., Tu, S., Kohler, E., Liskov, B.: Fast databases with fast durability and recovery through multicore parallelism. In: Flinn, J., Levy, H. (eds.) 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI ’14, Broomfield, CO, USA, October 6–8, 2014, pp. 465–477. USENIX Association (2014). https://www.usenix.org/conference/osdi14/technical-sessions/presentation/zheng_wenting

Download references

Author information

Authors and Affiliations

  1. Federal University of Piaui, Teresina, PI, Brazil

    Arlino Magalhaes

  2. Federal University of Ceara, Fortaleza, CE, Brazil

    Angelo Brayner & Jose Maria Monteiro

Authors
  1. Arlino Magalhaes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angelo Brayner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Maria Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arlino Magalhaes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Magalhaes, A., Brayner, A. & Monteiro, J.M. MM-DIRECT. The VLDB Journal 33, 859–882 (2024). https://doi.org/10.1007/s00778-024-00846-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-024-00846-z

Keywords

Search

Navigation