Abstract
Log-Structured Merge tree is a write optimized persistent storage engine consisting of memory buffer and multiple layers of disk files. Log-Structured Merge tree is built for block devices and suffers from write stalls due to its frequent internal L0–L1 compaction operation. Emerging storage hardware non-volatile memory (NVM) brings opportunities for optimization of LSM-tree with its byte-addressability, high bandwidth, low latency. In this paper, we introduce NVLSM, a novel LSM-tree design based on hybrid storage of NVM with SSD and NVM with HDD to improve write throughput of Log-Structured Merge tree. Experimental results show that NVLSM achieves 1.11x higher random write throughput compared to the baseline model, and the read performance is comparable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
2016: Persistent Memory Development Kit, 27 May 2021. https://pmem.io/pmdk
Caulfield, A.M., De, A., Coburn, J., Mollow, T.I., Gupta, R.K., Swanson, S.: Moneta: a high-performance storage array architecture for next-generation non-volatile memories. In: 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture, pp. 385–395 (2010). https://doi.org/10.1109/MICRO.2010.33
Chai, Y., Chai, Y., Wang, X., Wei, H., Bao, N., Liang, Y.: LDC: a lower-level driven compaction method to optimize SSD-oriented key-value stores. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 722–733 (2019). https://doi.org/10.1109/ICDE.2019.00070
Eisenman, A., et al.: Reducing DRAM footprint with NVM in Facebook. In: Proceedings of the Thirteenth EuroSys Conference, pp. 1–13 (2018)
Facebook, RocksDB: RocksDB: a library that provides an embeddable, persistent key-value store for fast storage (2019). https://github.com/facebook/rocksdb
Huai, Y.: Spin-Transfer Torque MRAM (STTMRAM): Challenges and Prospects (2008)
Intel and micron produce breakthrough memory technology (2019). https://newsroom.intel.com/news-releases/intel-and-micron-produce-breakthrough-memory-technology/. Accessed 27 May 2021
Condit, J., et al.: Better I/O through byte-addressable, PersistentMemory (n.d.)
Kannan, S., Bhat, N., Gavrilovska, A., Arpaci-Dusseau, A., Arpaci-Dusseau, R.: Redesigning LSMs for nonvolatile memory with NoveLSM. In: 2018 {USENIX} Annual Technical Conference ({USENIX}{ATC} 2018), pp. 993–1005 (2018)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. ACMSIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010)
Li, J., Pavlo, A., Dong, S.: NVMRocks: RocksDB on nonvolatile memory systems (2017)
O’Neil, P., Cheng, E., Gawlick, D., O’Neil, E.: The log structured merge-tree (LSM-tree). Acta Informatica 33(4), 351–385 (1996)
Ghemawat, S., Dean, J.: LevelDB: a fast key value storage library (2016). https://github.com/google/leveldb. Accessed 27 May 2021
Wong, H.S.P., et al.: PhaseChange memory. Proc. IEEE 98(12), 2201–2227 (2010). https://doi.org/10.1109/JPROC.2010.2070050
Yao, T., et al.: MatrixKV: reducing write stalls and write amplification in LSM tree based KVStores with matrix container in NVM. In: 2020 USENIX Annual Technical Conference (USENIX ATC 2020), pp. 17–31. USENIX Association (2020)
Acknowledgments
This work is supported by The National Key Research and Development Program of China (2019YFB1804502). NSFC: 61832020. Guangdong Natural Science Foundation (2018B030312002) the Major Program of Guangdong Basic and Applied Research: 2019B030302002. Supported by the Program for Guangdong Introducing Innovative and Entrepreneurial Teams under Grant NO. 2016ZT06D211.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhuang, Z., Wang, Y., Bai, S., Liu, Y., Chen, Z., Xiao, N. (2023). NVLSM: Virtual Split Compaction on Non-volatile Memory in LSM-Tree KV Stores. In: Tang, L.C., Wang, H. (eds) Big Data Management and Analysis for Cyber Physical Systems. BDET 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 150. Springer, Cham. https://doi.org/10.1007/978-3-031-17548-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-17548-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17547-3
Online ISBN: 978-3-031-17548-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)