Abstract
Full-path-indexed file systems use a key-value database to store the full path names of files and their metadata. With this pattern, the I/O efficiency can be improved because data is placed on persistent storage in scan order. However, it introduces intolerable overhead on renaming a directory because of the modification on the full path names of files under that directory. In this paper, we introduce prefix replacement mechanism on B+-tree to accelerate renaming directories on full-path-indexed file systems. It consists of three steps: pre-scan prefix deletion, key replacement and floating-split bulk insertion. Unnecessary searches and compares are reduced in these mechanisms. We use Kyoto Cabinet as the key-value database, and implement prefix replacement mechanism on it. We run tests on two benchmarks, the first is generated by Mdtest [18], and the second is the source code of Linux [19]. Compared with LocoFS [4], one kind of full-path-indexed file system, our design is about 5\(\times \) faster to rename large directories, and the performance is basically same on small directories.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
McKusick, M.K., Joy, W.N., Leffler, S.J., Fabry, R.S.: A fast file system for UNIX. ACM Trans. Comput. Syst. 2(3), 181–197 (1984)
Tao, X., Alei, L.: Small file access optimization based on GlusterFS. In: Proceedings of 2014 International Conference on Cloud Computing and Internet of Things, pp. 101–104 (2014)
Jannen, W., Yuan, J., Yang, Z., Esmet, J., Esmet, J., Jiao, Y.: BetrFS: a right-optimized write-optimized file system. In: 13th Conference on File and Storage Technologies, pp. 301–315 (2015)
Li, S., Lu, Y., Shu, J., Hu, Y., Li, T.: LocoFS: a loosely-coupled metadata service for distributed file systems. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 4–15 (2017)
Esmet, J., Bender, M.A., Farach-Colton, M., Kuszmaul, B.C.: The TokuFS streaming file system. In: USENIX Conference on Hot Topics in Storage & File Systems (2012)
Jannen, W., Yuan, J., Zhan, Y., Akshintala, A., Esmet, J., Jiao, Y., et al.: BetrFS: write-optimization in a kernel file system. ACM Trans. Storage 11(4), 1–29 (2015)
Ren, K., Gibson, G.A.: TABLEFS: enhancing metadata efficiency in the local file system. In: USENIX Annual Technical Conference, pp. 145–156 (2013)
Yuan, J., Zhan, Y., Jannen, W., Pandey, P., Akshintala, A., Chandnani, K., et al.: Optimizing every operation in a write-optimized file system. In: USENIX Conference on File & Storage Technologies (2016)
Zhan, Y., et al.: The full path to full-path indexing. In: Proceedings of the 16th USENIX Conference on File and Storage Technologies, pp. 123–138 (2018)
Ren, K., Zheng, Q., Patil, S., Gibson, G.: IndexFS: scaling file system metadata performance with stateless caching and bulk insertion. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 237–248 (2014)
O’Neil, P., Cheng, E., Gawlick, D., O’Neil, E.: The log-structured merge-tree (LSM-tree). Acta Informatica 33(4), 351–385 (1996)
Shetty, P.J., Spillane, R.P., Malpani, R.R., et al.: Building workload-independent storage with VT-trees. Presented as Part of the 11th USENIX Conference on File and Storage Technologies, pp. 17–30 (2013)
Sears, R., Ramakrishnan, R.: bLSM: a general purpose log structured merge tree. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 217–228 (2012)
Brodal, G.S., Fagerberg, R.: Lower bounds for external memory dictionaries. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 546–554 (2003)
Davies, A., Orsaria, A.: Scale out with GlusterFS. Linux J. 2013(235), 1 (2013)
Patil, S., Gibson, G.A.: Scale and concurrency of giga+: file system directories with millions of files. In: USENIX Conference on File and Storage Technologies, vol. 11, p. 13 (2011)
Brandt, S.A., Miller, E.L., et al.: Efficient metadata management in large distributed storage systems. In: Mass Storage Systems and Technologies, pp. 290–298 (2003)
MDTEST Benchmark. https://github.com/MDTEST-LANL/mdtest. Accessed 16 Apr 2019
Linux release. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/snapshot/linux-4.20-rc5.tar.gz. Accessed 16 Apr 2019
Lensing, P.H., Cortes, T., Hughes, J., Brinkmann, A.: File system scalability with highly decentralized metadata on independent storage devices. In: 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 366–375 (2016)
Shi, X., Lin, H., Jin, H., et al.: Giraffe: a scalable distributed coordination service for large-scale systems. In: 2014 IEEE International Conference on Cluster Computing, pp. 38–47 (2014)
Zheng, Q., Ren, K., Gibson, G., et al.: DeltaFS: exascale file systems scale better without dedicated servers. In: Proceedings of the 10th Parallel Data Storage Workshop, pp. 1–6 (2015)
Lensing, P.H., Cortes, T., Brinkmann, A.: Direct lookup and hash-based metadata placement for local file systems. In: Proceedings of the 6th International Systems and Storage Conference, p. 5 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, L., Lu, Y., Li, S., Yang, F., Shu, J. (2019). Reducing Rename Overhead in Full-Path-Indexed File System. In: Yew, PC., Stenström, P., Wu, J., Gong, X., Li, T. (eds) Advanced Parallel Processing Technologies. APPT 2019. Lecture Notes in Computer Science(), vol 11719. Springer, Cham. https://doi.org/10.1007/978-3-030-29611-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-29611-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29610-0
Online ISBN: 978-3-030-29611-7
eBook Packages: Computer ScienceComputer Science (R0)
