Abstract
Memory-mapped I/O (mmio) is emerging as a viable alternative for accessing directly-attached fast storage devices compared to explicit I/O with system calls. Mmio removes the need for costly lookups in the DRAM I/O cache for cache hits, as they are handled in hardware via the virtual memory mechanism. In this work we present HugeMap, a custom mmio path in the Linux kernel that uses huge pages for file-backed mappings to accelerate applications with sequential I/O access patterns or large I/O operations. HugeMap uses huge pages to reduce CPU processing in the kernel I/O path compared to regular mmap. We explore the benefits and trade-offs of huge pages in HugeMap using microbenchmarks, IOR, and an in-house persistent key-value store designed for mmio. Our experiments show up to \(3.7\times \) higher throughput and up to \(4.76\times \) lower system time, compared to regular page configurations.
Also with the Department of Computer Science, University of Crete, Greece.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amit, N.: Optimizing the TLB shootdown algorithm with page access tracking. In: 2017 USENIX Annual Technical Conference (USENIX ATC 2017), Santa Clara, CA. USENIX Association, July 2017
Brown, N.: Transparent huge pages in the page cache (2016). https://lwn.net/Articles/686690/
Choi, J., Kim, J., Han, H.: Efficient memory mapped file I/O for in-memory file systems. In: 9th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2017), Santa Clara, CA. USENIX Association (2017)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing. SoCC 2010. Association for Computing Machinery, New York (2010)
Corbet, J.: An introduction to compound pages. https://lwn.net/Articles/619514/
Harizopoulos, S., Abadi, D.J., Madden, S., Stonebraker, M.: OLTP through the looking glass, and what we found there. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. SIGMOD 2008, Association for Computing Machinery, New York (2008)
Kwon, Y., Yu, H., Peter, S., Rossbach, C.J., Witchel, E.: Coordinated and efficient huge page management with Ingens. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), Savannah, GA. USENIX Association, November 2016
Loewe, W., McLarty, T., Morrone, C.: The IOR Benchmark (2012). https://github.com/hpc/ior
Panwar, A., Bansal, S., Gopinath, K.: HawkEye: efficient fine-grained OS support for huge pages. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. ASPLOS 2019. Association for Computing Machinery, New York (2019)
Papagiannis, A., Saloustros, G., González-Férez, P., Bilas, A.: An efficient memory-mapped key-value store for flash storage. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC 2018. Association for Computing Machinery, New York (2018)
Papagiannis, A., Xanthakis, G., Saloustros, G., Marazakis, M., Bilas, A.: Optimizing memory-mapped I/O for fast storage devices. In: 2020 USENIX Annual Technical Conference (USENIX ATC 20), Boston, MA. USENIX Association (2020)
Song, N.Y., Son, Y., Han, H., Yeom, H.Y.: Efficient memory-mapped I/O on fast storage device. ACM Trans. Storage 12(4) 1–27 (2016)
The Linux Kernel Documentation: HugeTLB Pages. https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html
The Linux Kernel Documentation: Transparent Huge Page Support. https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html
Van Essen, B., Hsieh, H., Ames, S., Pearce, R., Gokhale, M.: DI-MMAP-a scalable memory-map runtime for out-of-core data-intensive applications. Cluster Comput. 18(1), 15–28 (2015)
Acknowledgements
We thankfully acknowledge the support of the European Commission through the H2020 project EVOLVE (GA825061) and the support of the Hellenic General Secretariat for Research and Technology through the MIA-RTDI Research-Create-Innovate project SentiTour (T1EDK-02857). Anastasios Papagiannis is also supported by the Facebook Graduate Fellowship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Malliotakis, I., Papagiannis, A., Marazakis, M., Bilas, A. (2021). HugeMap: Optimizing Memory-Mapped I/O with Huge Pages for Fast Storage. In: Balis, B., et al. Euro-Par 2020: Parallel Processing Workshops. Euro-Par 2020. Lecture Notes in Computer Science(), vol 12480. Springer, Cham. https://doi.org/10.1007/978-3-030-71593-9_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-71593-9_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71592-2
Online ISBN: 978-3-030-71593-9
eBook Packages: Computer ScienceComputer Science (R0)