Skip to main content
SpringerLink
Log in

Understanding the performance of storage class memory file systems in the NUMA architecture

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Recent developments in storage class memory (SCM) such as PCM, MRAM, resistive RAM (RRAM), and spin-transfer torque (STT)-RAM have strengthened their leadership as storage media for memory-based file systems. Traditional Linux memory-based file systems such as Ramfs and Tmpfs utilize the Linux page cache as a file system. These file systems have unnecessary overheads when adopted for SCM file system. Therefore, we propose a new memory-based file system using Memory Zone Partitioning called ZonFS, by extending the Linux Ramfs. In particular, we define a storage zone for SCM, modify the Ramfs to allocate a file system page from SCM. ZonFS avoids running Linux VM kernel codes such as (i) inserting pages allocated from SCM into the LRU list for VM page replacement and (ii) checking dirty pages for write-back to disk. Our extensive evaluations indicate that ZonFS has up to 9.1 and 14.1% higher I/O throughputs than native Ramfs and Tmpfs. Moreover, we also analyze performance behavior of ZonFS under the non-uniform memory access architecture of SCMs on a 40 manycore machine with various configurations such as file sharing level and file stripping level. Our evaluations show that memory controller contention and inter-node link congestion significantly affect the file system’s performance and scalability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Halupka, D., Huda, S., Song, W., Sheikholeslami, A., Tsunoda, K., Yoshida, C., Aoki, M.: Negative-resistance read and write schemes for STT-MRAM in 0.13 \(\upmu\)m CMOS. In: 2010 IEEE International Solid-State Circuits Conference—(ISSCC), February 2010, pp. 256–257

  2. Oh, H.R., Cho, B.H., Cho, W.Y., Kang, S., Choi, B.G., Kim, H.J., Kim, K., Kim, D., Kwak, C., Byun, H.G., Jeong, G.T., Jeong, H.: Enhanced write performance of a 64-Mb phase-change random access memory. IEEE J. Solid-State Circ. 41(1), 122–126 (2006)

  3. Chen, Z., Wu, H., Gao, B., Yao, P., Li, X., Qian, H.: Neuromorphic computing based on resistive RAM. In: Proceedings of the on Great Lakes Symposium on VLSI 2017, GLSVLSI ’17, New York, NY, USA, pp. 311–315. ACM (2017)

  4. Intel: Revolutionizing Memory and Storage. https://www.intel.com/content/www/us/en/architecture-and-technology/intel-optane-technology.html

  5. Dhiman, G., Ayoub, R., Rosing, T.: PDRAM: a hybrid PRAM and DRAM main memory system. In: Proceedings of the 46th Annual Design Automation Conference, DAC ’09, New York, NY, USA, pp. 664–469. ACM (2009)

  6. Kgil, T., Roberts, D., Mudge, T.: Improving NAND flash based disk caches. In: Proceedings of the 35th Annual International Symposium on Computer Architecture, ISCA ’08, Washington, DC, USA, pp. 327–338. IEEE Computer Society (2008)

  7. Qureshi, M.K., Srinivasan, V., Rivers, J.A.: Scalable high performance main memory system using phase-change memory technology. In: Proceedings of the 36th Annual International Symposium on Computer Architecture, ISCA ’09, New York, NY, USA, pp. 24–33. ACM (2009)

  8. Kim, J.W., Kim, J.-H., Khan, A., Kim, Y., Park, S.: ZonFS: a storage class memory file system with memory zone partitioning on Linux. In: 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS* W), pp. 277–282. IEEE (2017)

  9. Bovet, D., Cesati, M.: Understanding the Linux Kernel. O’Reilly and Associates, Inc., Sebastopol (2005)

    Google Scholar 

  10. Xu, J., Zhang, L., Memaripour, A., Gangadharaiah, A., Borase, A., Da Silva, T.B., Swanson, S., Rudoff, A.: NOVA-Fortis: a fault-tolerant non-volatile main memory file system. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp. 478–496. ACM (2017)

  11. Capps, D. IOzone Filesystem Benchmark. http://www.iozone.org

  12. Natarajan, C., Christenson, B., Briggs, F.: A study of performance impact of memory controller features in multi-processor server environment. In: Proceedings of the 3rd Workshop on Memory Performance Issues: In Conjunction with the 31st International Symposium on Computer Architecture, pp. 80–87. ACM (2004)

  13. Akram, S., Marazakis, M., Bilas, A.: NUMA implications for storage I/O throughput in modern servers. In: 3rd Workshop on Computer Architecture and Operating System Co-design (CAOS12) (2012)

  14. Mavridis, S., Sfakianakis, Y., Papagiannis, A., Marazakis, M., Bilas, A.: Jericho: achieving scalability through optimal data placement on multicore systems. In: 2014 30th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10. IEEE (2014)

  15. Chandru, V., Mueller, F.: Reducing NoC and memory contention for manycores. In: International Conference on Architecture of Computing Systems, pp. 293–305. Springer (2016)

  16. Intel: Intel Xeon Processor E5-4640 v2

  17. Intel: Intel Memory Latency Checker v3.5. https://software.intel.com/en-us/articles/intelr-memory-latency-checker

  18. Cheng, Y., Chen, W., Wang, Z., Yu, X.: Performance-monitoring-based traffic-aware virtual machine deployment on NUMA systems. IEEE Syst. J. 11(2), 973–982 (2017)

    Article  Google Scholar 

  19. Son, H., Lee, S., Won, Y.: Effect of timer interrupt interval on file system synchronization overhead. In: 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), pp. 99–102. IEEE (2016)

  20. Redhat: Configuring atime Update

  21. Min, C., Kashyap, S., Maass, S., Kim, T.: Understanding manycore scalability of file systems. In: USENIX Annual Technical Conference, pp. 71–85 (2016)

  22. Condit, J., Nightingale, E.B., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better I/O through byte-addressable, persistent memory. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP ’09, New York, NY, USA, pp. 133–146. ACM (2009)

  23. Wang, A.-I.A., Kuenning, G., Reiher, P., Popek, G.: The Conquest file system: better performance through a disk/persistent-RAM hybrid design. Trans. Storage 2(3), 309–348 (2006)

    Article  Google Scholar 

  24. Wu, X., Narasimha Reddy, A.L.: SCMFS: a file system for storage class memory. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’11, New York, NY, USA, pp. 39:1–39:11. ACM (2011)

  25. Xu, J., Swanson, S.: NOVA: a log-structured file system for hybrid volatile/non-volatile main memories. In: FAST, pp. 323–338 (2016)

  26. Liu, Z., Sha, E.H.-M., Chen, X., Jiang, W., Zhuge, Q.: Performance optimization for in-memory file systems on NUMA machines. In: 2016 17th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT), pp. 7–12. IEEE (2016)

Download references

Acknowledgements

We thank Mr. Jae-Hoon Kim for his initial ZonFS implementation. We also thank Dr. Amol Jaikar for his careful proofreading and constructive comments that have significantly improved the paper. This work was supported by Institute for Information and Communications Technology Promotion (IITP) Grant funded by the Korea Government (MSIT) (No. 2014-0-00035, Research on High Performance and Scalable Many-Core Operating System).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Sogang University, 35 Baekreom-ro, Mapo-gu, Seoul, Korea

    Jangwoong Kim, Youngjae Kim, Awais Khan & Sungyong Park

Authors
  1. Jangwoong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youngjae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Awais Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sungyong Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sungyong Park.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J., Kim, Y., Khan, A. et al. Understanding the performance of storage class memory file systems in the NUMA architecture. Cluster Comput 22, 347–360 (2019). https://doi.org/10.1007/s10586-018-2833-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-2833-4

Keywords

Search

Navigation