User-Space I/O for \(\mu \)s-level Storage Devices

  • Anastasios Papagiannis
  • Giorgos Saloustros
  • Manolis Marazakis
  • Angelos Bilas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9945)

Abstract

System software overheads in the I/O path, including VFS and file system code, become more pronounced with emerging low-latency storage devices. Currently, these overheads constitute the main bottleneck in the I/O path and they limit efficiency of modern storage systems. In this paper we present Iris, a new I/O path for applications, that minimizes overheads from system software in the common I/O path. The main idea is the separation of the control and data planes. The control plane consists of an unmodified Linux kernel and is responsible for handling data plane initialization and the normal processing path through the kernel for non-file related operations. The data plane is a lightweight mechanism to provide direct access to storage devices with minimum overheads and without sacrificing strong protection semantics. Iris requires neither hardware support from the storage devices nor changes in user applications. We evaluate our early prototype and we find that it achieves on a single core up to \(1.7\times \) and \(2.2\times \) better read and write random IOPS, respectively, compared to the xfs and ext4 file systems. It also scales with the number of cores; using 4 cores Iris achieves \(1.84\times \) and \(1.96\times \) better read and write random IOPS, respectively.

Keywords

NVM I/O Storage systems Low latency Protection 

References

  1. 1.
    AMD: Secure Virtual Machine Architecture Reference ManualGoogle Scholar
  2. 2.
    Axboe, J.: Flexible I/O Tester (2005). https://github.com/axboe
  3. 3.
    Baumann, A., Barham, P., Dagand, P.E., Harris, T., Isaacs, R., Peter, S., Roscoe, T., Schüpbach, A., Singhania, A.: The multikernel: a new OS architecture for scalable multicore systems. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles SOSP 2009, pp. 29–44. ACM, New York (2009). http://doi.acm.org/10.1145/1629575.1629579
  4. 4.
    Belay, A., Bittau, A., Mashtizadeh, A., Terei, D., Mazières, D., Kozyrakis, C.: Dune: Safe user-level access to privileged cpu features. In: 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2012), pp. 335–348. USENIX, Hollywood (2012). https://www.usenix.org/conference/osdi12/technical-sessions/presentation/belay
  5. 5.
    Belay, A., Prekas, G., Klimovic, A., Grossman, S., Kozyrakis, C., Bugnion, E.: IX: a protected dataplane operating system for high throughput and low latency. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 49–65. USENIX Association, Broomfield, October 2014. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/belay
  6. 6.
    Brodal, G.S., Fagerberg, R.: Lower bounds for external memory dictionaries. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2003, pp. 546–554. Society for Industrial and Applied Mathematics, Philadelphia (2003). http://dl.acm.org/citation.cfm?id=644108.644201
  7. 7.
    Caulfield, A.M., Mollov, T.I., Eisner, L.A., De, A., Coburn, J., Swanson, S.: Providing safe, user space access to fast, solid state disks. In: Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVII, pp. 387–400. ACM, New York (2012). http://doi.acm.org/10.1145/2150976.2151017
  8. 8.
    Chen, F., Mesnier, M., Hahn, S.: A protected block device for persistent memory. In: 2014 30th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–12, June 2014Google Scholar
  9. 9.
    Coburn, J., Caulfield, A.M., Akel, A., Grupp, L.M., Gupta, R.K., Jhala, R., Swanson, S.: NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories. In: Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pp. 105–118. ACM, New York (2011). http://doi.acm.org/10.1145/1950365.1950380
  10. 10.
    Condit, J., Nightingale, E.B., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better i/o through byte-addressable, persistent memory. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP 2009, pp. 133–146. ACM, New York (2009). http://doi.acm.org/10.1145/1629575.1629589
  11. 11.
    Dieny, B., Sousa, R., Prenat, G., Ebels, U.: Spin-dependent phenomena and their implementation in spintronic devices. In: International Symposium on VLSI Technology, Systems and Applications VLSI-TSA 2008, pp. 70–71, April 2008Google Scholar
  12. 12.
    DPDK: Data plane development kit (2016). http://dpdk.org/
  13. 13.
    Dulloor, S.R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., Jackson, J.: System software for persistent memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys 2014, pp. 15:1–15:15. ACM, New York (2014). http://doi.acm.org/10.1145/2592798.2592814
  14. 14.
    FusioIO: ioDrive2/ioDrive2 Duo Datasheet (2014). http://www.fusionio.com/load/-media-/2rezss/docsLibrary/FIO_DS_ioDrive2.pdf
  15. 15.
    Ho, Y., Huang, G., Li, P.: Nonvolatile memristor memory: device characteristics and design implications. In: IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers, ICCAD 2009, pp. 485–490, November 2009Google Scholar
  16. 16.
    Intel: Persistent memory block driver (PMBD) v0.9. (2013). https://github.com/linux-pmbd/pmbd
  17. 17.
    Kutch, P.: PCI-SIG SR-IOV primer: An introduction to SR-IOV technology, Intel application note, 321211–002 (2011)Google Scholar
  18. 18.
    Papagiannis, A., Saloustros, G., González-Férez, P., Bilas, A.: Tucana: Design and implementation of a fast and efficient scale-up key-value store. In: 2016 USENIX Annual Technical Conference (USENIX ATC 2016), USENIX Association, Denver, June 2016. https://www.usenix.org/conference/atc16/technical-sessions/presentation/papagiannis
  19. 19.
    Peter, S., Li, J., Zhang, I., Ports, D.R.K., Anderson, T., Krishnamurthy, A., Zbikowski, M., Woos, D.: Towards high-performance application-level storage management. In: 6th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2014). USENIX Association, Philadelphia, June 2014. https://www.usenix.org/conference/hotstorage14/workshop-program/presentation/peter
  20. 20.
    Peter, S., Li, J., Zhang, I., Ports, D.R.K., Woos, D., Krishnamurthy, A., Anderson, T., Roscoe, T.: Arrakis: the operating system is the control plane. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 1–16. USENIX Association, Broomfield, October 2014. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/peter
  21. 21.
    Raoux, S., Burr, G., Breitwisch, M., Rettner, C., Chen, Y., Shelby, R., Salinga, M., Krebs, D., Chen, S.H., Lung, H., Lam, C.: Phase-change random access memory: a scalable technology. IBM J. Res. Dev. 52(4.5), 465–479 (2008)CrossRefGoogle Scholar
  22. 22.
    Rodeh, O.: B-trees, shadowing, and clones. Trans. Storage 3(4), 2: 1–2: 27 (2008). http://doi.acm.org/10.1145/1326542.1326544 CrossRefGoogle Scholar
  23. 23.
    Uhlig, R., Neiger, G., Rodgers, D., Santoni, A., Martins, F., Anderson, A., Bennett, S., Kagi, A., Leung, F., Smith, L.: Intel virtualization technology. Computer 38(5), 48–56 (2005)CrossRefGoogle Scholar
  24. 24.
    Volos, H., Nalli, S., Panneerselvam, S., Varadarajan, V., Saxena, P., Swift, M.M.: Aerie: Flexible file-system interfaces to storage-class memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys 2014, pp. 14:1–14:14. ACM, New York (2014). http://doi.acm.org/10.1145/2592798.2592810
  25. 25.
    Volos, H., Tack, A.J., Swift, M.M.: Mnemosyne: lightweight persistent memory. In: Proceedings of the Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVI, pp. 91–104, ACM, New York (2011). http://doi.acm.org/10.1145/1950365.1950379
  26. 26.
    Wu, X., Reddy, A.L.N.: SCMFS: a file system for storage class memory. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2011, pp. 39:1–39:11. ACM, New York (2011). http://doi.acm.org/10.1145/2063384.2063436
  27. 27.
    Xu, J., Swanson, S.: Nova: a log-structured file system for hybrid volatile/non-volatile main memories. In: 14th USENIX Conference on File and Storage Technologies (FAST 2016), pp. 323–338. USENIX Association, Santa Clara, February 2016. https://www.usenix.org/conference/fast16/technical-sessions/presentation/xu

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Anastasios Papagiannis
    • 1
    • 2
  • Giorgos Saloustros
    • 1
  • Manolis Marazakis
    • 1
  • Angelos Bilas
    • 1
    • 2
  1. 1.Institute of Computer Science, FORTH (ICS)HeraklionGreece
  2. 2.Department of Computer ScienceUniversity of CreteHeraklionGreece

Personalised recommendations