Skip to main content
SpringerLink
Log in

A Novel network request scheduler for a large scale storage system

  • Special Issue Paper
  • Published:
Computer Science - Research and Development

Abstract

This paper presents a novel Network Request Scheduler (NRS) for a large-scale, LustreTM storage system. It proposes a quantum-based, Object Based Round Robin (OBRR) NRS algorithm that reorders the execution of I/O requests per data object, presenting a workload to backend storage that can be optimized more easily. According to the drawback of static deadlines in large-scale workloads, it proposes a novel two-level deadline setting strategy that not only avoids starvation, but also guarantees that urgent I/O requests are serviced in a specified time period. Via a series of simulation experiments using a Lustre simulator, it demonstrates that I/O performance increases as much as 40% when using the OBRR NRS algorithm, and the two-level deadline setting strategy can avoid starvation and ensure that urgent I/O requests are serviced in the required time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lebre A, Huard G, Denneulin Y (2006) I/O Scheduling Service for Multi-Application Clusters. Cluster, IEEE Int. Conf. Cluster Computing, pp 1–10

  2. Crandall PE (1995) Input/output characteristics of scalable parallel applications. Proc Supercomputing ’95, San Diego, CA, December 1995, IEEE Computer Society Press

  3. Aneesh Kumar KV (2008) Ext4 block and inode allocator improvements. Proc Linux Symposium, Volume One, 23–26 July 2008 Ottawa, Ontario Canada

  4. Nieuwejaar N, Kotz D (1996) File-access characteristics of parallel scientific workloads. IEEE Trans Parallel Distrib Syst 7(10):1075–1089

    Article  Google Scholar 

  5. Ross R (2000) Reactive scheduling for parallel I/O systems, PhD thesis, Dept. of Electrical and Computer Engineering, Clemson University, Clemson, SC

  6. Seelam S, Romero R, Teller P, Buros B (2005) Enhancements to Linux I/O Scheduling. In: Proceedings of the Linux Symposium, Volume Two, 20–23 July 2005, Ottawa, Ontario

  7. Lustre simulator, https://bugzilla.lustre.org/show_bug.cgi?id=%2013634

Download references

Author information

Authors and Affiliations

  1. National Laboratory for Parallel and Distributed Processing, Changsha, China

    Yingjin Qian

  2. Sun Microsystems, Inc, Lustre Group, PH, USA

    Eric Barton, Tom Wang, Nirant Puntambekar & Andreas Dilger

Authors
  1. Yingjin Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Barton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tom Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nirant Puntambekar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andreas Dilger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingjin Qian.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Qian, Y., Barton, E., Wang, T. et al. A Novel network request scheduler for a large scale storage system . Comp. Sci. Res. Dev. 23, 143–148 (2009). https://doi.org/10.1007/s00450-009-0073-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00450-009-0073-9

Keywords

Search

Navigation