Adaptive Selection of Partition Size for Supercomputer Requests

  • Walfredo Cirne
  • Francine Berman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1911)

Abstract

In this paper, we show how application scheduling can be used to reduce the turn-around time of supercomputer jobs. Our approach focuses on the use of SA, an AppLeS application scheduler, to adaptively craft the request to be submitted to the supercomputer based on the current state of the system. We demonstrate that SA significantly improves a job’s turn-around time in a variety of scenarios. We also identify how the state of the system, the characteristics of the job, and the quality of the information made available to SA influence its performance.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Kento Aida, Hironori Kasahara, and Seinosuke Narita. Job Scheduling Scheme for Pure Space Sharing among Rigid Jobs. In Job Scheduling Strategies for Parallel Processing, Springer-Verlag, Lecture Notes in Computer Science Vol. 1459.Google Scholar
  2. [2]
    Fran Berman, Richard Wolski, Silvia Figueira, Jennifer Schopf, and Gary Shao. Application Level Scheduling on Distributed Heterogeneous Networks. Supercomputing’96. http://www-cse.ucsd.edu/groups/hpcl/apples/hetpubs.html
  3. [3]
    Fran Berman and Rich Wolski. The AppLeS Project: A Status Report. In Proceedings of the 8th NEC Research Symposium, Berlin, Germany, May 1997. http://www.cs.ucsd.edu/groups/hpcl/apples/hetpubs.html
  4. [4]
    Walfredo Cirne and Fran Berman. Application Scheduling over Supercomputers: A Proposal. UCSD-CS99-631 Technical Report, October 1999. http://www.cs.ucsd.edu/Research/TechReports/dienst.html
  5. [5]
    Steve J. Chapin, Dimitrios Katramatos, and John Karpovich. The Legion Resource Management System. IPPS Workshop on Job Scheduling Strategies for Parallel Processing, pp. 105–114, San Juan, Puerto Rico, 1999. http://www.cs.virginia.edu/∼chapin/papers/allpub.html
  6. [6]
    Su-Hui Chiang and Mary K. Vernon. Dynamic vs. Static Quantum-Based Parallel Processor Allocation. In Job Scheduling Strategies for Parallel Processing, Springer-Verlag, Lectures Notes in Compututer Science vol. 1162, pp. 200–223, 1996. http://www.cs.wisc.edu/∼suhui/suhui.html CrossRefGoogle Scholar
  7. [7]
    Allen B. Downey. A model for speedup of parallel programs. U.C. Berkeley Technical Report CSD-97-933. http://www.sdsc.edu/~downey/model/
  8. [8]
    Allen B. Downey. Using Queue Time Predictions for Processor Allocation. In Job Scheduling Strategies for Parallel Processing, Springer-Verlag, Lect. Notes Comput. Sci. vol. 1162, 1997. http://www.sdsc.edu/∼downey/predalloc/
  9. [9]
    A. B. Downey and D. G. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14–29, Mar 1999. http://www.cs.huji.ac.il/~feit/pub.html CrossRefGoogle Scholar
  10. [10]
    D. G. Feitelson and A. Mu’alem Weil. Utilization and predictability in scheduling the IBM SP2 with backfilling. In 12th Intl. Parallel Processing Symp., pp. 542–546, April 1998. http://www.cs.huji.ac.il/~feit/pub.html
  11. [11]
    D. G. Feitelson and L. Rudolph. Metrics and benchmarking for parallel job scheduling. In Job Scheduling Strategies for Parallel Processing, pp. 1–24, Springer-Verlag, 1998. Lecture Notes in Computer Science Vol. 1459. http://www.cs.huji.ac.il/∼feit/pub.html Google Scholar
  12. [12]
    Ian Foster and Carl Kesselman (editors). The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers. 1999.Google Scholar
  13. [13]
    I. Foster, C. Kesselman, C. Lee, R. Lindell, K. Nahrstedt, A. Roy. A Distributed Resource Management Architecture that Supports Advance Reservations and Co-Allocation. International Workshop on Quality of Service, 1999.http://www-fp.globus.org/documentation/papers.html
  14. [14]
    Jochen Krallmann, Uwe Schwiegelshohn, and Ramin Yahyapour. On the Design and Evaluation of Job Scheduling Algorithms. In Job Scheduling Strategies for Parallel Processing, Springer-Verlag, Lectures Notes in Compututer Science vol. 1659, 1999.Google Scholar
  15. [15]
    David Lifka. The ANL/IBM SP Scheduling System. In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (Eds.), Springer-Verlag, Lecture Notes in Computer Science Vol. 949, 1995.http://www.tc.cornell.edu/UserDoc/SP/Batch/what.html Google Scholar
  16. [16]
    Maui High Performance Computing Center. The Maui Scheduler Web Page. http://wailea.mhpcc.edu/maui/
  17. [17]
    Thu D. Nguyen, Raj Vaswani and John Zahorjan. Parallel Application Characterization for Multiprocessor Scheduling Policy Design. In Job Scheduling Strategies for Parallel Processing, Springer-Verlag, Lectures Notes in Computure Science vol. 1162, pp. 175–199, 1996. http://bauhaus.cs.washington.edu/homes/thu/papers/list.html CrossRefGoogle Scholar
  18. [18]
    Bill Nitzberg. Advance Reservations and Co-Scheduling Workshop Web Page. May 11, 1999. http://www.nas.nasa.gov/~nitzberg/sched-g/AdvRes_May99/index.html
  19. [19]
    Platform Computing Corp. Load Sharing Facility Web Page. http://www.platform.com/platform/platform.nsf/webpage/LSF?OpenDocument
  20. [20]
    Shava Smallen, Walfredo Cirne, Jaime Frey, Fran Berman, Rich Wolski, Mei-Hui Su, Carl Kesselman, Steve Young, and Mark Ellisman. Combining Workstations and Supercomputers to Support Grid Applications: The Parallel Tomography Experience. 9th Heterogeneous Computing Workshop, held in conjunction with IPDPS’2000, Cancun, Mexico, May 2000. http://www-cse.ucsd.edu/users/walfredo/resume.html#publications
  21. [21]
    Quinn Snell, Mark Clement, David Jackson, and Chad Grogory. The Performance Impact of Advance Reservation Meta-scheduling. 6th Workshop on Job Scheduling Strateties for Parallel Processing. In Conjunction with IPDPS 2000, Cancun, Mexico, May 1 2000.Google Scholar
  22. [23]
    D. Zotkin and P. Keleher. Job-Length Estimation and Performance in Backfilling Schedulers. 8th International Symposium on High Performance Distributed Computing (HPDC’99), Redondo Beach, California, USA, 3-6 Aug 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Walfredo Cirne
    • 1
  • Francine Berman
    • 1
  1. 1.Computer Science and Engineering University of CaliforniaSan Diego

Personalised recommendations