A Self-Tuning Job Scheduler Family with Dynamic Policy Switching

  • Achim Streit
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2537)

Abstract

The performance of job scheduling policies strongly depends on the properties of the incoming jobs. If the job characteristics often change, the scheduling policy should follow these changes. For this purpose the dynP job scheduler family has been developed. The idea is to dynamically switch the scheduling policy during runtime. In a basic version the policy switching is controlled by two parameters. The basic concept of the self-tuning dynP scheduler is to compute virtual schedules for each policy in every scheduling step. That policy is chosen which generates the ‚best’ schedule. The performance of the self-tuning dynP scheduler no longer depends on a adequate setting of the input parameters.

We use a simulative approach to evaluate the performance of the self-tuning dynP scheduler and compare it with previous results. To drive the simulations we use synthetic job sets that are based on trace information from four computing centers (CTC, KTH, PC2, SDSC) with obviously different characteristics.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    C. Ernemann, V. Hamscher, U. Schwiegelshohn, A. Streit, and R. Yahyapour. On Advantages of Grid Computing for Parallel Job Scheduling. In Proceedings of the 2nd IEEE International Symposium on Cluster Computing and the Grid (CCGRID 2002), 2002. 2, 7Google Scholar
  2. [2]
    D.G. Feitelson and M.A. Jette. Improved Utilization and Responsiveness with Gang Scheduling. In D.G. Feitelson and L. Rudolph, editor, Proc. of 3rd Workshop on Job Scheduling Strategies for Parallel Processing, volume 1291, pages 238–262. Springer Verlag, 1997. 1Google Scholar
  3. [3]
    D.G. Feitelson and M. Naaman. Self-Tuning Systems. In IEEE Software 16(2), pages 52–60, April/May 1999. 3, 4CrossRefGoogle Scholar
  4. [4]
    D.G. Feitelson and A. Mu’alem Weil. Utilization and Predictability in Scheduling the IBM SP2 with Backfilling. In Proceedings of the 1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing (IPPS/SPDP-98), pages 542–547, Los Alamitos, March 1998. IEEE Computer Society. 1, 3, 10Google Scholar
  5. [5]
    H. Franke, J. Jann, J. Moreira, P. Pattnaik, and M. Jette. An Evaluation of Parallel Job Scheduling for ASCI Blue-Pacific. In Proceedings of SC’99, Portland, Oregon, pages 11–18. ACM Press and IEEE Computer Society Press, 1999. 1Google Scholar
  6. [6]
    J. Gehring and F. Ramme. Architecture-Independent Request-Scheduling with Tight Waiting-Time Estimations. In D.G. Feitelson and L. Rudolph, editor, Proc. of 2nd Workshop on Job Scheduling Strategies for Parallel Processing, volume 1162, pages 65–80. Springer Verlag, 1996. 2Google Scholar
  7. [7]
    V. Hamscher, U. Schwiegelshohn, A. Streit, and R. Yahyapour. Evaluation of job-scheduling strategies for grid computing. In Proceedings of 1st IEEE/ACM International Workshop on Grid Computing (Grid 2000), volume 1971, pages 191–202, 2000. 7Google Scholar
  8. [8]
    A. Keller and A. Reinefeld. Anatomy of a Resource Management System for HPC Clusters. In Annual Review of Scalable Computing, vol. 3, Singapore University Press, pages 1–31, 2001. 2MATHGoogle Scholar
  9. [9]
    J. Krallmann. A Quantative Analysis of Scheduling-Algorithms for Parallel Machines (in German). Master’s thesis, Dortmund University, 1998. 7Google Scholar
  10. [10]
    J. E. Moreira, H. Franke, W. Chan, and L. L. Fong. A Gang-Scheduling System for ASCI Blue-Pacific. In Proceedings of the 7th International Conference in High-Performance Computing and Networking (HPCN’99), volume 1593 of Lecture Notes in Computer Science, pages 831–840. Springer, 1999. 1Google Scholar
  11. [11]
    A.W. Mu’alem and D. G. Feitelson. Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. In IEEE Trans. Parallel & Distributed Systems 12(6), pages 529–543, June 2001. 1CrossRefGoogle Scholar
  12. [12]
    F. Ramme and K. Kremer. Scheduling a Metacomputer by an Implicit Voting System. In 3 rd Int. IEEE Symposium on High-Performance Distributed Computing, pages 106–113, 1994. 2Google Scholar
  13. [13]
    J. Skovira, W. Chan, H. Zhou, and D. Lifka. The EASY — LoadLeveler API Project. In D.G. Feitelson and L. Rudolph, editor, Proc. of 2nd Workshop on Job Scheduling Strategies for Parallel Processing, volume 1162, pages 41–47. Springer Verlag, 1996. 1Google Scholar
  14. [14]
    W. Smith, I. Foster, and V. Taylor. Predicting Application Run Times Using Historical Information. In D.G. Feitelson and L. Rudolph, editor, Proc. of 4th Workshop on Job Scheduling Strategies for Parallel Processing, volume 1459, pages 122–142. Springer Verlag, 1998. 1Google Scholar
  15. [15]
    A. Streit. On Job Scheduling for HPC-Clusters and the dynP Scheduler. In Proceedings of the 8th International Conference on High Performance Computing (HiPC 2001), volume 2228 of Lecture Notes in Computer Science, pages 58–67. Springer, December 2001. 3, 6, 17, 18Google Scholar
  16. [16]
    A. Streit. The Self-Tuning dynP Job-Scheduler. In Proceedings of the 11th International Heterogenous Computing Workshop (HCW) at IPDPS 2002, Lecture Notes in Computer Science, 2002. 5Google Scholar
  17. [18]
    D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. TR 98-13, Hebrew University, Jerusalem, April 1999. 1Google Scholar
  18. [19]
    Y. Zhang, H. Franke, J. E. Moreira, and A. Sivasubramaniam. Improving Parallel Job Scheduling by Combining Gang Scheduling and Backfilling Techniques. In D.G. Feitelson and L. Rudolph, editor, Proceedings of the 14th International Conference on Parallel and Distributed Processing Symposium (IPDPS-00), pages 133–144. Springer Verlag, 2000. 10Google Scholar
  19. [20]
    Y. Zhang, H. Franke, J. E. Moreira, and A. Sivasubramaniam. An Integrated Approach to Parallel Scheduling Using Gang-Scheduling, Back.lling, and Migration. In D.G. Feitelson and L. Rudolph, editor, Proc. of 7th Workshop on Job Scheduling Strategies for Parallel Processing, volume 2221, pages 133–158. Springer Verlag, 2001. 1Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Achim Streit
    • 1
  1. 1.PC2- Paderborn Center for Parallel ComputingPaderborn UniversityPaderbornGermany

Personalised recommendations