Annals of Operations Research

, Volume 180, Issue 1, pp 145–164 | Cite as

Greedy scheduling with custom-made objectives

  • Carsten Franke
  • Joachim Lepping
  • Uwe Schwiegelshohn
Article

Abstract

We present a methodology to automatically generate an online job scheduling method for a custom-made objective and real workloads. The scheduling problem comprises independent parallel jobs and parallel identical machines and occurs in Massively Parallel Processing systems and computational Grids. The system administrator defines the scheduling objective that may consider job properties and priorities of users or user groups. Our scheduling method combines a Greedy scheduling algorithm with the dynamic sorting of the waiting queue. This sorting algorithm uses a criterion that is modifiable by a set of parameters. Finding good parameter settings for the sorting criterion is viewed as a nonlinear optimization problem which is solved with the help of Evolution Strategies. We evaluate our scheduling method with real workload data and compare it to approximated optimal offline solutions and to the online results of the standard EASY backfill algorithm.

Keywords

Online job scheduling Workload based scheduling Computational intelligence Computational grids 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aarts, E. H. L., Korst, J. H. M., & van Laarhoven, P. J. M. (1997). Simulated annealing. In Local search in combinatorial optimization (pp. 91–120). New York: Wiley. Google Scholar
  2. Bäck, T., & Schwefel, H.-P. (1993). An overview of evolutionary algorithms for parameter optimization. Evolutionary Computation, 1(1), 1–23. CrossRefGoogle Scholar
  3. Balicki, J., & Kitowski, Z. (2001). Multicriteria evolutionary algorithm with tabu search for task assignment. In E. Zitzler, K. Deb, L. Thiele, C. A. Coello & D. Corne (Eds.), Lecture notes in computer science (LNCS) : Vol. 1993. Proceedings of the 1st international conference on evolutionary multi-criterion optimization (EMO01) (pp. 373–384). Berlin: Springer. CrossRefGoogle Scholar
  4. Becchetti, L., & Leonardi, S. (2004). Nonclairvoyant scheduling to minimize the total flow time on single and parallel machines. Journal of the ACM, 51(4), 517–539. CrossRefGoogle Scholar
  5. Beyer, H.-G., & Schwefel, H.-P. (2002). Evolution strategies—a comprehensive introduction. Natural Computing, 1(1), 3–52. CrossRefGoogle Scholar
  6. Burke, E. K., McCollum, B., Meisels, A., Petrovic, S., & Qu, R. (2007). Graph-based hyper-heuristic for timetabling problems. European Journal of Operational Research, 176(1), 177–192. CrossRefGoogle Scholar
  7. Cirne, W., & Berman, F. (2001). A comprehensive model of the supercomputer workload. In Proceedings of the 4th workshop on workload characterization (pp. 140–148). IEEE Press. Google Scholar
  8. Deb, K., Agrawal, S., Pratab, A., & Meyarivan, T. (2000). A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In M. Schoenauer et al. (Eds.), Lecture notes in computer science (LNCS) : Vol. 1917. Parallel Problem Solving from Nature VI (pp. 849–858). Berlin: Springer. CrossRefGoogle Scholar
  9. Downey, A. B., & Feitelson, D. G. (1999). The elusive goal of workload characterization. Performance Evaluation Review, 26(4), 14–29. CrossRefGoogle Scholar
  10. Ernemann, C., Schwiegelshohn, U., Emmerich, M., Schönemann, L., & Beume, N. (2005). Scheduling algorithm development based on complex owner defined objectives. Technical Report 191/05, SFB 531, CI-Report, University of Dortmund. Google Scholar
  11. Ernemann, C., Song, B., & Yahyapour, R. (2003). Scaling of workload traces. In D. G. Feitelson, L. Rudolph, & U. Schwiegelshohn (Eds.), Lecture notes in computer science (LNCS) : Vol. 2862. Proceedings of job scheduling strategies for parallel processing (JSSPP03) (pp. 166–183). Berlin: Springer. CrossRefGoogle Scholar
  12. Feitelson, D. G. (2002). Workload modeling for performance evaluation. In M. C. Calzarossa & S. Tucci (Eds.), Lecture notes in computer science (LNCS) : Vol. 2459. Performance evaluation of complex systems: techniques and tools (pp. 114–141). Berlin: Springer. CrossRefGoogle Scholar
  13. Feitelson, D. G. (2007). Parallel workload archive. http://www.cs.huji.ac.il/labs/parallel/workload/.
  14. Feitelson, D. G., & Nitzberg, B. (1995). Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/860. In D. G. Feitelson & L. Rudolph (Eds.), Lecture notes in computer science (LNCS) : Vol. 949. Proceedings of job scheduling strategies for parallel processing (JSSPP95) (pp. 337–360). Berlin: Springer. Google Scholar
  15. Feitelson, D. G., Rudolph, L., & Schwiegelshohn, U. (2004). Parallel job scheduling—a status report. In Lecture notes in computer science (LNCS) : Vol. 3277. Proceedings of job scheduling strategies for parallel processing (JSSPP04) (pp. 1–16). Berlin: Springer. CrossRefGoogle Scholar
  16. Feitelson, D. G., Rudolph, L., Schwiegelshohn, U., Sevcik, K. C., & Wong, P. (1997). Theory and practice in parallel job scheduling. In Lecture notes in computer science (LNCS) : Vol. 1291. Proceedings of job scheduling strategies for parallel processing (JSSPP97) (pp. 1–34). Berlin: Springer. Google Scholar
  17. Feitelson, D. G., & Weil, A. M. (1998). Utilization and predictability in scheduling the IBM SP2 with backfilling. In Proceedings of the 12th international parallel processing symposium and the 9th symposium on parallel and distributed processing (pp. 542–547). IEEE Computer Society Press. Google Scholar
  18. Foster, I., Kesselman, C., & Tuecke, S. (2001). The anatomy of the grid: enabling scalable virtual organizations. International Journal of High Performance Computing Applications, 15, 200–222. CrossRefGoogle Scholar
  19. Franke, C., Lepping, J., & Schwiegelshohn, U. (2006). On advantages of scheduling using genetic fuzzy systems. In E. Frachtenberg & U. Schwiegelshohn (Eds.), Lecture notes in computer science (LNCS) : Vol. 4376. Proc. 12th workshop on job scheduling strategies for parallel processing (JSSPP) (pp. 68–93). Berlin: Springer. CrossRefGoogle Scholar
  20. Garey, M., & Graham, R. L. (1975). Bounds for multiprocessor scheduling with resource constraints. SIAM Journal on Computing, 4(2), 187–200. CrossRefGoogle Scholar
  21. Hertz, A., Taillard, E., & de Werra, D. (1997). In Local search in combinatorial optimization (pp. 121–136). New York: Wiley. Google Scholar
  22. Krallmann, J., Schwiegelshohn, U., & Yahyapour, R. (1999). On the design and evaluation of job scheduling systems. In D. G. Feitelson & L. Rudolph (Eds.), Lecture notes in computer science (LNCS) : Vol. 1659. Proceedings of job scheduling strategies for parallel processing (JSSPP99). Berlin: Springer. CrossRefGoogle Scholar
  23. Lee, C. B., Schwartzman, Y., Hardy, J., & Snavely, A. (2005). Are user runtime estimates inherently inaccurate?. In Lecture notes in computer science (LNCS) : Vol. 3277. Proceedings of job scheduling strategies for parallel processing (JSSPP05) (pp. 253–263). Berlin: Springer. Google Scholar
  24. Lee, C. B., & Snavely, A. E. (2007). Precise and realistic utility functions for user-centric performance analysis of schedulers. In HPDC ’07: Proceedings of the 16th international symposium on High performance distributed computing (pp. 107–116). Google Scholar
  25. Lifka, D. A. (1995). The ANL/IBM SP scheduling system. In Lecture notes in computer science (LNCS) : Vol. 949. Proceedings of job scheduling strategies for parallel processing (JSSPP95) (pp. 295–303). Berlin: Springer. Google Scholar
  26. Lublin, U., & Feitelson, D. G. (2003). The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing, 63(11), 1105–1122. CrossRefGoogle Scholar
  27. Pinedo, M. (2002). Scheduling: theory, algorithms, and systems (2nd ed.). New York: Prentice-Hall. Google Scholar
  28. Schwiegelshohn, U., & Yahyapour, R. (2000). Fairness in parallel job scheduling. Journal of Scheduling, 3(5), 297–320. CrossRefGoogle Scholar
  29. Schwiegelshohn, U., & Yahyapour, R. (2003). In Grid resource management—state of the art and future trends (pp. 41–52). Dordrecht: Kluwer. Google Scholar
  30. Sleator, D., & Tarjan, R. E. (1985). Amortized efficiency of list update and paging rules. Communications of the ACM, 28, 202–208. CrossRefGoogle Scholar
  31. Tsafrir, D. (2006). Modeling, evaluating, and improving the performance of supercomputer scheduling. PhD thesis, School of Computer Science and Engineering, the Hebrew University, Jerusalem, Israel. Technical Report 2006-78. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Carsten Franke
    • 1
  • Joachim Lepping
    • 2
  • Uwe Schwiegelshohn
    • 2
  1. 1.ABB Schweiz AGBadenSwitzerland
  2. 2.Robotics Research InstituteTechnische Universität DortmundDortmundGermany

Personalised recommendations