Abstract
We present a methodology to automatically generate an online job scheduling method for a custom-made objective and real workloads. The scheduling problem comprises independent parallel jobs and parallel identical machines and occurs in Massively Parallel Processing systems and computational Grids. The system administrator defines the scheduling objective that may consider job properties and priorities of users or user groups. Our scheduling method combines a Greedy scheduling algorithm with the dynamic sorting of the waiting queue. This sorting algorithm uses a criterion that is modifiable by a set of parameters. Finding good parameter settings for the sorting criterion is viewed as a nonlinear optimization problem which is solved with the help of Evolution Strategies. We evaluate our scheduling method with real workload data and compare it to approximated optimal offline solutions and to the online results of the standard EASY backfill algorithm.
Similar content being viewed by others
References
Aarts, E. H. L., Korst, J. H. M., & van Laarhoven, P. J. M. (1997). Simulated annealing. In Local search in combinatorial optimization (pp. 91–120). New York: Wiley.
Bäck, T., & Schwefel, H.-P. (1993). An overview of evolutionary algorithms for parameter optimization. Evolutionary Computation, 1(1), 1–23.
Balicki, J., & Kitowski, Z. (2001). Multicriteria evolutionary algorithm with tabu search for task assignment. In E. Zitzler, K. Deb, L. Thiele, C. A. Coello & D. Corne (Eds.), Lecture notes in computer science (LNCS) : Vol. 1993. Proceedings of the 1st international conference on evolutionary multi-criterion optimization (EMO01) (pp. 373–384). Berlin: Springer.
Becchetti, L., & Leonardi, S. (2004). Nonclairvoyant scheduling to minimize the total flow time on single and parallel machines. Journal of the ACM, 51(4), 517–539.
Beyer, H.-G., & Schwefel, H.-P. (2002). Evolution strategies—a comprehensive introduction. Natural Computing, 1(1), 3–52.
Burke, E. K., McCollum, B., Meisels, A., Petrovic, S., & Qu, R. (2007). Graph-based hyper-heuristic for timetabling problems. European Journal of Operational Research, 176(1), 177–192.
Cirne, W., & Berman, F. (2001). A comprehensive model of the supercomputer workload. In Proceedings of the 4th workshop on workload characterization (pp. 140–148). IEEE Press.
Deb, K., Agrawal, S., Pratab, A., & Meyarivan, T. (2000). A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In M. Schoenauer et al. (Eds.), Lecture notes in computer science (LNCS) : Vol. 1917. Parallel Problem Solving from Nature VI (pp. 849–858). Berlin: Springer.
Downey, A. B., & Feitelson, D. G. (1999). The elusive goal of workload characterization. Performance Evaluation Review, 26(4), 14–29.
Ernemann, C., Schwiegelshohn, U., Emmerich, M., Schönemann, L., & Beume, N. (2005). Scheduling algorithm development based on complex owner defined objectives. Technical Report 191/05, SFB 531, CI-Report, University of Dortmund.
Ernemann, C., Song, B., & Yahyapour, R. (2003). Scaling of workload traces. In D. G. Feitelson, L. Rudolph, & U. Schwiegelshohn (Eds.), Lecture notes in computer science (LNCS) : Vol. 2862. Proceedings of job scheduling strategies for parallel processing (JSSPP03) (pp. 166–183). Berlin: Springer.
Feitelson, D. G. (2002). Workload modeling for performance evaluation. In M. C. Calzarossa & S. Tucci (Eds.), Lecture notes in computer science (LNCS) : Vol. 2459. Performance evaluation of complex systems: techniques and tools (pp. 114–141). Berlin: Springer.
Feitelson, D. G. (2007). Parallel workload archive. http://www.cs.huji.ac.il/labs/parallel/workload/.
Feitelson, D. G., & Nitzberg, B. (1995). Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/860. In D. G. Feitelson & L. Rudolph (Eds.), Lecture notes in computer science (LNCS) : Vol. 949. Proceedings of job scheduling strategies for parallel processing (JSSPP95) (pp. 337–360). Berlin: Springer.
Feitelson, D. G., Rudolph, L., & Schwiegelshohn, U. (2004). Parallel job scheduling—a status report. In Lecture notes in computer science (LNCS) : Vol. 3277. Proceedings of job scheduling strategies for parallel processing (JSSPP04) (pp. 1–16). Berlin: Springer.
Feitelson, D. G., Rudolph, L., Schwiegelshohn, U., Sevcik, K. C., & Wong, P. (1997). Theory and practice in parallel job scheduling. In Lecture notes in computer science (LNCS) : Vol. 1291. Proceedings of job scheduling strategies for parallel processing (JSSPP97) (pp. 1–34). Berlin: Springer.
Feitelson, D. G., & Weil, A. M. (1998). Utilization and predictability in scheduling the IBM SP2 with backfilling. In Proceedings of the 12th international parallel processing symposium and the 9th symposium on parallel and distributed processing (pp. 542–547). IEEE Computer Society Press.
Foster, I., Kesselman, C., & Tuecke, S. (2001). The anatomy of the grid: enabling scalable virtual organizations. International Journal of High Performance Computing Applications, 15, 200–222.
Franke, C., Lepping, J., & Schwiegelshohn, U. (2006). On advantages of scheduling using genetic fuzzy systems. In E. Frachtenberg & U. Schwiegelshohn (Eds.), Lecture notes in computer science (LNCS) : Vol. 4376. Proc. 12th workshop on job scheduling strategies for parallel processing (JSSPP) (pp. 68–93). Berlin: Springer.
Garey, M., & Graham, R. L. (1975). Bounds for multiprocessor scheduling with resource constraints. SIAM Journal on Computing, 4(2), 187–200.
Hertz, A., Taillard, E., & de Werra, D. (1997). In Local search in combinatorial optimization (pp. 121–136). New York: Wiley.
Krallmann, J., Schwiegelshohn, U., & Yahyapour, R. (1999). On the design and evaluation of job scheduling systems. In D. G. Feitelson & L. Rudolph (Eds.), Lecture notes in computer science (LNCS) : Vol. 1659. Proceedings of job scheduling strategies for parallel processing (JSSPP99). Berlin: Springer.
Lee, C. B., Schwartzman, Y., Hardy, J., & Snavely, A. (2005). Are user runtime estimates inherently inaccurate?. In Lecture notes in computer science (LNCS) : Vol. 3277. Proceedings of job scheduling strategies for parallel processing (JSSPP05) (pp. 253–263). Berlin: Springer.
Lee, C. B., & Snavely, A. E. (2007). Precise and realistic utility functions for user-centric performance analysis of schedulers. In HPDC ’07: Proceedings of the 16th international symposium on High performance distributed computing (pp. 107–116).
Lifka, D. A. (1995). The ANL/IBM SP scheduling system. In Lecture notes in computer science (LNCS) : Vol. 949. Proceedings of job scheduling strategies for parallel processing (JSSPP95) (pp. 295–303). Berlin: Springer.
Lublin, U., & Feitelson, D. G. (2003). The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing, 63(11), 1105–1122.
Pinedo, M. (2002). Scheduling: theory, algorithms, and systems (2nd ed.). New York: Prentice-Hall.
Schwiegelshohn, U., & Yahyapour, R. (2000). Fairness in parallel job scheduling. Journal of Scheduling, 3(5), 297–320.
Schwiegelshohn, U., & Yahyapour, R. (2003). In Grid resource management—state of the art and future trends (pp. 41–52). Dordrecht: Kluwer.
Sleator, D., & Tarjan, R. E. (1985). Amortized efficiency of list update and paging rules. Communications of the ACM, 28, 202–208.
Tsafrir, D. (2006). Modeling, evaluating, and improving the performance of supercomputer scheduling. PhD thesis, School of Computer Science and Engineering, the Hebrew University, Jerusalem, Israel. Technical Report 2006-78.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Franke, C., Lepping, J. & Schwiegelshohn, U. Greedy scheduling with custom-made objectives. Ann Oper Res 180, 145–164 (2010). https://doi.org/10.1007/s10479-008-0491-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-008-0491-2