Performance and Fairness for Users in Parallel Job Scheduling

  • Dalibor Klusác̆ek
  • Hana Rudová
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7698)

Abstract

In this work we analyze the performance of scheduling algorithms with respect to fairness. Existing works frequently consider fairness as a job related issue. In our work we analyze fairness with respect to different users of the system as this is a very important real-life problem. First, we discuss how fair are selected popular scheduling algorithms with respect to different users of the system. Next, we present an extension to the well known Conservative backfilling algorithm. Instead of “ad hoc” decisions, the schedule is now created subject to evaluation and optimization. Notably, the fairness is considered as an important metric, which accompanies standard performance related metrics such as slowdown or wait time. To achieve that, an inclusion of fairness as an optimization criterion is proposed. The new extension improves the performance and fairness of Conservative backfilling with respect to other classical techniques such as FCFS, EASY backfilling or aggressive backfilling without reservations.

Keywords

Scheduling Fairness Metaheuristic Backfilling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Adaptive Computing Enterprises, Inc. TORQUE Admininstrator Guide, version 3.0.3 (February 2012), http://www.adaptivecomputing.com/resources/docs/
  2. 2.
    Chlumský, V., Klusáček, D., Ruda, M.: The extension of TORQUE scheduler allowing the use of planning and optimization algorithms in Grids. Computer Science 13(2), 5–19 (2012)CrossRefGoogle Scholar
  3. 3.
    Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global Grid computing for job scheduling. In: GRID 2004: Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, pp. 374–379. IEEE (2004)CrossRefGoogle Scholar
  4. 4.
    Feitelson, D.G.: Experimental analysis of the root causes of performance evaluation results: A backfilling case study. IEEE Transactions on Parallel and Distributed Systems 16(2), 175–182 (2005)CrossRefGoogle Scholar
  5. 5.
    Feitelson, D.G.: Parallel workloads archive (PWA) (February 2012), http://www.cs.huji.ac.il/labs/parallel/workload/
  6. 6.
    Feitelson, D.G., Rudolph, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and practice in parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  7. 7.
    Feitelson, D.G., Weil, A.M.: Utilization and predictability in scheduling the IBM SP2 with backfilling. In: 12th International Parallel Processing Symposium, pp. 542–546. IEEE (1998)Google Scholar
  8. 8.
    Frachtenberg, E., Feitelson, D.G.: Pitfalls in Parallel Job Scheduling Evaluation. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 257–282. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Glover, F.W., Laguna, M.: Tabu search. Kluwer (1998)Google Scholar
  10. 10.
    Hovestadt, M., Kao, O., Keller, A., Streit, A.: Scheduling in HPC Resource Management Systems: Queuing vs. Planning. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 1–20. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    Jones, J.P.: PBS Professional 7, administrator guide. Altair (April 2005)Google Scholar
  12. 12.
    Keleher, P.J., Zotkin, D., Perkovic, D.: Attacking the bottlenecks of backfilling schedulers. Cluster Computing 3(4), 245–254 (2000)CrossRefGoogle Scholar
  13. 13.
    Kleban, S.D., Clearwater, S.H.: Fair share on high performance computing systems: What does fair really mean? In: Third IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2003, pp. 146–153. IEEE Computer Society (2003)Google Scholar
  14. 14.
    Klusáček, D.: Event-based Optimization of Schedules for Grid Jobs. PhD thesis, Masaryk University (2011)Google Scholar
  15. 15.
    Klusáček, D., Rudová, H.: Alea 2 – job scheduling simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools 2010), ICST (2010)Google Scholar
  16. 16.
    Klusáček, D., Rudová, H.: Handling inaccurate runtime estimates by event-based optimization. In: Cracow Grid Workshop 2010 Abstracts (CGW 2010), Cracow, Poland (2010)Google Scholar
  17. 17.
    Klusáček, D., Rudová, H.: Efficient Grid scheduling through the incremental schedule-based approach. Computational Intelligence 27(1), 4–22 (2011)MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Klusáček, D., Rudová, H., Baraglia, R., Pasquali, M., Capannini, G.: Comparison of multi-criteria scheduling techniques. In: Grid Computing Achievements and Prospects, pp. 173–184. Springer (2008)Google Scholar
  19. 19.
    LaTorre, A., Pena, J., Robles, V., De Miguel, P.: Supercomputer Scheduling with Combined Evolutionary Techniques. In: Xhafa, F., Abraham, A. (eds.) Metaheuristics for Scheduling in Distributed Computing Environments. SCI, vol. 146, pp. 95–120. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Lee, C.B.: On the User-Scheduler Relationship in High-Performance Computing. PhD thesis, University of California, San Diego (2009)Google Scholar
  21. 21.
    Leung, V.J., Sabin, G., Sadayappan, P.: Parallel job scheduling policies to improve fairness: a case study. Technical Report SAND 2008-1310, Sandia National Laboratories (2008)Google Scholar
  22. 22.
    Li, B., Zhao, D.: Performance impact of advance reservations from the Grid on backfill algorithms. In: Sixth International Conference on Grid and Cooperative Computing, GCC 2007, pp. 456–461 (2007)Google Scholar
  23. 23.
    Lifka, D.A.: Lifka. The ANL/IBM SP Scheduling System. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  24. 24.
    MetaCentrum (February 2012), http://www.metacentrum.cz/
  25. 25.
    Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Transactions on Parallel and Distributed Systems 12(6), 529–543 (2001)CrossRefGoogle Scholar
  26. 26.
    Ngubiri, J.: Techniques and Evaluation of Processor Co-allocation in Multi-cluster Systems. PhD thesis, Radboud University Nijmegen (2008)Google Scholar
  27. 27.
    Pinedo, M.: Scheduling: Theory, Algorithms, and Systems. Prentice-Hall (2002)Google Scholar
  28. 28.
    Sabin, G.: Unfairness in parallel job scheduling. PhD thesis, The Ohio State University (2006)Google Scholar
  29. 29.
    Sabin, G., Kochhar, G., Sadayappan, P.: Job fairness in non-preemptive job scheduling. In: International Conference on Parallel Processing, ICPP 2004, pp. 186–194. IEEE Computer Society (2004)Google Scholar
  30. 30.
    Sabin, G., Sadayappan, P.: Unfairness Metrics for Space-Sharing Parallel Job Schedulers. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 238–256. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  31. 31.
    Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Selective Reservation Strategies for Backfill Job Scheduling. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 55–71. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  32. 32.
    Srinivasan, S., Kettimuthu, R., Subrarnani, V., Sadayappan, P.: Characterization of backfilling strategies for parallel job scheduling. In: Proceedings of 2002 International Workshops on Parallel Processing, pp. 514–519. IEEE Computer Society (2002)Google Scholar
  33. 33.
    Sulistio, A., Cibej, U., Venugopal, S., Robic, B., Buyya, R.: A toolkit for modelling and simulating data Grids: an extension to GridSim. Concurrency and Computation: Practice & Experience 20(13), 1591–1609 (2008)CrossRefGoogle Scholar
  34. 34.
    Talby, D., Feitelson, D.G.: Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling. In: IPPS 1999/SPDP 1999: Proceedings of the 13th International Symposium on Parallel Processing and the 10th Symposium on Parallel and Distributed Processing, pp. 513–517. IEEE Computer Society (1999)Google Scholar
  35. 35.
    Tsafrir, D.: Using Inaccurate Estimates Accurately. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 208–221. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  36. 36.
    Tsafrir, D., Feitelson, D.G.: The dynamics of backfilling: Solving the mystery of why increased inaccuracy help. In: IEEE International Symposium on Workload Characterization (IISWC), pp. 131–141. IEEE Computer Society (2006)Google Scholar
  37. 37.
    Vasupongayya, S., Chiang, S.-H.: On job fairness in non-preemptive parallel job scheduling. In: Zheng, S.Q. (ed.) International Conference on Parallel and Distributed Computing Systems (PDCS 2005), pp. 100–105. IASTED/ACTA Press (2005)Google Scholar
  38. 38.
    Wolberg, J.: Data Analysis Using the Method of Least Squares: Extracting the Most Information from Experiments. Springer (2006)Google Scholar
  39. 39.
    Xhafa, F., Abraham, A.: Computational models and heuristic methods for Grid scheduling problems. Future Generation Computer Systems 26(4), 608–621 (2010)CrossRefGoogle Scholar
  40. 40.
    Xhafa, F., Carretero, J., Alba, E., Dorronsoro, B.: Design and evaluation of Tabu search method for job scheduling in distributed environments. In: International Symposium on Parallel and Distributed Processing (IPDPS 2008), pp. 1–8. IEEE (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Dalibor Klusác̆ek
    • 1
    • 2
  • Hana Rudová
    • 1
  1. 1.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic
  2. 2.CESNET z.s.p.o.PragueCzech Republic

Personalised recommendations