Advertisement

On Interactions among Scheduling Policies: Finding Efficient Queue Setup Using High-Resolution Simulations

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8632)

Abstract

Many studies in the past two decades focused on the problem of efficient job scheduling in HPC and Grid-like systems. While many new scheduling algorithms have been proposed for systems with specific requirements, mainstream resource management systems and schedulers are still only using a limited set of scheduling policies. Production systems need to balance various policies that are set in place to satisfy both the resource providers and users (or virtual organizations) in the system. While many works address these separate policies, e.g., fairshare for fair resource allocation, only few works try to address the interactions between these separate solutions. In this paper we describe how to approach these interactions when developing site-specific policies. Notably, we describe how (priority) queues interact with scheduling algorithms, fairshare and with anti-starvation mechanisms. Moreover, we present a case study describing how an advanced simulation tool was used to find new configuration for an actual resource manager deployed in the Czech National Grid, significantly increasing its performance.

Keywords

Scheduling Queues Fairshare Simulation 

References

  1. 1.
    Adaptive Computing Enterprises, Inc. Maui Scheduler Administrator’s Guide, version 3.2 (January 2014), http://docs.adaptivecomputing.com
  2. 2.
    Adaptive Computing Enterprises, Inc. Moab workload manager administrator’s guide, version 7.2.6 (January 2014), http://docs.adaptivecomputing.com
  3. 3.
    Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global Grid computing for job scheduling. In: GRID 2004: Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, pp. 374–379. IEEE (2004)Google Scholar
  4. 4.
    Flis, L., Lason, P., Magrys, M., Ozieblo, A., Twardy, M.: Effective utilization of mixed computing resources on zeus cluster. In: Cracow Grid Workshop, pp. 105–106. ACC Cyfronet AGH (2012)Google Scholar
  5. 5.
    Frachtenberg, E., Feitelson, D.G.: Pitfalls in parallel job scheduling evaluation. In: Feitelson, D., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 257–282. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: Fair allocation of multiple resource types. In: 8th USENIX Symposium on Networked Systems Design and Implementation (2011)Google Scholar
  7. 7.
    Jackson, D., Snell, Q., Clement, M.: Core algorithms of the Maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  8. 8.
    Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multi-resource allocation: Fairness-efficiency tradeoffs in a unifying framework. In: 31st Annual International Conference on Computer Communications (IEEE INFOCOM), pp. 1206–1214 (2012)Google Scholar
  9. 9.
    Kleban, S.D., Clearwater, S.H.: Fair share on high performance computing systems: What does fair really mean? In. In: Third IEEE International Symposium on Cluster Computing and the Grid, pp. 146–153. IEEE Computer Society (2003)Google Scholar
  10. 10.
    Klusáček, D., Rudová, H.: Alea 2 – job scheduling simulator. In: 3rd International ICST Conference on Simulation Tools and Technique, ICST (2010)Google Scholar
  11. 11.
    Klusáček, D., Rudová, H.: Performance and fairness for users in parallel job scheduling. In: Cirne, W., Desai, N., Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2012. LNCS, vol. 7698, pp. 235–252. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  12. 12.
    Klusáček, D., Rudová, H.: Multi-resource aware fairsharing for heterogeneous systems. In: Job Scheduling Strategies for Parallel Processing (2014)Google Scholar
  13. 13.
    Lifka, D.A.: The ANL/IBM SP Scheduling System. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  14. 14.
    MetaCentrum (January 2014), http://www.metacentrum.cz/
  15. 15.
    Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Transactions on Parallel and Distributed Systems 12(6), 529–543 (2001)CrossRefGoogle Scholar
  16. 16.
    Ohio Supercomputer Center. Batch Processing at OSC (February 2014), https://www.osc.edu/supercomputing/batch-processing-at-osc
  17. 17.
    PBS Works, PBS Professional 12.1, Administrator’s Guide (January 2014), http://www.pbsworks.com
  18. 18.
    Schwiegelshohn, U.: How to design a job scheduling algorithm. In: Job Scheduling Strategies for Parallel Processing (2014)Google Scholar
  19. 19.
    Wierman, A., Harchol-Balter, M.: Classifying scheduling policies with respect to unfairness in an M/GI/1. In: 2003 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pp. 238–249. ACM (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.CESNET a.l.e.PragueCzech Republic
  2. 2.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic

Personalised recommendations