Advertisement

Multi-resource Aware Fairsharing for Heterogeneous Systems

  • Dalibor KlusáčekEmail author
  • Hana Rudová
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8828)

Abstract

Current production resource management and scheduling systems often use some mechanism to guarantee fair sharing of computational resources among different users of the system. For example, the user who so far consumed small amount of CPU time gets higher priority and vice versa. However, different users may have highly heterogeneous demands concerning system resources, including CPUs, RAM, HDD storage capacity or, e.g., GPU cores. Therefore, it may not be fair to prioritize them only with respect to the consumed CPU time. Still, applied mechanisms often do not reflect other consumed resources or they use rather simplified and “ad hoc” solutions to approach these issues. We show that such solutions may be (highly) unfair and unsuitable for heterogeneous systems. We provide a survey of existing works that try to deal with this situation, analyzing and evaluating their characteristics. Next, we present new enhanced approach that supports multi-resource aware user prioritization mechanism. Importantly, this approach is capable of dealing with the heterogeneity of both jobs and resources. A working implementation of this new prioritization scheme is currently applied in the Czech National Grid Infrastructure MetaCentrum.

Keywords

Multi-resource fairness Fairshare Heterogeneity 

Notes

Acknowledgments

We highly appreciate the support of the Grant Agency of the Czech Republic under the grant No. P202/12/0306. The support provided under the programme “Projects of Large Infrastructure for Research, Development, and Innovations” LM2010005 funded by the Ministry of Education, Youth, and Sports of the Czech Republic is highly appreciated. The access to the MetaCentrum computing facilities and workloads is kindly acknowledged.

References

  1. 1.
    Adaptive Computing Enterprises, Inc. Maui Scheduler Administrator’s Guide, version 3.2, January 2014. http://docs.adaptivecomputing.com
  2. 2.
    Adaptive Computing Enterprises, Inc. Moab workload manager administrator’s guide, version 7.2.6, January 2014. http://docs.adaptivecomputing.com
  3. 3.
    Adaptive Computing Enterprises, Inc. TORQUE Admininstrator Guide, version 4.2.6, January 2014. http://docs.adaptivecomputing.com
  4. 4.
    Apache.org. Hadoop Capacity Scheduler, January 2014. http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html
  5. 5.
    Apache.org. Hadoop Fair Scheduler, January 2014. http://hadoop.apache.org/docs/r1.2.1/fair_scheduler.html
  6. 6.
    Apache.org. Hadoop Next Generation Fair Scheduler, January 2014. http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
  7. 7.
    Blazewicz, J., Drozdowski, M., Markiewicz, M.: Divisible task scheduling - concept and verification. Parallel Comput. 25(1), 87–98 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Dolev, D., Feitelson, D.G., Halpern, J.Y., Kupferman, R., Linial, N.: No justified complaints: on fair sharing of multiple resources. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS 2012, pp. 68–75. ACM, New York (2012)Google Scholar
  9. 9.
    Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: fair allocation of multiple resource types. In: 8th USENIX Symposium on Networked Systems Design and Implementation (2011)Google Scholar
  10. 10.
    Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 261–276 (2009)Google Scholar
  11. 11.
    Jackson, D.B., Snell, Q.O., Clement, M.J.: Core algorithms of the maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  12. 12.
    Jain, R., Chiu, D.-M., Hawe, W.: A quantitative measure of fairness and discrimination for resource allocation in shared computer systems. Technical report TR-301, Digital Equipment Corporation (1984)Google Scholar
  13. 13.
    Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multi-resource allocation: fairness-efficiency tradeoffs in a unifying framework. In: 31st Annual International Conference on Computer Communications (IEEE INFOCOM), pp. 1206–1214 (2012)Google Scholar
  14. 14.
    Kleban, S.D., Clearwater, S.H.: Fair share on high performance computing systems: what does fair really mean? In: Third IEEE International Symposium on Cluster Computing and the Grid (CCGrid2003), pp. 146–153. IEEE Computer Society (2003)Google Scholar
  15. 15.
    Klusáček, D., Rudová, H.: Alea 2 - job scheduling simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools 2010). ICST (2010)Google Scholar
  16. 16.
    Klusáček, D., Rudová, H.: New multi-resource fairshare prioritization mechanisms for heterogeneous computing platforms. In: Cracow Grid Workshop, pp. 89–90. ACC Cyfronet AGH (2013)Google Scholar
  17. 17.
    Klusáček, D., Rudová, H., Jaroš, M.: Multi resource fairness: problems and challenges. In: Desai, N., Cirne, W. (eds.) JSSPP 2013. LNCS, vol. 8429, pp. 81–95. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  18. 18.
    MetaCentrum, January 2014. http://www.metacentrum.cz/
  19. 19.
    PBS Works. PBS Professional 12.1, Administrator’s Guide, January 2014. http://www.pbsworks.com

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.CESNET z.s.p.o.PragueCzech Republic
  2. 2.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic

Personalised recommendations