Advertisement

Multi Resource Fairness: Problems and Challenges

  • Dalibor Klusáček
  • Hana Rudová
  • Michal Jaroš
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8429)

Abstract

Current production resource management and scheduling systems often use some mechanism to guarantee fair sharing of computational resources among different users of the system. For example, the user who so far consumed small amount of CPU time gets higher priority and vice versa. The problem with such a solution is that it does not reflect other consumed resources like RAM, HDD storage capacity or GPU cores. Clearly, different users may have highly heterogeneous demands concerning aforementioned resources, yet they are all prioritized only with respect to consumed CPU time. In this paper we show that such a single resource-based approach is unfair and is no longer suitable for nowadays systems. We provide a survey of existing works that somehow try to deal with this situation and we closely analyze and evaluate their characteristics. Next, we propose new enhanced approaches that would allow the development of usable multi resource-aware user prioritization mechanisms. We demonstrate that different consumed resources can be weighted and combined together within a single formula which can be used to establish users’ priorities. Moreover, we show that when it comes to multiple resources, it is not always possible to find a suitable solution that would fulfill all fairness-related requirements.

Keywords

Multi resource fairness Fairshare Penalty Scheduling 

Notes

Acknowledgments

We highly appreciate the support of the Grant Agency of the Czech Republic under the grant No. P202/12/0306. The access to the MetaCentrum computing facilities provided under the programme LM2010005 funded by the Ministry of Education, Youth, and Sports of the Czech Republic is highly appreciated. The Zewura workload log was kindly provided by the Czech NGI MetaCentrum. The access to the CERIT-SC computing and storage facilities provided under the programme Center CERIT Scientific Cloud, part of the Operational Program Research and Development for Innovations, reg. no. CZ. 1.05/3.2.00/08.0144 is appreciated.

References

  1. 1.
    Adaptive Computing Enterprises, Inc. Maui Scheduler Administrator’s Guide, version 3.2, February 2013. http://docs.adaptivecomputing.com
  2. 2.
    Adaptive Computing Enterprises, Inc. Moab workload manager administrator’s guide, version 7.2.1, February 2013. http://docs.adaptivecomputing.com
  3. 3.
    Adaptive Computing Enterprises, Inc. TORQUE Admininstrator Guide, version 4.2.0, February 2013. http://docs.adaptivecomputing.com
  4. 4.
    Apache.org. Hadoop Capacity Scheduler, February 2013. http://hadoop.apache.org/docs/r1.1.1/capacity_scheduler.html
  5. 5.
    Apache.org. Hadoop Fair Scheduler, February 2013. http://hadoop.apache.org/docs/r1.1.1/fair_scheduler.html
  6. 6.
    Blazewicz, J., Drozdowski, M., Markiewicz, M.: Divisible task scheduling - concept and verification. Parallel Comput. 25(1), 87–98 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Dolev, D., Feitelson, D.G., Halpern, J.Y., Kupferman, R., Linial, N.: No justified complaints: on fair sharing of multiple resources. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS ’12, pp. 68–75. ACM, New York (2012)Google Scholar
  8. 8.
    Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: fair allocation of multiple resource types. In: 8th USENIX Symposium on Networked Systems Design and Implementation (2011)Google Scholar
  9. 9.
    Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: SOSP’09 (2009)Google Scholar
  10. 10.
    Jackson, D.B., Snell, Q.O., Clement, M.J.: Core algorithms of the Maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  11. 11.
    Jain, R., Chiu, D.-M., Hawe, W.: A quantitative measure of fairness and discrimination for resource allocation in shared computer systems. Technical report TR-301, Digital Equipment Corporation (1984)Google Scholar
  12. 12.
    Joe-Wong, C., Sen, S., Lan, T., Chiang, M.: Multi-resource allocation: fairness-efficiency tradeoffs in a unifying framework. In: INFOCOM (2012)Google Scholar
  13. 13.
    Jones, J.P.: PBS Professional 7, administrator guide. Altair, April 2005Google Scholar
  14. 14.
    Kleban, S.D., Clearwater, S.H.: Fair share on high performance computing systems: what does fair really mean? In: Third IEEE International Symposium on Cluster Computing and the Grid (CCGrid’03), pp. 146–153. IEEE Computer Society (2003)Google Scholar
  15. 15.
    Klusáček, D., Ruda, M., Rudová, H.: New fairness and performance metrics for current grids. In: Cracow Grid Workshop, pp. 73–74. ACC Cyfronet AGH (2012)Google Scholar
  16. 16.
    MetaCentrum, February 2013. http://www.metacentrum.cz/
  17. 17.
    Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)CrossRefGoogle Scholar
  18. 18.
    Ohio Supercomputer Center. Batch Processing at OSC, February 2013. https://www.osc.edu/supercomputing/batch-processing-at-osc
  19. 19.
    Shmueli, E., Feitelson, D.G.: Backfilling with lookahead to optimize the performance of parallel job scheduling. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 228–251. Springer, Heidelberg (2003) CrossRefGoogle Scholar
  20. 20.
    Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18(6), 789–803 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Dalibor Klusáček
    • 1
    • 2
  • Hana Rudová
    • 1
  • Michal Jaroš
    • 3
  1. 1.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic
  2. 2.CESNET z.s.p.o.PragueCzech Republic
  3. 3.Institute of Computer ScienceMasaryk UniversityBrnoCzech Republic

Personalised recommendations