On Scheduling Algorithms for MapReduce Jobs in Heterogeneous Clouds with Budget Constraints

  • Yang Wang
  • Wei Shi
Conference paper

DOI: 10.1007/978-3-319-03850-6_18

Volume 8304 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Wang Y., Shi W. (2013) On Scheduling Algorithms for MapReduce Jobs in Heterogeneous Clouds with Budget Constraints. In: Baldoni R., Nisse N., van Steen M. (eds) Principles of Distributed Systems. OPODIS 2013. Lecture Notes in Computer Science, vol 8304. Springer, Cham

Abstract

In this paper, we consider task-level scheduling algorithms with respect to budget constraints for a bag of MapReduce jobs on a set of provisioned heterogeneous (virtual) machines in cloud platforms. The heterogeneity is manifested in the popular ”pay-as-you-go” charging model where the service machines with different performance would have different service rates. We organize a bag of jobs as a κ-stage workflow and consider the scheduling problem with budget constraints. In particular, given a total monetary budget, by combining a greedy-based local optimal algorithm and dynamic programming techniques, we first propose a global optimal scheduling algorithm to achieve a minimum scheduling length of the workflow in pseudo-polynomial time. Then, we extend the idea in the greedy algorithm to efficient global distribution of the budget among the tasks in different stages for overall scheduling length reduction. Our empirical studies verify the proposed optimal algorithm and show the efficiency of the greedy algorithm to minimize the scheduling length.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Yang Wang
    • 1
  • Wei Shi
    • 2
  1. 1.Faculty of Computer ScienceUniversity of New BrunswickFrederictonCanada
  2. 2.Faculty of Business and Information TechnologyUniversity of Ontario Institute of TechnologyOntarioCanada