Budget Constraint Bag-of-Task Based Workflow Scheduling in Public Clouds
Bag-of-Tasks (BoT) workflows have appeared in distributed computing platforms such as Spark, MapReduce, and Pegasus. Budget constraints usually exist for these applications. It is crucial to design scheduling algorithms to minimize makespans under budget constraints for BoT workflows. However, most existing workflow algorithms are tailored for general workflows without considering batch structures of Bot-workflows. The main challenge for scheduling BoT workflows is to distribute the budget to different BoTs appropriately considering BoT structures. In this paper, a configuration-and-serialization iterative adjusting based heuristic algorithm (CSIA) is proposed to minimize the makespans under budget constraints. CSIA allocates VM configurations and serial degrees to different BoTs appropriately to decrease the makespan. Experimental results illustrate that the proposal gets shorter makespans on several types of workflow instances than existing algorithms under budget constraints.
KeywordsCloud computing Bag of tasks Workflow scheduling Budget constraint Makespan
Zhicheng Cai is supported by the National Natural Science Foundation of China (Grant No. 61602243), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160846), the Fundamental Research Funds for the Central Universities (No. 30919011235) and the Fundamental Research Funds for the Central Universities (No. 30920120180101). Duan Liu is supported by the Postgraduate Research and Practice Innovation Program of Jiangsu Province (No. KYCX18_0434).
- 1.Spark lightning-fast unified analytics engine. http://spark.apache.org. Accessed 14 May 2019
- 2.A workflow generator. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator. Accessed 30 June 2016
- 4.Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10. IEEE, Austin (2008)Google Scholar
- 5.Cai, Z., Li, Q., Li, X.: ElasticSim: a toolkit for simulating workflows with cloud resource runtime auto-scaling and stochastic task execution times. J. Grid Comput. 15, 1–16 (2016)Google Scholar
- 9.Chopra, N., Singh, S.: Heft based workflow scheduling algorithm for cost optimization within deadline in hybrid clouds. In: Fourth International Conference on Computing, pp. 1–6 (2014)Google Scholar
- 10.Ghafouri, R., Movaghar, A., Mohsenzadeh, M.: A budget constrained scheduling algorithm for executing workflow application in infrastructure as a service clouds. Peer-to-Peer Netw. Appl. 3, 1–28 (2018)Google Scholar
- 13.Lin, X., Wu, C.Q.: On scientific workflow scheduling in clouds under budget constraint. In: IEEE International Conference on Parallel Processing, vol. 46, no. 1, pp. 90–99 (2013)Google Scholar
- 14.Mao, M., Humphrey, M.: Scaling and scheduling to maximize application performance within budget constraints in cloud workflows. In: IEEE International Symposium on Parallel and Distributed Processing, pp. 67–78 (2013)Google Scholar
- 17.Shi, J., Luo, J., Fang, D., Zhang, J.: A budget and deadline aware scientific workflow resource provisioning and scheduling mechanism for cloud. In: IEEE International Conference on Computer Supported Cooperative Work in Design, pp. 672–677 (2014)Google Scholar
- 19.Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: USENIX Conference on Hot Topics in Cloud Computing, pp. 1–10 (2010)Google Scholar
- 20.Zeng, L., Veeravalli, B., Li, X.: ScaleStar: budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In: IEEE International Conference on Advanced Information Networking and Applications, pp. 534–541 (2012)Google Scholar
- 21.Zhao, H., Sakellariou, R.: Scheduling multiple DAGs onto heterogeneous systems. In: International Parallel and Distributed Processing Symposium, pp. 1–14 (2006)Google Scholar