Advertisement

Budget Constraint Bag-of-Task Based Workflow Scheduling in Public Clouds

  • Pengfei Sun
  • Zhicheng CaiEmail author
  • Duan Liu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1042)

Abstract

Bag-of-Tasks (BoT) workflows have appeared in distributed computing platforms such as Spark, MapReduce, and Pegasus. Budget constraints usually exist for these applications. It is crucial to design scheduling algorithms to minimize makespans under budget constraints for BoT workflows. However, most existing workflow algorithms are tailored for general workflows without considering batch structures of Bot-workflows. The main challenge for scheduling BoT workflows is to distribute the budget to different BoTs appropriately considering BoT structures. In this paper, a configuration-and-serialization iterative adjusting based heuristic algorithm (CSIA) is proposed to minimize the makespans under budget constraints. CSIA allocates VM configurations and serial degrees to different BoTs appropriately to decrease the makespan. Experimental results illustrate that the proposal gets shorter makespans on several types of workflow instances than existing algorithms under budget constraints.

Keywords

Cloud computing Bag of tasks Workflow scheduling Budget constraint Makespan 

Notes

Acknowledgements

Zhicheng Cai is supported by the National Natural Science Foundation of China (Grant No. 61602243), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160846), the Fundamental Research Funds for the Central Universities (No. 30919011235) and the Fundamental Research Funds for the Central Universities (No. 30920120180101). Duan Liu is supported by the Postgraduate Research and Practice Innovation Program of Jiangsu Province (No. KYCX18_0434).

References

  1. 1.
    Spark lightning-fast unified analytics engine. http://spark.apache.org. Accessed 14 May 2019
  2. 2.
  3. 3.
    Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Deadline-constrained workflow scheduling algorithms for infrastructure; as a service clouds. Future Gener. Comput. Syst. 29(1), 158–169 (2013)CrossRefGoogle Scholar
  4. 4.
    Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10. IEEE, Austin (2008)Google Scholar
  5. 5.
    Cai, Z., Li, Q., Li, X.: ElasticSim: a toolkit for simulating workflows with cloud resource runtime auto-scaling and stochastic task execution times. J. Grid Comput. 15, 1–16 (2016)Google Scholar
  6. 6.
    Cai, Z., Li, X., Ruiz, R.: Resource provisioning for task-batch based workflows with deadlines in public clouds. IEEE Trans. Cloud Comput. 7(3), 814–826 (2019)CrossRefGoogle Scholar
  7. 7.
    Cai, Z., Li, X., Ruiz, R., Li, Q.: A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds. Future Gener. Comput. Syst. 71(C), 57–72 (2017)CrossRefGoogle Scholar
  8. 8.
    Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)CrossRefGoogle Scholar
  9. 9.
    Chopra, N., Singh, S.: Heft based workflow scheduling algorithm for cost optimization within deadline in hybrid clouds. In: Fourth International Conference on Computing, pp. 1–6 (2014)Google Scholar
  10. 10.
    Ghafouri, R., Movaghar, A., Mohsenzadeh, M.: A budget constrained scheduling algorithm for executing workflow application in infrastructure as a service clouds. Peer-to-Peer Netw. Appl. 3, 1–28 (2018)Google Scholar
  11. 11.
    Li, X., Cai, Z.: Elastic resource provisioning for cloud workflow applications. IEEE Trans. Autom. Sci. Eng. 14(2), 1195–1210 (2017)CrossRefGoogle Scholar
  12. 12.
    Li, Z., Ge, J., Hu, H., Wei, S., Luo, B.: Cost and energy aware scheduling algorithm for scientific workflows with deadline constraint in clouds. IEEE Trans. Serv. Comput. 11(1), 713–726 (2018)CrossRefGoogle Scholar
  13. 13.
    Lin, X., Wu, C.Q.: On scientific workflow scheduling in clouds under budget constraint. In: IEEE International Conference on Parallel Processing, vol. 46, no. 1, pp. 90–99 (2013)Google Scholar
  14. 14.
    Mao, M., Humphrey, M.: Scaling and scheduling to maximize application performance within budget constraints in cloud workflows. In: IEEE International Symposium on Parallel and Distributed Processing, pp. 67–78 (2013)Google Scholar
  15. 15.
    Nudtasomboon, N., Randhawa, S.U.: Resource-constrained project scheduling with renewable and non-renewable resources and time-resource tradeoffs. Comput. Ind. Eng. 32(1), 227–242 (1997)CrossRefGoogle Scholar
  16. 16.
    Sahni, J., Vidyarthi, D.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2018)CrossRefGoogle Scholar
  17. 17.
    Shi, J., Luo, J., Fang, D., Zhang, J.: A budget and deadline aware scientific workflow resource provisioning and scheduling mechanism for cloud. In: IEEE International Conference on Computer Supported Cooperative Work in Design, pp. 672–677 (2014)Google Scholar
  18. 18.
    Shi, J., Luo, J., Fang, D., Zhang, J., Zhang, J.: Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints. Cluster Comput. 19(1), 167–182 (2016)CrossRefGoogle Scholar
  19. 19.
    Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: USENIX Conference on Hot Topics in Cloud Computing, pp. 1–10 (2010)Google Scholar
  20. 20.
    Zeng, L., Veeravalli, B., Li, X.: ScaleStar: budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In: IEEE International Conference on Advanced Information Networking and Applications, pp. 534–541 (2012)Google Scholar
  21. 21.
    Zhao, H., Sakellariou, R.: Scheduling multiple DAGs onto heterogeneous systems. In: International Parallel and Distributed Processing Symposium, pp. 1–14 (2006)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingChina

Personalised recommendations