Abstract
Cloud computing, emerged as a commercial service model, has been widely concerned in both industry and academia. Massive workflow applications could be performed simultaneously on the cloud platforms, which significantly benefits from the elasticity and convenience of cloud computing. However, it is still a challenge to schedule virtualized resources for the concurrent workflows in cloud environment, with limited high-performance resources in a timesaving and efficient manner. In view of this challenge, a dynamic scheduling method for concurrent workflows, named as DSM, in cloud environment is proposed to satisfy the various resource requirements of the workflows. Technically, a time overhead model for the workflows and a resource utilization model for cloud datacenter are presented. Then a relevant dynamic scheduling method is designed based on critical path lookup, which aims at minimizing the makespan of workflows, and maximizing the resource utilization of the datacenter during the execution of the workflows. Extensive experimental evaluations demonstrate the efficiency and effectiveness of our proposed method.
Similar content being viewed by others
References
Netjinda, N., Sirinaovakul, B., Achalakul, T.: Cost optimal scheduling in IaaS for dependent workload with particle swarm optimization. J. Supercomput. 68(3), 1579–1603 (2014)
Rao, J., Zhou, X.: Towards fair and efficient SMP virtual machine scheduling. In: ACM Sigplan Symposium on Principles and Practice of Parallel Programming. vol. 49(8), pp. 273–286 (2014)
Wang, P., Huang, Y., Li, K., Guo, Y.: Load balancing degree first algorithm on phase space for cloud computing cluster. J. Comput. Res. Dev. 51(5), 1095–1107 (2014)
Armbrust, M., Fox, A., Griffith, R., et al.: Above the Clouds: A Berkeley View of Cloud Computing. Eecs Department University of California Berkeley, vol. 53(4), pp. 50–58 (2009)
Zhang, S., Qian, Z., Wu, J., Lu, S., Epstein, L.: Virtual network embedding with opportunistic resource sharing. IEEE Trans. Parallel Distrib. Syst. 25(3), 816–827 (2014)
Li, W., Zhang, Q., Wu, J., Li, J., Hao, H.: Trust-driven and QoS demand clustering analysis based cloud workflow scheduling strategies. Clust. Comput. 17(3), 1–18 (2014)
Dou, W., Xu, X., Meng, S., Zhang, X., Hu, C., Yu, S., Yang, J.: An energy-aware virtual machine scheduling method for service QoS enhancement in clouds over big data. Concurr. Comput. Pract. Exp. 29(14), 1–20 (2017)
Shen, H., Li, X.: Algorithm for the cloud service workflow scheduling with setup time and deadline constraints. J. Commun. 36(6), 183–192 (2015)
Guo, H., Chen, Z., Yu, Y., Wang, Y., Chen, X.: A communication aware DAG workflow cost optimization model and algorithm. J. Comput. Res. Dev. 52(6), 1400–1408 (2015)
Chen, W., Lee, Y.C., Fekete, A., Zomaya, A.Y.: Adaptive multiple-workflow scheduling with task rearrangement. J. Supercomput. 71(4), 1297–1317 (2015)
Dong, J., WANG, H., Cheng, S.: Energy-performance tradeoffs in laaS cloud with virtual machine scheduling. China Commun. 12(2), 155–166 (2015)
Durao, F., Carvalho, J.F.S., Fonseka, A., Garcia, V.C.: A systematic review on cloud computing. J. Supercomput. 68(3), 1321–1346 (2014)
Ahmad, S.G., Liew, C.S., Rafique, M.M., Munir, E.U., Khan, S.U.: Data-intensive workflow optimization based on application task graph partitioning in heterogeneous computing systems. In: IEEE Fourth International Conference on Big Data and Cloud Computing pp. 129–136 (2015)
Chen, C.: Workflow task scheduling in cloud computing based on hybrid improved CS algorithm and decision tree. J. Univ. Electron. Sci. Technol. China 45(6), 974–980 (2016)
Liu, J.X., Yang, X.F., X. Y.: Cloud workflow scheduling method based on batch processing strategy. Comput. Integr. Manufac. Syst. 21(2), 336–343 (2015)
Luo, Z., Wang, P., You, B., Jie, S.U.: Optimization scheduling of workflow’s accuracy based on reverse reduction under constraint time. J. Beijing Univ. Posts Telecommun. 40(1), 99–104 (2017)
Doulamis, N.D., Kokkino, P., Varvarigos, E.: Resource selection for tasks with time requirements using spectral clustering. IEEE Trans. Comput. 63(2), 461–474 (2014)
Kong, Y., Zhang, M., Ye, D.: A belief propagation-based method for task allocation in open and dynamic cloud environments. Knowl. Based Syst. 115, 123–132 (2016)
Xing, G., Xu, X., Xiang, H., Xue, S., Ji, S., Yang, J.: Fair energy-efficient virtual machine scheduling for internet of things applications in cloud environment. Int. J. Distrib. Sensor Netw. 13(2), 1–11 (2017)
Hao, L., Cui, G., Qu, M., Ke, W.: Resource scheduling optimization algorithm of energy consumption for cloud computing based on task tolerance. J. Softw. 9(4), 895–901 (2014)
Cao, F., Zhu, M.M., Wu, C.Q.: Energy-efficient resource management for scientific workflows in clouds. In: 2014 IEEE World Congress on Services (SERVICES), pp. 402–409 (2014)
Jrad, F., Tao, J., Brandic, I., Streit, A.: SLA enactment for large-scale healthcare workflows on multi-Cloud. Future Gener. Comput. Syst. 43(4), 135–148 (2015)
Wang, Y., Wang, J., Han, Y.: A two-stage resource scheduling method for workflow cloud computing system. J. Sourth China Univ. Technol. 45(1), 80–87 (2017)
Rodriguez, M.A., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)
Luo, Z.Y., Wang, P., You, B., Zhu, X.S.: Serial reduction optimization research of complex product workflow’s accuracy under the tine constraint. Adv. Mech. Eng. 8(10), 1–9 (2016)
Ahmed, W., Wu, Y.: Estimation of cloud node acquisition. Tsinghua Sci. Technol. 19(1), 1–12 (2014)
Cao, B., Wang, X., Xiong, L., Fan, J.: Searching method for partical swarm optimization of cloud workflow scheduling with time constraint. Comput. Integr. Manufac. Syst. 22(2), 372–380 (2016)
Xu, Y., Li, K., Hu, J., Li, K.: A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues. Inf. Sci. 270(6), 255–287 (2014)
Chen, H., Zhu, J., Manho, M.A., Zhu, X.: Scheduling for stochastic tasks and resources in virtualized clouds. Syst. Eng. Electron. 39(2), 348–354 (2017)
Wu, C.M., Chang, R.S., Chan, H.Y.: A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters. Future Gener. Comput. Syst. 37(7), 141–147 (2014)
Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific work-flows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)
Guo, Y.Q., Song, J.X.: Optimal task-level scheduling based on multimedia applications in cloud. Comput. Sci. 42(11), 413–416 (2015)
Mandal, Vasundhara, D., Kar, R., Ghoshal, S.P.: Digital FIR filter design using fitness based hybrid adaptive differential evolution with particle swarm optimization. Nat. Comput. 13(1), 55–64 (2014)
Prasad, A.S., Rao, S.: A mechanism design approach to resource procurement in cloud computing. IEEE Trans. Comput. 63(1), 17–30 (2014)
Chen, H.K., Zhu, J.H., Zhu, X.M., Ma, M.H., Zhang, Z.S.: Resource-delay-aware scheduling for real-time tasks in clouds. J. Comput. Res. Dev. 54(2), 446–456 (2017)
Xu, X., Dou, W., Zhang, X., Chen, J.: EnReal: an energy-aware resource allocation method for scientific workflow executions in cloud environment. IEEE Trans. Cloud Comput. 4, 166–179 (2016)
Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener. Comput. Syst. 29(1), 158–169 (2013)
Yang, G., Stolyar, A.L., Walid, A.: Shadow-routing based dynamic algorithms for virtual machine placement in a network cloud. IEEE Infocom 12(11), 620–628 (2013)
Li, X., Wu, J., Tang, S., Lu, S.: Let’s stay together: towards traffic aware virtual machine placement in data centers. In: 2014 IEEE Conference on Computer Communications (INFOCOM), pp. 1842–1850 (2014)
Xiaohu, W., Loiseau, P.: Algorithms for scheduling deadline-sensitive malleable tasks. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 530–537 (2015)
Acknowledgements
This research is supported by the National Science Foundation of China under Grant Nos. 61702277, 61672276, 61402167 and 61672290. Besides, this work is also supported by The Startup Foundation for Introducing Talent of NUIST, the open project from State Key Laboratory for Novel Software Technology, Nanjing University under grant no. KFKT2017B04, the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund, Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET), the project “Six Talent Peaks Project in Jiangsu Province” under grant no. XYDXXJS-040, and Innovation Platform Open Foundation of Hunan Provincial Education Department (No. 17K033).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xue, S., Peng, Y., Xu, X. et al. DSM: a dynamic scheduling method for concurrent workflows in cloud environment. Cluster Comput 22 (Suppl 1), 693–706 (2019). https://doi.org/10.1007/s10586-017-1189-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-017-1189-5