Abstract
Workflow scheduling in the cloud is the process of allocating tasks to limited cloud resources to maximise resource utilization and minimise makespan. This is often achieved by adopting an effective scheduling heuristic. Most existing heuristics rely on a small number of features when making scheduling decisions, ignoring many impacting factors that are important to workflow scheduling. For example, the MINMIN algorithm only considers the size of the tasks when making scheduling decisions. Meanwhile, many existing works focused on scheduling a static set of workflow tasks, neglecting the dynamic nature of cloud computing. In this paper, we introduce a new and more realistic workflow scheduling problem that considers different kinds of workflows, cloud resources, and impacting features. We propose a Dynamic Workflow Scheduling Genetic Programming (DSGP) algorithm to automatically design scheduling heuristics for workflow scheduling to minimise the overall makespan of executing a long sequence of dynamically arriving workflows. Our proposed DSGP algorithm can work consistently well regardless of the size of workflows, the number of available resources, or the pattern of workflows. It is evaluated on a well-known benchmark dataset by using the popular WorkflowSim simulator. Our experiments show that scheduling heuristics designed by DSGP can significantly outperform several manually designed and widely used workflow scheduling heuristics.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdelkader, D.M., Omara, F.: Dynamic task scheduling algorithm with load balancing for heterogeneous computing system. Egypt. Inform. J. 13(2), 135–145 (2012)
Arabnejad, V., Bubendorfer, K., Ng, B.: Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1), 29–44 (2018)
Blythe, J., et al.: Task scheduling strategies for workflow-based applications in grids. In: 2005 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2005, vol. 2, pp. 759–767. IEEE (2005)
Braun, T.D., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 61(6), 810–837 (2001)
Buyya, R., Murshed, M.: GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. Concur. Comput.: Pract. Exp. 14(13–15), 1175–1220 (2002)
Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A., Buyya, R.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw.: Pract. Exp. 41(1), 23–50 (2011)
Chen, W., Deelman, E.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)
Mahmood, A., Khan, S.A., Bahlool, R.A.: Hard real-time task scheduling in cloud computing using an adaptive genetic algorithm. Computers 6(2), 15 (2017)
Masood, A., Mei, Y., Chen, G., Zhang, M.: Many-objective genetic programming for job-shop scheduling. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 209–216. IEEE (2016)
Nguyen, S., Zhang, M., Johnston, M., Tan, K.C.: A computational study of representations in genetic programming to evolve dispatching rules for the job shop scheduling problem. IEEE Trans. Evol. Comput. 17(5), 621–639 (2012)
Raghavan, S., Sarwesh, P., Marimuthu, C., Chandrasekaran, K.: Bat algorithm for scheduling workflow applications in cloud. In: 2015 International Conference on Electronic Design, Computer Networks & Automated Verification (EDCAV), pp. 139–144. IEEE (2015)
Rahman, M., Hassan, R., Ranjan, R., Buyya, R.: Adaptive workflow scheduling for dynamic grid and cloud computing environment. Concurr. Comput.: Pract. Exp. 25(13), 1816–1842 (2013)
Sahni, J., Vidyarthi, D.P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2015)
Sonmez, O., Yigitbasi, N., Abrishami, S., Iosup, A., Epema, D.: Performance analysis of dynamic workflow scheduling in multicluster grids. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp. 49–60 (2010)
Tay, J.C., Ho, N.B.: Evolving dispatching rules using genetic programming for solving multi-objective flexible job-shop problems. Comput. Ind. Eng. 54(3), 453–473 (2008)
Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M., et al.: Workflows for e-Science: Scientific Workflows for Grids, vol. 1. Springer, London (2007). https://doi.org/10.1007/978-1-84628-757-2
Xie, J., Mei, Y., Ernst, A.T., Li, X., Song, A.: A genetic programming-based hyper-heuristic approach for storage location assignment problem. In: 2014 IEEE congress on evolutionary computation (CEC), pp. 3000–3007. IEEE (2014)
Yu, Y., Feng, Y., Ma, H., Chen, A., Wang, C.: Achieving flexible scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 3102–3109. IEEE (2019)
Zhan, Z.H., Liu, X.F., Gong, Y.J., Zhang, J., Chung, H.S.H., Li, Y.: Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Comput. Surv. (CSUR) 47(4), 63 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Escott, KR., Ma, H., Chen, G. (2020). Genetic Programming Based Hyper Heuristic Approach for Dynamic Workflow Scheduling in the Cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2020. Lecture Notes in Computer Science(), vol 12392. Springer, Cham. https://doi.org/10.1007/978-3-030-59051-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-59051-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59050-5
Online ISBN: 978-3-030-59051-2
eBook Packages: Computer ScienceComputer Science (R0)