Abstract
The variety of pricing models offered by cloud service providers and the availability of a wide diversity of computing resources has increased the popularity of this paradigm for scientific applications. Such a scalable platform can be an ideal option for the execution of loosely coupled parallel applications, such as scientific workflows. Scientific workflows are regarded as one of the most important elements in different scientific fields in which a complex application may be divided into several dependent tasks. Given that the cost of leasing multicore VMs on the cloud will rise with an increase in the number of processing cores, an efficient scheduling algorithm focusing on utilizing multicore resources can significantly reduce execution costs. As an extension of its authors’ previous research, the current paper proposes a heuristic scheduling algorithm, the Cluster Dividing Algorithm that concentrates on expanding the utilization of multicore resources to reduce execution costs while also meeting the user-defined deadline. To increase resource utilization, the proposed scheduling employs different techniques, such as task clustering, directed graph leveling, and task duplicating. The experimental results reveal that the presented algorithm leads to lower execution costs while complying with the deadline.
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
“Who’s Using Amazon Web Services? (2021) Contino | Global Transformation Consultancy.” [Online]. Available: https://www.contino.io/insights/whos-using-aws. [Accessed: 31-Oct-2021].
Abrishami S, Naghibzadeh M, Epema DHJ (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener Comput Syst 29(1):158–169
Wang Y, Guo Y, Wang W, Liang H, Huo S (2021) INHIBITOR: an intrusion tolerant scheduling algorithm in cloud-based scientific workflow system. Future Gener Comput Syst 114:272–284
Naghibzadeh M (2016) Modeling and scheduling hybrid workflows of tasks and task interaction graphs on the cloud. Future Gener Comput Syst. https://doi.org/10.1016/j.future.2016.05.029
Kozma D, Varga P, Larrinaga F (2021) Dynamic multilevel workflow management concept for industrial IoT systems. IEEE Trans Autom Sci Eng 18(3):1354–1366
Yan J, Yang Y, Raikundalia GK (2006) SwinDeW—A P2P-based decentralized workflow management system. IEEE Trans Syst Man Cybern Part A Syst Humans 36(5):922–935
Reichert M, Rinderle S, Dadam P (2003) “ADEPT workflow management system. Lect Notes Comput Sci 2678:370–379
Fahringer T et al (2005) “ASKALON: a grid application development and computing environment. Proc IEEE/ACM Int Work Grid Comput 2005:122–131
Amin K, Von Laszewski G, Hategan M, Zaluzec NJ, Hampton S, Rossi A (2004) GridAnt: a client-controllable grid workflow system. Proc Hawaii Int Conf Syst Sci 37:3293–3301
Guan Z et al (2006) Grid-flow: a grid-enabled scientific workflow system with a Petri-net-based interface. Concurr Comput Pract Exp 18(10):1115–1140
Altintas I, Berkley C, Jaeger E, Jones M, Ludäscher B, Mock S (2004) Kepler: an extensible system for design and execution of scientific workflows. Proc Int Conf Sci Stat Database Manag SSDBM 16:423–424
Deelman E et al (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program 13(3):219–237
Ahmad Z, Nazir B, Umer A (2021) A fault-tolerant workflow management system with quality-of-service-aware scheduling for scientific workflows in cloud computing. Int J Commun Syst 34(1):e4649
Dubey K, Shams MY, Sharma SC, Alarifi A, Amoon M, Nasr AA (2019) A management system for servicing multi-organizations on community cloud model in secure cloud environment. IEEE Access 7:159535–159546
Nadjaran Toosi A, Sinnott RO, Buyya R (2018) Resource provisioning for data-intensive applications with deadline constraints on hybrid clouds using Aneka. Future Gener Comput Syst 79:765–775
Zhang L, Zhou L, Salah A (2020) Efficient scientific workflow scheduling for deadline-constrained parallel tasks in cloud computing environments. Inf Sci (Ny) 531:31–46
Dubey K, Sharma SC (2020) An extended intelligent water drop approach for efficient VM allocation in secure cloud computing framework. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2020.11.001
Dubey K, Sharma SC (2021) A novel multi-objective CR-PSO task scheduling algorithm with deadline constraint in cloud computing. Sustain Comput Inform Syst 32:100605
Rizvi N, Ramesh D (2020) Fair budget constrained workflow scheduling approach for heterogeneous clouds. Clust Comput 23(4):3185–3201
Mohammadzadeh A, Masdari M, Gharehchopogh FS (2021) Energy and cost-aware workflow scheduling in cloud computing data centers using a multi-objective optimization algorithm. J Netw Syst Manag 29(3):1–34
Iranmanesh A, Naji HR (2020) DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing. Clust Comput 24(2):667–681
Ahmad W, Alam B, Atman A (2021) An energy-efficient big data workflow scheduling algorithm under budget constraints for heterogeneous cloud environment. J Supercomput 77(10):11946–11985
Ismayilov G, Topcuoglu HR (2020) Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing. Future Gener Comput Syst 102:307–322
Chakravarthi KK, Shyamala L (2021) TOPSIS inspired budget and deadline aware multi-workflow scheduling for cloud computing. J Syst Archit 114:101916
Belgacem A, Beghdad-Bey K (2021) Multi-objective workflow scheduling in cloud computing: trade-off between makespan and cost. Clust Comput 2021:1–17
Bugingo E, Zhang D, Chen Z, Zheng W (2020) Towards decomposition based multi-objective workflow scheduling for big data processing in clouds. Clust Comput 24(1):115–139
Deldari A, Naghibzadeh M, Abrishami S (2017) CCA: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J Supercomput 73(2):756–781
Malawski M, Juve G, Deelman E, Nabrzyski J (2015) Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Future Gener Comput Syst 48:1–18
Smanchat S, Viriyapant K (2015) Taxonomies of workflow scheduling problem and techniques in the cloud. Future Gener Comput Syst 52:1–12
Zhang M, Li H, Liu L, Buyya R (2018) An adaptive multi-objective evolutionary algorithm for constrained workflow scheduling in clouds. Distrib Parallel Databases 36(2):339–368
Chakravarthi KK, Shyamala L, Vaidehi V (2021) Cost-effective workflow scheduling approach on cloud under deadline constraint using firefly algorithm. Appl Intell 51(3):1629–1644
Paknejad P, Khorsand R, Ramezanpour M (2021) Chaotic improved PICEA-g-based multi-objective optimization for workflow scheduling in cloud environment. Future Gener Comput Syst 117:12–28
Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Futue Gener Comput Syst 83:14–26
Wu Q, Zhou M, Zhu Q, Xia Y, Wen J (2019) Moels: multiobjective evolutionary list scheduling for cloud workflows. IEEE Trans Autom Sci Eng 17(1):166–176
Alkhanak EN, Lee SP (2018) A hyper-heuristic cost optimisation approach for scientific workflow scheduling in cloud computing. Future Gener Comput Syst 86:480–506
Singh P, Dutta M, Aggarwal N (2021) Hybrid meta-heuristic approach for workflow scheduling in IaaS cloud. Arab J Sci Eng. https://doi.org/10.1007/s13369-021-05774-6
Thekkepuryil JKV, Suseelan DP, Keerikkattil PM (2021) An effective meta-heuristic based multi-objective hybrid optimization method for workflow scheduling in cloud computing environment. Clust Comput 24(3):1–18
Shirvani MH (2020) A hybrid meta-heuristic algorithm for scientific workflow scheduling in heterogeneous distributed computing systems. Eng Appl Artif Intell 90:103501
Maheswaran M, Ali S, Siegel HJ, Hensgen D, Freund RF (1999) Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. J Parallel Distrib Comput 59(2):107–131
Kwok Y-K, Maciejewski AA, Siegel HJ, Ahmad I, Ghafoor A (2006) A semi-static approach to mapping dynamic iterative tasks onto heterogeneous computing systems. J Parallel Distrib Comput 66(1):77–98
Kwok Y-K, Ahmad I (1996) Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors. Parallel Distrib Syst IEEE Trans 7(5):506–521
Topcuoglu H, Hariri S, Wu M (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. Parallel Distrib Syst IEEE Trans 13(3):260–274
Chang W-L, Ren T-T, Feng M (2015) Quantum algorithms and mathematical formulations of biomolecular solutions of the vertex cover problem in the finite-dimensional Hilbert space. Nano Biosci IEEE Trans 14(1):121–128
Abazari F, Analoui M, Takabi H, Fu S (2019) MOWS: multi-objective workflow scheduling in cloud computing based on heuristic algorithm. Simul Model Pract Theory 93:119–132
Rizvi N, Ramesh D (2020) HBDCWS: heuristic-based budget and deadline constrained workflow scheduling approach for heterogeneous clouds. Soft Comput 24(24):18971–18990
Ahmad I, Kwok Y-K (1998) On exploiting task duplication in parallel program scheduling. IEEE Trans parallel Distrib Syst 9(9):872–892
Ilavarasan E, Thambidurai P (2005) Levelized scheduling of directed a-cyclic precedence constrained task graphs onto heterogeneous computing system. In: First International Conference on Distributed Frameworks for Multimedia Applications, pp 262–269
Bittencourt LF, Madeira ERM (2011) HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds. J Internet Serv Appl 2(3):207–227
Amazon EC2 Pricing–Amazon Web Services (2021) [Online]. Available: https://aws.amazon.com/ec2/pricing/. [Accessed: 11-Nov-2021].
Medara R, Singh RS (2021) Energy efficient and reliability aware workflow task scheduling in cloud environment. Wirel Pers Commun 119(2):1301–1320
Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Workflows in Support of Large-Scale Science, pp 1–10.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Deldari, A., Yousofi, A., Naghibzadeh, M. et al. CDA: a novel multicore scheduling for cost-aware deadline-constrained scientific workflows on the IaaS cloud. J Supercomput 78, 17027–17054 (2022). https://doi.org/10.1007/s11227-022-04551-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-022-04551-y