Skip to main content

CDA: a novel multicore scheduling for cost-aware deadline-constrained scientific workflows on the IaaS cloud

The Journal of Supercomputing (2022)Cite this article

Abstract

The variety of pricing models offered by cloud service providers and the availability of a wide diversity of computing resources has increased the popularity of this paradigm for scientific applications. Such a scalable platform can be an ideal option for the execution of loosely coupled parallel applications, such as scientific workflows. Scientific workflows are regarded as one of the most important elements in different scientific fields in which a complex application may be divided into several dependent tasks. Given that the cost of leasing multicore VMs on the cloud will rise with an increase in the number of processing cores, an efficient scheduling algorithm focusing on utilizing multicore resources can significantly reduce execution costs. As an extension of its authors’ previous research, the current paper proposes a heuristic scheduling algorithm, the Cluster Dividing Algorithm that concentrates on expanding the utilization of multicore resources to reduce execution costs while also meeting the user-defined deadline. To increase resource utilization, the proposed scheduling employs different techniques, such as task clustering, directed graph leveling, and task duplicating. The experimental results reveal that the presented algorithm leads to lower execution costs while complying with the deadline.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. “Who’s Using Amazon Web Services? (2021) Contino | Global Transformation Consultancy.” [Online]. Available: https://www.contino.io/insights/whos-using-aws. [Accessed: 31-Oct-2021].

  2. Abrishami S, Naghibzadeh M, Epema DHJ (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener Comput Syst 29(1):158–169

    Article  Google Scholar 

  3. Wang Y, Guo Y, Wang W, Liang H, Huo S (2021) INHIBITOR: an intrusion tolerant scheduling algorithm in cloud-based scientific workflow system. Future Gener Comput Syst 114:272–284

    Article  Google Scholar 

  4. Naghibzadeh M (2016) Modeling and scheduling hybrid workflows of tasks and task interaction graphs on the cloud. Future Gener Comput Syst. https://doi.org/10.1016/j.future.2016.05.029

    Article  Google Scholar 

  5. Kozma D, Varga P, Larrinaga F (2021) Dynamic multilevel workflow management concept for industrial IoT systems. IEEE Trans Autom Sci Eng 18(3):1354–1366

    Article  Google Scholar 

  6. Yan J, Yang Y, Raikundalia GK (2006) SwinDeW—A P2P-based decentralized workflow management system. IEEE Trans Syst Man Cybern Part A Syst Humans 36(5):922–935

    Article  Google Scholar 

  7. Reichert M, Rinderle S, Dadam P (2003) “ADEPT workflow management system. Lect Notes Comput Sci 2678:370–379

    Article  Google Scholar 

  8. Fahringer T et al (2005) “ASKALON: a grid application development and computing environment. Proc IEEE/ACM Int Work Grid Comput 2005:122–131

    Google Scholar 

  9. Amin K, Von Laszewski G, Hategan M, Zaluzec NJ, Hampton S, Rossi A (2004) GridAnt: a client-controllable grid workflow system. Proc Hawaii Int Conf Syst Sci 37:3293–3301

    Google Scholar 

  10. Guan Z et al (2006) Grid-flow: a grid-enabled scientific workflow system with a Petri-net-based interface. Concurr Comput Pract Exp 18(10):1115–1140

    Article  Google Scholar 

  11. Altintas I, Berkley C, Jaeger E, Jones M, Ludäscher B, Mock S (2004) Kepler: an extensible system for design and execution of scientific workflows. Proc Int Conf Sci Stat Database Manag SSDBM 16:423–424

    Google Scholar 

  12. Deelman E et al (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program 13(3):219–237

    Google Scholar 

  13. Ahmad Z, Nazir B, Umer A (2021) A fault-tolerant workflow management system with quality-of-service-aware scheduling for scientific workflows in cloud computing. Int J Commun Syst 34(1):e4649

    Google Scholar 

  14. Dubey K, Shams MY, Sharma SC, Alarifi A, Amoon M, Nasr AA (2019) A management system for servicing multi-organizations on community cloud model in secure cloud environment. IEEE Access 7:159535–159546

    Article  Google Scholar 

  15. Nadjaran Toosi A, Sinnott RO, Buyya R (2018) Resource provisioning for data-intensive applications with deadline constraints on hybrid clouds using Aneka. Future Gener Comput Syst 79:765–775

    Article  Google Scholar 

  16. Zhang L, Zhou L, Salah A (2020) Efficient scientific workflow scheduling for deadline-constrained parallel tasks in cloud computing environments. Inf Sci (Ny) 531:31–46

    MathSciNet  MATH  Article  Google Scholar 

  17. Dubey K, Sharma SC (2020) An extended intelligent water drop approach for efficient VM allocation in secure cloud computing framework. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2020.11.001

    Article  Google Scholar 

  18. Dubey K, Sharma SC (2021) A novel multi-objective CR-PSO task scheduling algorithm with deadline constraint in cloud computing. Sustain Comput Inform Syst 32:100605

    Google Scholar 

  19. Rizvi N, Ramesh D (2020) Fair budget constrained workflow scheduling approach for heterogeneous clouds. Clust Comput 23(4):3185–3201

    Article  Google Scholar 

  20. Mohammadzadeh A, Masdari M, Gharehchopogh FS (2021) Energy and cost-aware workflow scheduling in cloud computing data centers using a multi-objective optimization algorithm. J Netw Syst Manag 29(3):1–34

    Article  Google Scholar 

  21. Iranmanesh A, Naji HR (2020) DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing. Clust Comput 24(2):667–681

    Article  Google Scholar 

  22. Ahmad W, Alam B, Atman A (2021) An energy-efficient big data workflow scheduling algorithm under budget constraints for heterogeneous cloud environment. J Supercomput 77(10):11946–11985

    Article  Google Scholar 

  23. Ismayilov G, Topcuoglu HR (2020) Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing. Future Gener Comput Syst 102:307–322

    Article  Google Scholar 

  24. Chakravarthi KK, Shyamala L (2021) TOPSIS inspired budget and deadline aware multi-workflow scheduling for cloud computing. J Syst Archit 114:101916

    Article  Google Scholar 

  25. Belgacem A, Beghdad-Bey K (2021) Multi-objective workflow scheduling in cloud computing: trade-off between makespan and cost. Clust Comput 2021:1–17

    Google Scholar 

  26. Bugingo E, Zhang D, Chen Z, Zheng W (2020) Towards decomposition based multi-objective workflow scheduling for big data processing in clouds. Clust Comput 24(1):115–139

    Article  Google Scholar 

  27. Deldari A, Naghibzadeh M, Abrishami S (2017) CCA: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J Supercomput 73(2):756–781

    Article  Google Scholar 

  28. Malawski M, Juve G, Deelman E, Nabrzyski J (2015) Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Future Gener Comput Syst 48:1–18

    Article  Google Scholar 

  29. Smanchat S, Viriyapant K (2015) Taxonomies of workflow scheduling problem and techniques in the cloud. Future Gener Comput Syst 52:1–12

    Article  Google Scholar 

  30. Zhang M, Li H, Liu L, Buyya R (2018) An adaptive multi-objective evolutionary algorithm for constrained workflow scheduling in clouds. Distrib Parallel Databases 36(2):339–368

    Article  Google Scholar 

  31. Chakravarthi KK, Shyamala L, Vaidehi V (2021) Cost-effective workflow scheduling approach on cloud under deadline constraint using firefly algorithm. Appl Intell 51(3):1629–1644

    Article  Google Scholar 

  32. Paknejad P, Khorsand R, Ramezanpour M (2021) Chaotic improved PICEA-g-based multi-objective optimization for workflow scheduling in cloud environment. Future Gener Comput Syst 117:12–28

    Article  Google Scholar 

  33. Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Futue Gener Comput Syst 83:14–26

    Article  Google Scholar 

  34. Wu Q, Zhou M, Zhu Q, Xia Y, Wen J (2019) Moels: multiobjective evolutionary list scheduling for cloud workflows. IEEE Trans Autom Sci Eng 17(1):166–176

    Article  Google Scholar 

  35. Alkhanak EN, Lee SP (2018) A hyper-heuristic cost optimisation approach for scientific workflow scheduling in cloud computing. Future Gener Comput Syst 86:480–506

    Article  Google Scholar 

  36. Singh P, Dutta M, Aggarwal N (2021) Hybrid meta-heuristic approach for workflow scheduling in IaaS cloud. Arab J Sci Eng. https://doi.org/10.1007/s13369-021-05774-6

    Article  Google Scholar 

  37. Thekkepuryil JKV, Suseelan DP, Keerikkattil PM (2021) An effective meta-heuristic based multi-objective hybrid optimization method for workflow scheduling in cloud computing environment. Clust Comput 24(3):1–18

    Google Scholar 

  38. Shirvani MH (2020) A hybrid meta-heuristic algorithm for scientific workflow scheduling in heterogeneous distributed computing systems. Eng Appl Artif Intell 90:103501

    Article  Google Scholar 

  39. Maheswaran M, Ali S, Siegel HJ, Hensgen D, Freund RF (1999) Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. J Parallel Distrib Comput 59(2):107–131

    Article  Google Scholar 

  40. Kwok Y-K, Maciejewski AA, Siegel HJ, Ahmad I, Ghafoor A (2006) A semi-static approach to mapping dynamic iterative tasks onto heterogeneous computing systems. J Parallel Distrib Comput 66(1):77–98

    MATH  Article  Google Scholar 

  41. Kwok Y-K, Ahmad I (1996) Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors. Parallel Distrib Syst IEEE Trans 7(5):506–521

    Article  Google Scholar 

  42. Topcuoglu H, Hariri S, Wu M (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. Parallel Distrib Syst IEEE Trans 13(3):260–274

    Article  Google Scholar 

  43. Chang W-L, Ren T-T, Feng M (2015) Quantum algorithms and mathematical formulations of biomolecular solutions of the vertex cover problem in the finite-dimensional Hilbert space. Nano Biosci IEEE Trans 14(1):121–128

    Article  Google Scholar 

  44. Abazari F, Analoui M, Takabi H, Fu S (2019) MOWS: multi-objective workflow scheduling in cloud computing based on heuristic algorithm. Simul Model Pract Theory 93:119–132

    Article  Google Scholar 

  45. Rizvi N, Ramesh D (2020) HBDCWS: heuristic-based budget and deadline constrained workflow scheduling approach for heterogeneous clouds. Soft Comput 24(24):18971–18990

    Article  Google Scholar 

  46. Ahmad I, Kwok Y-K (1998) On exploiting task duplication in parallel program scheduling. IEEE Trans parallel Distrib Syst 9(9):872–892

    Article  Google Scholar 

  47. Ilavarasan E, Thambidurai P (2005) Levelized scheduling of directed a-cyclic precedence constrained task graphs onto heterogeneous computing system. In: First International Conference on Distributed Frameworks for Multimedia Applications, pp 262–269

  48. Bittencourt LF, Madeira ERM (2011) HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds. J Internet Serv Appl 2(3):207–227

    Article  Google Scholar 

  49. Amazon EC2 Pricing–Amazon Web Services (2021) [Online]. Available: https://aws.amazon.com/ec2/pricing/. [Accessed: 11-Nov-2021].

  50. Medara R, Singh RS (2021) Energy efficient and reliability aware workflow task scheduling in cloud environment. Wirel Pers Commun 119(2):1301–1320

    Article  Google Scholar 

  51. Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Workflows in Support of Large-Scale Science, pp 1–10.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, University of Torbat Heydarieh, Torbat Heydariyeh, Iran

    Arash Deldari & Alireza Salehan

  2. Department of Computer Engineering, Salman Institute of Higher Education, Mashhad, Iran

    Abolghasem Yousofi

  3. Department of Computer Engineering, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

    Mahmoud Naghibzadeh

Authors
  1. Arash Deldari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abolghasem Yousofi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahmoud Naghibzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alireza Salehan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arash Deldari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Deldari, A., Yousofi, A., Naghibzadeh, M. et al. CDA: a novel multicore scheduling for cost-aware deadline-constrained scientific workflows on the IaaS cloud. J Supercomput (2022). https://doi.org/10.1007/s11227-022-04551-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-022-04551-y

Keywords

  • Workflow scheduling
  • Cloud computing
  • Infrastructure as a service
  • Multicore VMs
  • Clustering