Skip to main content
SpringerLink
Log in

Electricity-cost-aware multi-workflow scheduling in heterogeneous cloud

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Multi-workflows are commonly deployed on cloud platforms to achieve efficient computational power. Diverse task configuration requirements, the heterogeneous nature and dynamic electricity price of cloud servers impose significant challenges for economically scheduling multi-workflows. In this paper, we propose a Heuristic Electricity-cost-aware Multi-workflow Scheduling algorithm (HEMS) to search for an optimal scheduling plan which determines the optimal scheduling scheme for each task in each workflow, specifying the server to perform the task with determined resources in specific time. The objective is to minimize the total electricity cost of all servers while satisfying the deadline constraints of all workflows. The HEMS algorithm consists of five components: Workflow Scheduling Sequence Generation, Task Scheduling Sequence Initialization for each workflow, Optimal Scheduling Scheme Determination for each task, initial Task Scheduling Sequence Optimization, and Optimal Scheduling Plan Optimization. Experimental results demonstrate that HEMS consistently achieves the optimal scheduling plan with the lower total electricity cost (saving 54.5–69.1% on average) within slightly longer CPU time for various multi-workflows compared to existing three scheduling approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://docs.aws.amazon.com/zh_cn/AmazonECS/latest/developerguide/task_definition_parameters.htm.

References

  1. Elshamy A, Alquraan A, Al-Kiswany S (2023) A study of orchestration approaches for scientific workflows in serverless computing. In: 1st workshop on serverless systems, applications and methodologies (SESAME), pp. 34–40

  2. Li X, Yu W, Ruiz R, Zhu J (2022) Energy-aware cloud workflow applications scheduling with geo-distributed data. IEEE Trans Serv Comput 15(2):891–903

    Article  Google Scholar 

  3. Ebrahimi K, Jones GF, Fleischer AS (2014) A review of data center cooling technology, operating conditions and the corresponding low-grade waste heat recovery opportunities. Renew Sustain Energy Rev 31:622–638

    Article  Google Scholar 

  4. Taghinezhad-Niar A, Pashazadeh S, Taheri J (2022) Energy-efficient workflow scheduling with budget-deadline constraints for cloud. Computing 104(3):1–25

    Article  MathSciNet  Google Scholar 

  5. Arabnejad V, Bubendorfer K, Ng B (2018) Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans Parallel Distrib Syst 30(1):29–44

    Article  Google Scholar 

  6. Hussain M, Wei L-F, Lakhan A, Wali S, Ali S, Hussain A (2021) Energy and performance-efficient task scheduling in heterogeneous virtualized cloud computing. Sustain Comput Inf Syst 30:1–12

    Google Scholar 

  7. Zhang C, Wang Y, Feng Z, Guo H (2017) Power consumption optimization for deadline-constrained workflows in cloud data center. In: 2017 IEEE international symposium on parallel and distributed processing with applications (ISPA), Guangzhou, China, pp 206–213

  8. Xu H, Li R, Pan C, Li K (2019) Minimizing energy consumption with reliability goal on heterogeneous embedded systems. J Parallel Distrib Comput 127:44–57

    Article  Google Scholar 

  9. Xie G, Jiang J, Liu Y, Li R, Li K (2017) Minimizing energy consumption of real-time parallel applications using downward and upward approaches on heterogeneous systems. IEEE Trans Ind Inf 13(3):1068–1078

    Article  Google Scholar 

  10. Zhong Z, Buyya R (2020) A cost-efficient container orchestration strategy in kubernetes-based cloud computing infrastructures with heterogeneous resources. ACM Trans Internet Technol 2(20):1–24

    Article  Google Scholar 

  11. Wang Y, Zuo X, Wu Z, Wang H, Zhao X (2022) Variable neighborhood search based multiobjective aco-list scheduling for cloud workflows. J Supercomput 78(17):18856–18886

    Article  Google Scholar 

  12. Wang Y, Zuo X (2021) An effective cloud workflow scheduling approach combining PSO and idle time slot-aware rules. IEEE/CAA J Automatica Sinica 8(5):1079–1094

    Article  Google Scholar 

  13. Alboaneen D, Tianfield H, Zhang Y, Pranggono B (2021) A metaheuristic method for joint task scheduling and virtual machine placement in cloud data centers. Futur Gener Comput Syst 115:201–212

    Article  Google Scholar 

  14. Shan X, Zhang H, Xie Y (2023) Two stage coevolutionary genetic algorithm with two dimensional coding for cloud workflow scheduling. Comput Integr Manuf Syst 29(2):568–580

    Google Scholar 

  15. Cheng D, Zhou X, Lama P, Ji M, Jiang C (2017) Energy efficiency aware task assignment with DVFS in heterogeneous hadoop clusters. IEEE Trans Parallel Distrib Syst 29(1):70–82

    Article  Google Scholar 

  16. Mainak Adhikari SNS (2019) Multi-objective accelerated particle swarm optimization with a container-based scheduling for internet-of-things in cloud environment. J Netw Comput Appl 137:35–61

    Article  Google Scholar 

  17. Li F, Tan WJ, Cai W (2022) A wholistic optimization of containerized workflow scheduling and deployment in the cloud-edge environment. Simul Model Pract Theory Int J Feder Eur Simul Soc 118:102521–102536

    Article  Google Scholar 

  18. Yuan H, Liu H, Bi J, Zhou M (2020) Revenue and energy cost-optimized biobjective task scheduling for green cloud data centers. IEEE Trans Autom Sci Eng 18(2):817–830

    Article  Google Scholar 

  19. Tao S, Xia Y, Ye L, Yan C, Gao R (2023) Db-aco: a deadline-budget constrained ant colony optimization for workflow scheduling in clouds. IEEE Trans Autom Sci Eng. https://doi.org/10.1109/TASE.2023.3247973

    Article  Google Scholar 

  20. Kaur A, Kaur B (2022) Load balancing optimization based on hybrid heuristic-metaheuristic techniques in cloud environment. J King Saud Univ Comput Inf Sci 34(3):813–824

    Google Scholar 

  21. Yuan H, Bi J, Tan W, Li BH (2016) Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans Autom Sci Eng 14(1):337–348

    Article  Google Scholar 

  22. Zhou L, Bhuyan LN, Ramakrishnan K (2019) Goldilocks: Adaptive resource provisioning in containerized data centers. In: 39th IEEE international conference on distributed computing systems (ICDCS), Dallas, TX, USA, pp 666–677

  23. Abdullah M, Iqbal W, Bukhari F, Erradi A (2020) Diminishing returns and deep learning for adaptive CPU resource allocation of containers. IEEE Trans Netw Serv Manage 17(4):2052–2063

    Article  Google Scholar 

  24. Zhang W, Wen Y, Lai LL, Liu F, Fan R (2017) Electricity cost minimization for interruptible workload in datacenter servers. IEEE Trans Serv Comput 13(6):1059–1071

    Google Scholar 

  25. Taghinezhad-Niar A, Taheri J (2023) Reliability, rental-cost and energy-aware multi-workflow scheduling on multi-cloud systems. IEEE Trans Cloud Comput 11(3):2681–2692

    Google Scholar 

  26. Xia Y, Zhan Y, Dai L, Chen Y (2023) A cost and makespan aware scheduling algorithm for dynamic multi-workflow in cloud environment. J Supercomput 79:1814–1833

    Article  Google Scholar 

  27. Li H, Xu G, Wang D, Zhou M, Yuan Y, Alabdulwahab A (2022) Chaotic-nondominated-sorting owl search algorithm for energy-aware multi-workflow scheduling in hybrid clouds. IEEE Trans Serv Comput 7(3):595–608

    Google Scholar 

  28. Wang X, Cao J, Buyya R (2023) Adaptive cloud bundle provisioning and multi-workflow scheduling via coalition reinforcement learning. IEEE Trans Comput 72(4):1041–1054

    Article  ADS  Google Scholar 

  29. Li H, Huang J, Wang B, Fan Y (2022) Weighted double deep q-network based reinforcement learning for bi-objective multi-workflow scheduling in the cloud. Clust Comput 25:751–768

    Article  Google Scholar 

  30. Qureshi B (2019) Profile-based power-aware workflow scheduling framework for energy-efficient data centers. Futur Gener Comput Syst 94:453–467

    Article  Google Scholar 

  31. Feller E, Rilling L, Morin C (2011) Energy-aware ant colony based workload placement in clouds. In: 12th IEEE/ACM international conference on grid computing (GRID), Lyon, France, pp 26–33

  32. Demeulemeester EL, Herroelen WS (2006) Project Scheduling: A Research Handbook. Springer, Germany

    Google Scholar 

  33. Topcuoglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  34. Khojasteh Toussi G, Naghibzadeh M (2021) A divide and conquer approach to deadline constrained cost-optimization workflow scheduling for the cloud. Clust Comput 24(3):1711–1733

    Article  Google Scholar 

  35. Chen W, Deelman E (2012) Workflowsim: A toolkit for simulating scientific workflows in distributed environments. In: 8th IEEE international conference on E-science, Chicago, IL, USA, pp 1–8

  36. Da Silva RF, Chen W, Juve G, Vahi K, Deelman E (2014) Community resources for enabling research in distributed scientific workflows. In: 10th IEEE international conference on e-Science, Sao Paulo, Brazil, pp 177–184

  37. Forestiero A, Mastroianni C, Meo M, Papuzzo G, Sheikhalishahi M (2016) Hierarchical approach for efficient workload management in geo-distributed data centers. IEEE Trans Green Commun Netw 1(1):97–111

    Article  Google Scholar 

  38. Bartz-Beielstein T, Chiarandini M, Paquete L, Preuss M (2013) Experimental methods for the analysis of optimization algorithms. Springer, Germany

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program (No. 2022YFF0902800), Natural Science Foundation of Jiangsu (No.BK20220803), and the National Natural Science Foundation of China (No.62302095).

Author information

Authors and Affiliations

  1. The School of Computer Science and Engineering, Southeast University, Nanjing, 211189, Jiangsu, China

    Shuang Wang, Yibing Duan, Yamin Lei & Peng Du

  2. China Mobile Research Institute, Beijing, 100053, China

    Yamin Wang

Authors
  1. Shuang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yibing Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yamin Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yamin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SW: Conceptualization, Investigation, Data curation, Validation, Writing-review & Supervision. YD: Conceptualization, Methodology, Writing-original draft. YL: Conceptualization, Writing-review & editing. PD: Writing-original draft. YW: Writing-review & editing.

Corresponding author

Correspondence to Yamin Lei.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Duan, Y., Lei, Y. et al. Electricity-cost-aware multi-workflow scheduling in heterogeneous cloud. Computing (2024). https://doi.org/10.1007/s00607-024-01264-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00607-024-01264-3

Keywords

Mathematics Subject Classification

Search

Navigation