Skip to main content

Advertisement

SpringerLink
Log in

Tri-objective Optimization for Large-Scale Workflow Scheduling and Execution in Clouds

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Cloud computing has become the most popular distributed paradigm with massive computing resources and a large data storage capacity to run large-scale scientific workflow applications without the need to own any infrastructure. Scheduling workflows in a distributed system is a well-known NP-complete problem, which has become even more challenging with a dynamic and heterogeneous pool of resources in a cloud computing platform. The aim of this work is to design efficient and effective scheduling algorithms for multi-objective optimization of large-scale scientific workflows in cloud environments. We propose two novel genetic algorithm (GA)-based scheduling algorithms to assign workflow tasks to different cloud resources in order to simultaneously optimize makespan, monetary cost, and energy consumption. One is multi-objective optimization for makespan, cost and energy (MOMCE), which combines the strengths of two widely adopted solutions, genetic algorithm and particle swarm optimization, for multi-objective optimization problems. The other is pareto dominance for makespan, cost and energy (PDMCE), which is based on genetic algorithm and non-dominated solutions to achieve a better convergence and a uniform distribution of the approximate Pareto front. The proposed solutions are evaluated by an extensive set of different workflow applications and cloud environments, and compared with other existing methods in the literature to show the performance stability and superiority. We also conduct performance evaluation and comparison between MOMCE and PDMCE for different criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Algorithm 3
Fig. 4
Fig. 5
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Data Availibility

Data available upon request.

References

  1. Deelman, E., Singh, G., Su, M., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G., Good, J., Laity, A., Jacob, J., Katz, D.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)

    Google Scholar 

  2. Fahringer, T., Prodan, R., Duan, R., Nerieri, F., Podlipnig, S., Qin, J., Siddiqui, M., Truong, H.L., Villazon, A., Wieczorek, M.: \(\text{ASKALON}\): a grid application development and computing environment. In: The 6th IEEE/ACM International Workshop on Grid Computing, pp. 122–131 (2005)

  3. Li, Z., Ge, J., Hu, H., Song, W., Luo, B.: Cost and energy aware scheduling algorithm for scientific workflows with deadline constraint in clouds. IEEE Tran. Serv. Comput. 11, 713–726 (2015)

    Article  Google Scholar 

  4. Annie, S., Yu, H., Jin, S., Lin, K.C.: An incremental genetic algorithm approach to multiprocessor scheduling. IEEE Trans. Parallel Distrib. Syst. 15(9), 824–834 (2004)

    Article  Google Scholar 

  5. Kwok, Y., Ahmad, I.: Dynamic critical-path scheduling: An effective technique for allocating task graph to multiprocessors. IEEE Trans. Parallel Distrib. Syst. 7(5), 506–521 (1996)

    Article  Google Scholar 

  6. Pirozmand, P., Hosseinabadi, A., Farrokhzad, M., Sadeghilalimi, M., Mirkamali, S., Slowik, A.: Multi-objective hybrid genetic algorithm for task scheduling problem in cloud computing. Neural Comput. Appl. 2021, 1 (2021)

    Google Scholar 

  7. Mohammadzadeh, A., Masdari, M., Gharehchopogh, F.S.: Energy and cost-aware workflow scheduling in cloud computing data centers using a multiobjective optimization algorithm. J. Netw. Syst. Manag. 29, 1–10 (2021)

    Article  Google Scholar 

  8. Murad, S., Badeel, R., Alsandi, N., Faraj, R., Ahmed, R., Muhammed, A., Derahman, M., Salih, N.: Optimized min-min task scheduling algorithm for scientific workflows in a cloud environment. J. Theor. Appl. Info. Technol. 2022, 480–506 (2022)

    Google Scholar 

  9. Farid, M., Latip, R., Hussin, M., Hamid, N.: Weighted-adaptive inertia strategy for multi-objective scheduling in multi-clouds. Comput. Mater. Contin. 72, 1529–1560 (2022)

    Google Scholar 

  10. Behera, I., Sobhanayak, S.: Task scheduling optimization in heterogeneous cloud computing environments: a hybrid ga-gwo approach. J. Parallel Distrib. Comput. (2024). https://doi.org/10.1016/j.jpdc.2023.104766

    Article  Google Scholar 

  11. Liu, B., Li, J., Lin, W., Bai, W., Li, P., Gao, Q.: \(\text{ K-PSO }\): An improved \(\text{ PSO }\)-based container scheduling algorithm for big data applications. Int. J. Netw. Manag. 31, e2092 (2020)

    Article  Google Scholar 

  12. Arunagiri, R., Kandasamy, V.: Workflow scheduling in cloud environment using a novel metaheuristic optimization algorithm. Int. J. Commun. Syst. 34, 4746 (2021)

    Article  Google Scholar 

  13. Ma, X., Xu, H., Gao, H.: Real-time multiple-workflow scheduling in cloud environments. IEEE Trans. Netw. Serv. Manag. 18, 4002 (2021)

    Article  Google Scholar 

  14. Singh, S.: Performance optimization in gang scheduling in cloud computing. IOSR J. Comput. Eng. (IOSRJCE) 2, 49–52 (2012)

    Google Scholar 

  15. Sharma, N., Tyagi, S.: A survey on heuristic approach for task scheduling in cloud computing. Int. J. Adv. Res. Comput. Sci. 8(3), 1–4 (2017)

    Google Scholar 

  16. Gu, Y., Wu, Q., Rao, N.S.V.: Analyzing execution dynamics of scientific workflows for latency minimization in resource sharing environments. In: Proceedings of the 7th IEEE World Congress on Services, Washington DC, pp. 153–160 (2011)

  17. Topcuouglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002). https://doi.org/10.1109/71.993206

    Article  Google Scholar 

  18. Rodriguez, M., Buyya, R.: A responsive knapsack-based algorithm for resource provisioning and scheduling of scientific workflows in clouds. In: the 44th International Conference on Parallel Processing (ICPP), pp. 839–848 (2015)

  19. Verma, A., Kaushal, S.: Cost-time efficient scheduling plan for executing workflows in the cloud. J. Grid Comput. 13(4), 495–506 (2015)

    Article  Google Scholar 

  20. Konjaang, J., Xu, L.: Multi-objective workflow optimization strategy (MOWOS) for cloud computing. J. Cloud Comput. 2021, 1–19 (2021)

    Google Scholar 

  21. Tang, Z., Qi, L., Cheng, Z., Li, K., Khan, S., Li, K.: An energy-efficient task scheduling algorithm in dvfs-enabled cloud environment. J. Grid Comput. 14(1), 55–74 (2016). https://doi.org/10.1007/s10723-015-9334-y

    Article  Google Scholar 

  22. Chen, H., Zhu, X., Qiu, D., Guo, H., Yang, L., Lu, P.: EONS: minimizing energy consumption for executing real-time workflows in virtualized cloud data centers. In: 45th International Conference on Parallel Processing Workshops, ICPP, pp. 385–392 (2016)

  23. Boeres, C., Filho, J., Rebello, V.: A cluster-based strategy for scheduling task on heterogeneous processors. In: Proceedings of 16th Symposium on Computer Architecture and High Performance Computing, pp. 214–221 (2004)

  24. Maurya, A.: Resource and task clustering based scheduling algorithm for workflow applications in cloud computing environment. In: International Conference on Parallel, Distributed and Grid Computing, pp. 566–570 (2020)

  25. Bajaj, R., Agrawal, D.: Improving scheduling of tasks in a heterogeneous environment. IEEE Trans. Parallel Distrib. Syst. 15(2), 107–118 (2004)

    Article  Google Scholar 

  26. Lin, X., Wu, C.Q.: On scientific workflow scheduling in clouds under budget constraint. In: Proceedings of the 42nd International Conference on Para. Proc., pp. 90–99 (2013)

  27. Bozdag, D., Catalyurek, U., Ozguner, F.: A task duplication based bottom-up scheduling algorithm for heterogeneous environments. In: Proceedings of the 20th International Conference on Parallel and Distributed Processing (2006)

  28. Durillo, J., Nae, V., Prodan, R.: Multi-objective energy-efficient workflow scheduling using list-based heuristics. Futur. Gener. Comput. Syst. 36, 221–236 (2014). https://doi.org/10.1016/j.future.2013.07.005

    Article  Google Scholar 

  29. Abualigah, L., Diabat, A.: A novel hybrid antlion optimization algorithm for multi-objective task scheduling problems in cloud computing environments. Clust. Comput. 2020, 205–223 (2020)

    Google Scholar 

  30. Kumar, D., Sahoo, B., Mondal, B., Mandal, T.: A genetic algorithmic approach for energy efficient task consolidation in cloud computing. Int. J. Comput. Appl. 118(2), 1–6 (2015). https://doi.org/10.5120/20714-3066

    Article  Google Scholar 

  31. Leena, V.A., Beegom, A.S., Rajasree, M.S.: Genetic algorithm based bi-objective task scheduling in hybrid cloud platform. Int. J. Comput. Theo. Eng. 8(1), 10 (2016)

    Google Scholar 

  32. Zhang, L., Li, K., Li, C., Li, K.: Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems. Inf. Sci. 379, 241–256 (2016)

    Article  Google Scholar 

  33. Meena, J., Kumar, M., Vardhan, M.: Cost effective genetic algorithm for workflow scheduling in cloud under deadline constraint. IEEE Access 4, 5065–5082 (2016)

    Article  Google Scholar 

  34. Zhu, Z., Zhang, G., Li, M., Liu, X.: Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans. Parallel Distrib. Syst. 27(5), 1344–1357 (2016)

    Article  Google Scholar 

  35. Rehman, A., Hussain, S., Rehman, Z., Zia, S.: Multi-objective approach of energy efficient workflow scheduling in cloud environments. Concurr. Comput. Pract. Exp. 34, e4949 (2018)

    Google Scholar 

  36. Nagar, R., Gupta, D., Singh, R.: Time effective workflow scheduling using genetic algorithm in cloud computing. Int. J. Info. Technol. Comput. Sci. 10, 68–75 (2018)

    Google Scholar 

  37. Ruan, F., Gu, R., Huang, T., Xue, S.: A big data placement method using nsga-iii in meteorological cloud platform. EURASIP J. Wireless Commun. Netw. 2019, 143 (2019)

    Article  Google Scholar 

  38. Talukder, A., Kirley, M., Buyya, R.: Multiobjective differential evolution for workflow execution on grids. Concurr. Comput.: Pract. Exp. 21(13), 1742–1756 (2009)

    Article  Google Scholar 

  39. Yassa, S., Chelouah, R., Kadima, H., Granado, B.: Multi-objective approach for energy-aware workflow scheduling in cloud computing environments. Sci. World J. 2013, 350934 (2013)

    Article  Google Scholar 

  40. Rodriguez, M., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2, 222–235 (2014)

    Article  Google Scholar 

  41. Garg, R., Singh, A.: Multi-objective workflow grid scheduling using \(\epsilon\)-fuzzy dominance sort based discrete particle swarm optimization. J. Supercomput. 68(2), 709–732 (2014)

    Article  Google Scholar 

  42. Verma, A., Kaushal, S.: A hybrid multi-objective particle swarm optimization for scientific workflow scheduling. Parallel Comput. 62, 1–19 (2017)

    Article  MathSciNet  Google Scholar 

  43. Beegom, A., Rajasree, M.: Integer-pso: a discrete PSO algorithm for task scheduling in cloud computing systems. Evol. Intell. 2019, 227–239 (2019)

    Article  Google Scholar 

  44. Storn, R., Price, K.: Differential evolution: a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)

    Article  MathSciNet  Google Scholar 

  45. Cheng, M., Prayogo, D.: Symbiotic organisms search: a new metaheuristic optimization algorithm. Comput. Struct. 139, 98–112 (2014)

    Article  Google Scholar 

  46. Anwar, N., Deng, H.: A hybrid metaheuristic for multi-objective scientific workflow scheduling in a cloud environment. Appl. Sci. 8(4), 13 (2018)

    Article  Google Scholar 

  47. Arabnejad, H., Barbosa, J.: List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans. Parallel Distrib. Syst. 25, 628–694 (2014)

    Article  Google Scholar 

  48. Durillo, J., Prodan, R.: Multi-objective workflow scheduling in amazon ec2. Clust. Comput. 17(2), 169–189 (2014)

    Article  Google Scholar 

  49. Fard, H.M., Prodan, R., Barrionuevo, J.J.D., Fahringer, T.: A multi-objective approach for workflow scheduling in heterogeneous environments. In: 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 300–309 (2012). https://doi.org/10.1109/CCGrid.2012.114

  50. Zhou, X., Zhang, G., Sun, J., Zhou, J., Wei, T., Hu, S.: Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based heft. FGCS 93, 278–289 (2019)

    Article  Google Scholar 

  51. Manasrah, A., Ali, H.: Workflow scheduling using hybrid ga-pso algorithm in cloud computing. Wirel. Commun. Mobile Comput. 2018(1), 1934784 (2018)

    Article  Google Scholar 

  52. Ghasemzadeh, M., Arabnejad, H., Barbosa, J.: Deadline-budget constrained scheduling algorithm for scientific workflows in a cloud environment. In: 20th International Conference on Principles of Distributed System (OPODIS), pp. 1–16 (2017)

  53. Alrammah, H., Gu, Y., Wu, C., Ju, S.: Scheduling for energy efficiency and throughput maximization in a faulty cloud environment. In: The International Conference on Parallel and Distributed Systems, pp. 561–569 (2017)

  54. Mao, M., Humphrey, M.: A performance study on the VM startup time in the cloud. In: 2012 IEEE 5th International Conference on Cloud Computing, pp. 423–430 (2012)

  55. Kliazovich, D., Bouvry, P., Khan, S.: DENS: data center energy-efficient network-aware scheduling. Clust. Comput. 16(1), 65–75 (2013)

    Article  Google Scholar 

  56. Chen, Y., Das, A., Qin, W., Sivasubramaniam, A., Wang, Q., Gautam, N.: Managing server energy and operational costs in hosting centers. Proceedings of the 2005 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pp. 303–314 (2005)

  57. Cao, F., Zhu, M., Wu, Q.: Energy-efficient resource management for scientific workflows in clouds. In: 2014 IEEE World Congress on Services, pp. 402–409 (2014) https://doi.org/10.1109/SERVICES.2014.76

  58. Marler, R.T., Arora, J.S.: Survey of multi-objective optimization methods for engineering. Struct. Multidisc. Optim. 26, 369–395 (2004). https://doi.org/10.1007/s00158-003-0368-6

    Article  MathSciNet  Google Scholar 

  59. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evol. Comput. 6, 182–197 (2002)

    Article  Google Scholar 

  60. Wu, Q., Gu, Y.: Supporting distributed application workflows in heterogeneous computing environments. In: Proceedings of the 14th IEEE International Conference on Parallel and Distributed Systems, Melbourne, Australia, pp. 3–10 (2008)

  61. Wu, Q., Gu, Y., Zhu, M., Rao, N.S.V.: Optimizing network performance of computing pipelines in distributed environments. In: Proceedings of the 22nd IEEE International Parallel and Distributed Processing Symposium Miami, Florida (2008)

  62. Alrammah, H., Gu, Y., Wu, C., Ju, S.: Scheduling for energy efficiency and throughput maximization in a faulty cloud environment. In: The International Conference on Parallel and Distributed Systems, pp. 561–569 (2017)

  63. Deb, K., Jain, S.: Running performance metrics for evolutionary multi-objective optimization. In: Proceedings of Simulated Evolution and, Learning, pp. 13–20 (2002)

  64. Abido, M.: Environmental/economic power dispatch using multiobjective evolutionary algorithms. IEEE Trans. Power Syst. 18(4), 1529–1537 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Middle Tennessee State University, Murfreesboro, TN, 37132, USA

    Huda Alrammah & Yi Gu

  2. Computer and Information Sciences, Harrisburg University of Science and Technology, Harrisburg, PA, 17101, USA

    Daqing Yun

  3. Department of Mathematics and Computer Science, Fisk University, Nashville, TN, 37208, USA

    Ning Zhang

Authors
  1. Huda Alrammah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daqing Yun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YG provided the research topic, guided the direction, discussed the main idea. HA did the experiments and collected the data. HA and YG wrote the main manuscript. DY and NZ participated in the discussions and provided insights into the solution finding and constructive comments on the paper writing and proofreading.

Corresponding author

Correspondence to Yi Gu.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alrammah, H., Gu, Y., Yun, D. et al. Tri-objective Optimization for Large-Scale Workflow Scheduling and Execution in Clouds. J Netw Syst Manage 32, 89 (2024). https://doi.org/10.1007/s10922-024-09863-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-024-09863-3

Keywords

Search

Navigation