Skip to main content
SpringerLink
Log in

QoS-aware online scheduling of multiple workflows under task execution time uncertainty in clouds

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud computing, with elasticity and pay-as-you-go pricing, is a suitable platform for executing workflow applications. Workflow as a Service (WaaS) systems provide scientists with an easy-to-use, and cost-effective platform to execute their workflow applications in the cloud at any time or location worldwide. Quality of Service (QoS) is recognized as a key requirement in WaaS. Monetary cost and time are two primary QoS from a clients’ perspective; whereas, energy consumption is considered a significant problem for cloud providers’ profitability and ability to provide low-cost services. Most workflow scheduling studies assume that workflow tasks have a deterministic Execution Time (ET), which is generally unrealistic in the real world. However, there are few approaches for scheduling in WaaS considering deadlines, and monetary costs with uncertain task ET. These studies typically assume that a cloud resource can execute all types of workflow applications without any need for additional software components. However, using containers is a suitable solution to provide an executable environment for the execution of any workflow type on cloud resources. To this end, we present two cost and energy-aware workflow scheduling that consider the uncertainty in tasks’ ETs. Both solutions are designed for WaaS, leveraging containers to enhance resource utilization rate and reduce energy consumption, resource monetary cost, and workflows deadline violations. Simulated experiments demonstrate that our proposed methods outperform two recent state-of-the-art scheduling algorithms in terms of success rate, monetary cost, energy consumption, and resource utilization rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Workflows data used in this paper is from Pegasus workflow generator [21, 22] and is commonly used in workflow scheduling algorithms [2, 3].

Notes

  1. https://github.com/taghinejad/MultiWorkflowScheduler.

References

  1. Stavrinides, G.L., Karatza, H.D.: An energy-efficient, QoS-aware and cost-effective scheduling approach for real-time workflow applications in cloud computing systems utilizing DVFS and approximate computations. Future Gener. Comput. Syst. 96, 216–226 (2019). https://doi.org/10.1016/j.future.2019.02.019

    Article  Google Scholar 

  2. Liu, J., Ren, J., Dai, W., Zhang, D., Zhou, P., Zhang, Y., Min, G., Najjari, N.: Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans. Cloud Comput. (2019). https://doi.org/10.1109/TCC.2019.2906300

    Article  Google Scholar 

  3. Garg, N., Singh, D., Goraya, M.S.: Energy and resource efficient workflow scheduling in a virtualized cloud environment. Clust. Comput. 4, 1–31 (2020). https://doi.org/10.1007/s10586-020-03149-4

    Article  Google Scholar 

  4. Xie, G., Zeng, G., Li, R., Li, K.: Scheduling Parallel Applications on Heterogeneous Distributed Systems. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-6557-7

  5. Safari, M., Khorsand, R.: Energy-aware scheduling algorithm for time-constrained workflow tasks in DVFS-enabled cloud environment. Simul. Model. Pract. Theory 87(July), 311–326 (2018). https://doi.org/10.1016/j.simpat.2018.07.006

    Article  Google Scholar 

  6. Zheng, W., Qin, Y., Bugingo, E., Zhang, D., Chen, J.: Cost optimization for deadline-aware scheduling of big-data processing jobs on clouds. Future Gener. Comput. Syst. 82, 244–255 (2018). https://doi.org/10.1016/j.future.2017.12.004

    Article  Google Scholar 

  7. Rodriguez, M.A., Buyya, R.: Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Future Gener. Comput. Syst. 79, 739–750 (2018). https://doi.org/10.1016/j.future.2017.05.009

    Article  Google Scholar 

  8. Gerlach, W., Tang, W., Keegan, K., Harrison, T., Wilke, A., Bischof, J., D’Souza, M., Devoid, S., Murphy-Olson, D., Desai, N., Meyer, F.: Skyport—container-based execution environment management for multi-cloud scientific workflows. In: 2014 5th International Workshop on Data-Intensive Computing in the Clouds, pp. 25–32 (2014). https://doi.org/10.1109/DataCloud.2014.6

  9. Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Future Gener. Comput. Syst. 100, 98–108 (2019). https://doi.org/10.1016/j.future.2019.04.029

    Article  Google Scholar 

  10. Cai, Z., Li, X., Ruiz, R., Li, Q.: A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds. Future Gener. Comput. Syst. 71, 57–72 (2017). https://doi.org/10.1016/j.future.2017.01.020

    Article  Google Scholar 

  11. Chen, H., Zhu, X., Liu, G., Pedrycz, W.: Uncertainty-aware online scheduling for real-time workflows in cloud service environment. IEEE Trans. Serv. Comput. (2018). https://doi.org/10.1109/TSC.2018.2866421

    Article  Google Scholar 

  12. Iranmanesh, A., Naji, H.R.: DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing. Clust. Comput. 24(2), 667–681 (2021)

    Article  Google Scholar 

  13. Taghinezhad-Niar, A., Pashazadeh, S., Taheri, J.: Workflow scheduling of scientific workflows under simultaneous deadline and budget constraints. Clust. Comput. (2021). https://doi.org/10.1007/s10586-021-03314-3

    Article  Google Scholar 

  14. Zhu, Z., Tang, X.: Deadline-constrained workflow scheduling in IaaS clouds with multi-resource packing. Future Gener. Comput. Syst. 101(December), 880–893 (2019). https://doi.org/10.1016/j.future.2019.07.043

    Article  Google Scholar 

  15. Sahni, J., Vidyarthi, P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2018). https://doi.org/10.1109/TCC.2015.2451649

    Article  Google Scholar 

  16. Filgueira, R., Da Silva, R.F., Krause, A., Deelman, E., Atkinson, M.: Asterism: Pegasus and Dispel4py hybrid workflows for data-intensive science. In: 2016 Seventh International Workshop on Data-Intensive Computing in the Clouds (DataCloud), pp. 1–8 (2016). https://doi.org/10.1109/DataCloud.2016.004

  17. Di Tommaso, P., Palumbo, E., Chatzou, M., Prieto, P., Heuer, M.L., Notredame, C.: The impact of Docker containers on the performance of genomic pipelines. PeerJ 3, e1273 (2015)

    Article  Google Scholar 

  18. Sun, T., Xiao, C., Xu, X.: A scheduling algorithm using sub-deadline for workflow applications under budget and deadline constrained. Clust. Comput. 22(3), 5987–5996 (2019). https://doi.org/10.1007/s10586-018-1751-9

    Article  Google Scholar 

  19. Singh, V., Gupta, I., Jana, P.K.: An energy efficient algorithm for workflow scheduling in IaaS cloud. J. Grid Comput. (2019). https://doi.org/10.1007/s10723-019-09490-2

    Article  Google Scholar 

  20. Deelman, E., Singh, G., Su, M.H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., et al.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)

  21. Chen, W., Rey, M., Rey, M.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: The 8th IEEE International Conference on eScience (eScience 2012), pp. 1–8 (2012). https://doi.org/10.1109/eScience.2012.6404430

  22. Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29, 682–692 (2013). https://doi.org/10.1016/j.future.2012.08.015

    Article  Google Scholar 

  23. Taghinezhad-Niar, A., Pashazadeh, S., Taheri, J.: Energy-efficient workflow scheduling with budget-deadline constraints for cloud. Computing (2022). https://doi.org/10.1007/s00607-021-01030-9

Download references

Funding

There has been no significant financial support for this work.

Author information

Authors and Affiliations

  1. Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

    Ahmad Taghinezhad-Niar & Saeid Pashazadeh

  2. Department of Mathematics and Computer Science, Karlstad University, Karlstad, Sweden

    Javid Taheri

Authors
  1. Ahmad Taghinezhad-Niar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeid Pashazadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javid Taheri
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

Corresponding author

Correspondence to Saeid Pashazadeh.

Ethics declarations

Conflict of interest

There are no conflicts of interest associated with this publication.

Research involving human and/or animal participants

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taghinezhad-Niar, A., Pashazadeh, S. & Taheri, J. QoS-aware online scheduling of multiple workflows under task execution time uncertainty in clouds. Cluster Comput 25, 3767–3784 (2022). https://doi.org/10.1007/s10586-022-03600-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-022-03600-8

Keywords

Search

Navigation