Skip to main content
SpringerLink
Log in

Cost-efficient Workflow as a Service using Containers

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Workflows are special applications used to solve complex scientific problems. The emerging Workflow as a Service (WaaS) model provides scientists with an effective way of deploying their workflow applications in Cloud environments. The WaaS model can execute multiple workflows in a multi-tenant Cloud environment. Scheduling the tasks of the workflows in the WaaS model has several challenges. The scheduling approach must properly utilize the underlying Cloud resources and satisfy the users’ Quality of Service (QoS) requirements for all the workflows. In this work, we have proposed a heurisine-sensitive workflows in a containerized Cloud environment for the WaaS model. We formulated the problem of minimizing the MIPS (million instructions per second) requirement of tasks while satisfying the deadline of the workflows as a non-linear optimization problem and applied the Lagranges multiplier method to solve it. It allows us to configure/scale the containers’ resources and reduce costs. We also ensure maximum utilization of VM’s resources while allocating containers to VMs. Furthermore, we have proposed an approach to effectively scale containers and VMs to improve the schedulability of the workflows at runtime to deal with the dynamic arrival of the workflows. Extensive experiments and comparisons with other state-of-the-art works show that the proposed approach can significantly improve resource utilization, prevent deadline violation, and reduce the cost of renting Cloud resources for the WaaS model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhao, Y., Li, Y., Raicu, I., Lu, S., Lin, C., Zhang, Y., Tian, W., Xue, R.: A service framework for scientific workflow management in the Cloud. IEEE Trans. Serv. Comput. 8(6), 930–944 (2014)

  2. Zhao, Y., Li, Y., Raicu, I., Lu, S., Tian, W., Liu, H.: Enabling scalable scientific workflow management in the cloud. Futur. Gener. Comput. Syst. 46, 3–16 (2015)

  3. Ahmad, Z., Nazir, B., Umer, A.: A fault-tolerant workflow management system with quality-of-service-aware scheduling for scientific workflows in cloud computing. Int. J. Commun. Syst. 34(1), 4649 (2021)

  4. Hilman, M.H., Rodriguez, M.A., Buyya, R.: Resource-sharing policy in multi-tenant scientific workflow as a service platform (2019). Available from https://doi.org/10.48550/arXiv.1903.01113

  5. Stavrinides, G.L., Karatza, H.D.: An energy-efficient, Qos-aware and cost-effective scheduling approach for real-time workflow applications in cloud computing systems utilizing DVFS and approximate computations. Futur. Gener. Comput. Syst. 96, 216–226 (2019)

  6. Qin, Y., Wang, H., Yi, S., Li, X., Zhai, L.: An energy-aware scheduling algorithm for budget-constrained scientific workflows based on multi-objective reinforcement learning. J. Supercomput. 76(1), 455–480 (2020)

  7. Tarafdar, A., Karmakar, K., Khatua, S., Das, R.K.: Energy-efficient scheduling of deadline-sensitive and budget-constrained workflows in the cloud. In: International conference on distributed computing and internet technology. Springer, pp. 65–80 (2021)

  8. Rodriguez, M.A., Buyya, R.: Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Futur. Gener. Comput. Syst. 79, 739–750 (2018)

  9. Taibi, D., Lenarduzzi, V., Pahl, C.: Architectural patterns for microservices: a systematic mapping study. In: CLOSER 2018: Proceedings of the 8th international conference on cloud computing and services science. SciTePress, Funchal (2018)

  10. Wang, J., Korambath, P., Altintas, I., Davis, J., Crawl, D.: Workflow as a service in the cloud: architecture and scheduling algorithms. Procedia Comput. Sci. 29, 546–556 (2014)

  11. Esteves, S., Veiga, L.: Waas: workflow-as-a-service for the cloud with scheduling of continuous and data-intensive workflows. Comput. J. 59(3), 371–383 (2016)

  12. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.-H., Vahi, K.: Characterization of scientific workflows. In: 2008 third workshop on workflows in support of large-scale science. IEEE, pp. 1–10 (2008)

  13. Rizvi, N., Ramesh, D.: Fair budget constrained workflow scheduling approach for heterogeneous clouds. Clust. Comput. 23(4), 3185–3201 (2020)

  14. Karmakar, K., Das, R.K., Khatua, S.: Resource scheduling of workflow tasks in cloud environment. In: 2019 IEEE international conference on advanced networks and telecommunications systems (ANTS). IEEE, pp. 1–6 (2019)

  15. Karmakar, K., Das, R.K., Khatua, S.: Resource scheduling for tasks of a workflow in cloud environment. In: International conference on distributed computing and internet technology. Springer, pp. 214–226 (2020)

  16. Tarafdar, A., Karmakar, K., Das, R.K., Khatua, S.: Multi-criteria scheduling of scientific workflows in the workflow as a service platform. Comput. Electr. Eng. 105, 108458 (2023)

  17. Qasha, R., Cala, J., Watson, P.: Dynamic deployment of scientific workflows in the cloud using container virtualization. In: 2016 IEEE international conference on cloud computing technology and science (CloudCom). IEEE, pp. 269–276 (2016)

  18. Liu, K., Aida, K., Yokoyama, S., Masatani, Y.: Flexible container-based computing platform on cloud for scientific workflows. In: 2016 international conference on cloud computing research and innovations (ICCCRI). IEEE, pp. 56–63 (2016)

  19. Chen, H., Zhu, X., Liu, G., Pedrycz, W.: Uncertainty-aware online scheduling for real-time workflows in cloud service environment. IEEE Trans. Serv. Comput. 14(4), 1167–1178 (2018)

    Article  Google Scholar 

  20. B Burkat, K., Pawlik, M., Balis, B., Malawski, M., Vahi, K., Rynge, M., da Silva, R.F., Deelman, E.: Serverless Containers–rising viable approach to Scientific Workflows. In: 17th International Conference on eScience (eScience). IEEE, pp. 40-49 (2021)

  21. Ranjan, R., Thakur, I.S., Aujla, G.S., Kumar, N., Zomaya, A.Y.: Energy-efficient workflow scheduling using container-based virtualization in software-defined data centers. IEEE Trans. Ind. Inform. 16(12), 7646–7657 (2020)

  22. Bao, L., Wu, C., Bu, X., Ren, N., Shen, M.: Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Trans. Parallel Distrib. Syst. 30(9), 2114–2129 (2019)

  23. Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., Merle, P.: Autonomic vertical elasticity of docker containers with Elasticdocker. In: 2017 IEEE 10th international conference on cloud computing (CLOUD). IEEE, pp. 472–479 (2017)

  24. Paraiso, F., Challita, S., Al-Dhuraibi, Y., Merle, P.: Model-driven management of docker containers. In: 2016 IEEE 9th international conference on cloud computing (CLOUD). IEEE, pp. 718–725 (2016)

  25. Wu, Q., Datla, V.V.: On performance modeling and prediction in support of scientific workflow optimization. In: 2011 IEEE world congress on services. IEEE, pp. 161–168 (2011)

  26. Sadjadi, S.M., Shimizu, S., Figueroa, J., Rangaswami, R., Delgado, J., Duran, H., Collazo-Mojica, X.J.: A modeling approach for estimating execution time of long-running scientific applications. In: 2008 IEEE international symposium on parallel and distributed processing. IEEE, pp. 1–8 (2008)

  27. Saeedizade, E., Ashtiani, M.: Ddbws: A dynamic deadline and budget-aware workflow scheduling algorithm in workflow-as-a-service environments. J. Supercomput. 77(12), 14525–14564 (2021)

  28. Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Futur. Gener. Comput. Syst. 100, 98–108 (2019)

  29. Silva, R.F., Pottier, L., Coleman, T., Deelman, E., Casanova, H.: Workflowhub: community framework for enabling scientific workflow research and development. In: 2020 IEEE/ACM workflows in support of large-scale science (WORKS). IEEE, pp. 49–56 (2020)

  30. Piraghaj, S.F., Dastjerdi, A.V., Calheiros, R.N., Buyya, R.: Containercloudsim: an environment for modeling and simulation of containers in cloud data centers. Softw. Pract. Experience 47(4), 505–521 (2017)

  31. Mao, M., Humphrey, M.: A performance study on the VM startup time in the cloud. In: 2012 IEEE fifth international conference on cloud computing. IEEE, pp. 423–430 (2012)

  32. Da Silva, R.F., Chen, W., Juve, G., Vahi, K., Deelman, E.: Community resources for enabling research in distributed scientific workflows. In: 2014 IEEE 10th international conference on e-science, vol. 1. IEEE, pp. 177–184 (2014)

  33. Silva, R.F.d., Chen, W., Juve, G., Vahi, K., Deelman, E.: Community resources for enabling research in distributed scientific workflows. In: Proceedings of the 2014 IEEE 10th international conference on e-science-volume 01, pp. 177–184 (2014)

Download references

Funding

No funding was obtained for this study.

Author information

Author notes

  1. Anurina Tarafdar, Rajib K. Das and Sunirmal Khatua contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Calcutta, JD-2, JD Block, Sector 3, Bidhannagar, Kolkata, 700106, W.B., India

    Kamalesh Karmakar, Rajib K. Das & Sunirmal Khatua

  2. Department of Computer Science and Engineering, Heritage Institute of Technology, Chowbaga Rd, Anandapur, Mundapara, Kolkata, 700107, W.B., India

    Anurina Tarafdar

Authors
  1. Kamalesh Karmakar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anurina Tarafdar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajib K. Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sunirmal Khatua
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sunirmal Khatua.

Ethics declarations

Competing interests

I declare that the authors have no competing interests as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karmakar, K., Tarafdar, A., Das, R.K. et al. Cost-efficient Workflow as a Service using Containers. J Grid Computing 22, 40 (2024). https://doi.org/10.1007/s10723-024-09745-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09745-7

Keywords

Search

Navigation