Skip to main content
SpringerLink
Log in

An efficient resource provisioning algorithm for workflow execution in cloud platform

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud Computing provides a promising platform for executing large scale workflow applications with enormous computational resources to offer on-demand services. Tasks in a workflow may need different type of computing resources such as storage, compute and memory type. However, inappropriate selection of these resources may lead to higher makespan and resource wastage. In this paper, we propose an effective two-phase algorithm for provisioning of cloud resources for workflow applications by using its structural features to minimize makespan and resource wastage. The proposed approach considers the nature of the tasks which may be compute intensive, memory intensive or storage intensive. We assume a realistic cloud model similar to Amazon EC2 that provides virtual machines for different types of workloads. Most importantly, the workflow model used in our approach is assumed to contain limited information about the task which is applicable for real situation. The performance of the proposed work is measured using five benchmark scientific workflows. The simulation results show that the proposed approach outperforms two existing algorithms for all these workflows.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Ahmad, W., Alam, B., Ahuja, S., Malik, S.: A dynamic vm provisioning and de-provisioning based cost-efficient deadline-aware scheduling algorithm for big data workflow applications in a cloud environment. Cluster Comput. 24, 249–278 (2021). https://doi.org/10.1007/s10586-020-03100-7

    Article  Google Scholar 

  2. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: A view of cloud computing. Commun. ACM 53(4), 50–58 (2010)

    Article  Google Scholar 

  3. AWS (????) Scheduled reserved instances prices

  4. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10 (2008)

  5. Bugingo, E., Zheng, W., Zhang, D., Chen, J.: Dynamic virtual machine number selection for processing-capacity constrained workflow scheduling in cloud computing environments. In: IEEE Intl. Conf. on ISPA/BDCloud/SocialCom/SustainCom, pp. 71–78. https://doi.org/10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00021 (2019)

  6. Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)

    Article  Google Scholar 

  7. Cai, Z., Li, X., Ruiz, R.: Resource provisioning for task-batch based workflows with deadlines in public clouds. IEEE Trans. Cloud Comput. 7(3), 814–826 (2019). https://doi.org/10.1109/TCC.2017.2663426

    Article  Google Scholar 

  8. Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014). https://doi.org/10.1109/TPDS.2013.238

    Article  Google Scholar 

  9. Costache, S., Dib, D., Parlavantzas, N., Morin, C.: Resource management in cloud platform as a service systems: analysis and opportunities. J. Syst. Softw. 132, 98–118 (2017). https://doi.org/10.1016/j.jss.2017.05.035. (https://www.sciencedirect.com/science/article/pii/S0164121217300845)

    Article  Google Scholar 

  10. Faragardi, H.R., Saleh Sedghpour, M.R., Fazliahmadi, S., Fahringer, T., Rasouli, N.: Grp-heft: a budget-constrained resource provisioning scheme for workflow scheduling in iaas clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2020). https://doi.org/10.1109/TPDS.2019.2961098

    Article  Google Scholar 

  11. Garg, N., Singh, D., Goraya, M.: Energy and resource efficient workflow scheduling in a virtualized cloud environment. Cluster Comput. 24, 767–797 (2021). https://doi.org/10.1007/s10586-020-03149-4

    Article  Google Scholar 

  12. Iqbal, W., Dailey, M.N., Carrera, D., Janecek, P.: Adaptive resource provisioning for read intensive multi-tier applications in the cloud. Future Gener. Comput. Syst. 27(6), 871–879 (2011)

    Article  Google Scholar 

  13. Javadi, B., Abawajy, J., Buyya, R.: Failure-aware resource provisioning for hybrid cloud infrastructure. J. Parallel Distrib. Comput. 72(10), 1318–1331 (2012)

    Article  Google Scholar 

  14. Javadi, B., Abawajy, J., Sinnott, R.O.: Hybrid cloud resource provisioning policy in the presence of resource failures. In: IEEE 4th Intl. Conf. on Cloud Computing Technology and Science (CloudCom), pp. 10–17 (2012b)

  15. Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29(3), 682–692 (2013)

    Article  Google Scholar 

  16. Kanagaraj, K., Swamynathan, S.: Structure aware resource estimation for effective scheduling and execution of data intensive workflows in cloud. Future Gener. Comput. Syst. 79, 878–891 (2018). https://doi.org/10.1016/j.future.2017.09.001. (http://www.sciencedirect.com/science/article/pii/S0167739X16308111)

    Article  Google Scholar 

  17. Kecskemeti, G., Nemeth, Z., Kertesz, A., Ranjan, R.: Cloud workload prediction based on workflow execution time discrepancies. Cluster Comput. 22, 737–755 (2019). https://doi.org/10.1007/s10586-018-2849-9

    Article  Google Scholar 

  18. Kim, H., El-Khamra, Y., Rodero, I., Jha, S., Parashar, M.: Autonomic management of application workflows on hybrid computing infrastructure. Sci. Program. 19(2–3), 75–89 (2011)

    Google Scholar 

  19. Kumar, D., Baranwal, G., Raza, Z., Vidyarthi, D.P.: A systematic study of double auction mechanisms in cloud computing. J. Syst. Softw. 125, 234–255 (2017)

    Article  Google Scholar 

  20. Kumar, M.S., Gupta, I., Panda, S.K., Jana, P.K.: Granularity-based workflow scheduling algorithm for cloud computing. J. Supercomput. 73, 5440–5464 (2017). https://doi.org/10.1007/s11227-017-2094-7

    Article  Google Scholar 

  21. Li, C., Li, L.Y.: Optimal resource provisioning for cloud computing environment. J. Supercomput. 62(2), 989–1022 (2012)

    Article  Google Scholar 

  22. Liu, J., Ren, J., Dai, W., Zhang, D., Zhou, P., Zhang, Y., Min, G., Najjari, N.: Online multi-workflow scheduling under uncertain task execution time in iaas clouds. IEEE Trans. Cloud Comput. https://doi.org/10.1109/TCC.2019.2906300 (2019)

  23. Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in iaas clouds. Future Gener. Comput. Syst. 48, 1–18 (2015)

    Article  Google Scholar 

  24. Toosi, N.A., Sinnott, R.O., Buyya, R.: Resource provisioning for data-intensive applications with deadline constraints on hybrid clouds using aneka. Future Gener. Comput. Syst. 79, 765–775 (2018). https://doi.org/10.1016/j.future.2017.05.042

    Article  Google Scholar 

  25. Ndamlabin Mboula, J., Kamla, V., Tayou Djamégni, C.: Dynamic provisioning with structure inspired selection and limitation of vms based cost-time efficient workflow scheduling in the cloud. Clust. Comput. 24, 2697–2721 (2021). https://doi.org/10.1007/s10586-021-03289-1

    Article  Google Scholar 

  26. Nelson, V., Uma, V.: Semantic based resource provisioning and scheduling in inter-cloud environment. In: Intl. Conf. on Recent Trends in Information Technology (ICRTIT), pp. 250–254 (2012)

  27. Niu, S., Zhai, J., Ma, X., Tang, X., Chen, W., Zheng, W.: Building semi-elastic virtual clusters for cost-effective hpc cloud resource provisioning. IEEE Trans. Parallel Distrib. Syst. 27(7), 1915–1928 (2016). https://doi.org/10.1109/TPDS.2015.2476459

    Article  Google Scholar 

  28. Pegasus: Workflow generator. https://github.com/pegasus-isi/WorkflowGenerator. Accessed 24 Dec 2016

  29. Ramakrishnan, L., Gannon, D.: A survey of distributed workflow characteristics and resource requirements. Indiana University pp. 1–23 (2008)

  30. Rimal, B.P., Maier, M.: Workflow scheduling in multi-tenant cloud computing environments. IEEE Trans. Parallel Distrib. Syst. 28(1), 290–304 (2017). https://doi.org/10.1109/TPDS.2016.2556668

    Article  Google Scholar 

  31. Sahni, J., Vidyarthi, D.P.: Workflow-and-platform aware task clustering for scientific workflow execution in cloud environment. Future Gener. Comput. Syst. 64, 61–74 (2016). https://doi.org/10.1016/j.future.2016.05.008

    Article  Google Scholar 

  32. Setlur, A.R., Nirmala, S.J., Singh, H.S., Khoriya, S.: An efficient fault tolerant workflow scheduling approach using replication heuristics and checkpointing in the cloud. J. Parallel Distrib. Comput. 136, 14–28 (2020). https://doi.org/10.1016/j.jpdc.2019.09.004. (http://www.sciencedirect.com/science/article/pii/S0743731518306580)

    Article  Google Scholar 

  33. Shahidinejad, A., Ghobaei-Arani, M., Masdari, M.: Resource provisioning using workload clustering in cloud computing environment: a hybrid approach. Cluster Comput. 24, 319–342 (2021). https://doi.org/10.1007/s10586-020-03107-0

    Article  Google Scholar 

  34. Singh, S., Chana, I.: A survey on resource scheduling in cloud computing: issues and challenges. J. Grid Comput. 14(2), 217–264 (2016)

    Article  Google Scholar 

  35. Vecchiola, C., Calheiros, R.N., Karunamoorthy, D., Buyya, R.: Deadline-driven provisioning of resources for scientific applications in hybrid clouds with aneka. Future Gener. Comput. Syst. 28(1), 58–65 (2012)

    Article  Google Scholar 

  36. Zhang, X., Wu, C., Li, Z., Lau, F.C.M.: A truthful \((1-\epsilon )\)-optimal mechanism for on-demand cloud resource provisioning. IEEE Trans. Cloud Comput. 8(3), 735–748 (2020). https://doi.org/10.1109/TCC.2018.2822718

    Article  Google Scholar 

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (ISM), Dhanbad, Jharkhand, 826004, India

    Madhu Sudan Kumar, Anubhav Choudhary, Indrajeet Gupta & Prasanta K. Jana

  2. School of Computer Science Engineering and Technology, Bennett University, Greater Noida, Uttar Pradesh, 201310, India

    Indrajeet Gupta

Authors
  1. Madhu Sudan Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anubhav Choudhary
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Indrajeet Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Prasanta K. Jana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors contributed equally to this work.

Corresponding author

Correspondence to Madhu Sudan Kumar.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Research involving human and animal rights

This work does not involve the explicit participation of humans and animals.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, M.S., Choudhary, A., Gupta, I. et al. An efficient resource provisioning algorithm for workflow execution in cloud platform. Cluster Comput 25, 4233–4255 (2022). https://doi.org/10.1007/s10586-022-03648-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-022-03648-6

Keywords

Search

Navigation