Skip to main content
SpringerLink
Log in

Cloud workflow scheduling with hybrid resource provisioning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Resource provisioning strategies are crucial for workflow scheduling problems which are widespread in cloud computing. The main challenge lies in determining the amounts of reserved and on-demand resources to meet users’ requirements. In this paper, we consider the cloud workflow scheduling problem with hybrid resource provisioning to minimize the total renting cost, which is NP-hard and has not been studied yet. An iterative population-based meta-heuristic is developed. According to the shift vectors obtained during the search procedure, timetables are computed quickly. The appropriate amounts of reserved and on-demand resources are determined by an incremental optimization method. The utilization of each resource is balanced in a swaying way, in terms of which the probabilistic matrix is updated for the next iteration. The proposed algorithm is compared with modified existing algorithms for similar problems. Experimental results demonstrate effectiveness and efficiency of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abrishami S, Naghibzadeh M, Epema D (2012) Cost-driven scheduling of grid workflows using partial critical paths. IEEE Trans Parallel Distrib Syst 23(8):1400–1414

    Article  Google Scholar 

  2. Abrishami S, Naghibzadeh M, Epema DH (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener Comput Syst 29(1):158–169

    Article  Google Scholar 

  3. AmazonEC2 (2016) Amazon elastic compute cloud (Amazon EC2). http://aws.amazon.com/ec2/pricing

  4. Blythe J, Jain S, Deelman E, Gil Y, Vahi K, Mandal A, Kennedy K (2005). Task scheduling strategies for workflow-based applications in grids. In: IEEE International symposium on cluster computing and the grid (CCGrid 2005), vol 2. IEEE, pp 759–767

  5. Byun EK, Kee YS, Kim JS, Deelman E, Maeng S (2011) BTS: resource capacity estimate for time-targeted science workflows. J Parallel Distrib Comput 71(6):848–862

    Article  Google Scholar 

  6. Byun EK, Kee YS, Kim JS, Maeng S (2011) Cost optimized provisioning of elastic resources for application workflows. Future Gener Comput Syst 27(8):1011–1026

    Article  Google Scholar 

  7. Cai Z, Li X, Gupta JND (2016) Heuristics for provisioning services to workflows in XaaS Clouds. IEEE Trans Serv Comput 9(2):250–263. doi:10.1109/TSC.2014.2361320

    Article  Google Scholar 

  8. Cai Z, Li X, Gupta JND (2013) Critical path-based iterative heuristic for workflow scheduling in utility and cloud computing. In: Service-Oriented Computing. Springer, pp 207–221

  9. Chaisiri S, Lee BS, Niyato D (2012) Optimization of resource provisioning cost in cloud computing. IEEE Trans Serv Comput 5(2):164–177

    Article  Google Scholar 

  10. Chen Q, Wang L, Shang Z (2008) Mrgis: a mapreduce-enabled high performance workflow system for GIS. In: IEEE Fourth International Conference on eScience (eScience 2008). IEEE, pp 646–651

  11. Chen WN, Zhang J (2009) An ant colony optimization approach to a grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern C Appl Rev 39(1):29–43

    Article  Google Scholar 

  12. Deelman E, Singh G, Su MH, Blythe J, Gil Y, Kesselman C, Mehta G, Vahi K, Berriman GB, Good J et al (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program 13(3):219–237

    Google Scholar 

  13. Deelman E, Gannon D, Shields M, Taylor I (2009) Workflows and e-science: an overview of workflow system features and capabilities. Future Gener Comput Syst 25(5):528–540

    Article  Google Scholar 

  14. Demeulemeester E (1995) Minimizing resource availability costs in time-limited project networks. Manag Sci 41:1590–1598

    Article  Google Scholar 

  15. Demeulemeester E, Herroelen WS (2002) Project scheduling: a research handbook, vol 49. Kluwer Academic Publishers, Dordrecht

    MATH  Google Scholar 

  16. Demeulemeester E, Herroelen WS, Elmaghraby SE (1996) Optimal procedures for the discrete time/cost trade-off problem in project networks. Eur J Oper Res 88(1):50–68

    Article  Google Scholar 

  17. Demeulemeester E, De Reyck B, Foubert B, Herroelen WS, Vanhoucke M (1998) New computational results on the discrete time/cost trade-off problem in project networks. J Oper Res Soc 49(11):1153–1163

    Article  Google Scholar 

  18. Hartmann S, Kolisch R (2000) Experimental evaluation of state-of-the-art heuristics for the resource-constrained project scheduling problem. Eur J Oper Res 127(2):394–407

    Article  Google Scholar 

  19. Hazır Ö, Haouari M, Erel E (2010) Discrete time/cost trade-off problem: a decomposition-based solution algorithm for the budget version. Comput Oper Res 37(4):649–655

    Article  Google Scholar 

  20. Juve G, Deelman E, Vahi K, Mehta G, Berriman B, Berman BP, Maechling P (2009) Scientific workflow applications on amazon EC2. In: 2009 5th IEEE International Conference on E-Science Workshops. IEEE, pp. 59–66

  21. Kolisch R, Sprecher A (1997) Psplib-a project scheduling problem library: or software-orsep operations research software exchange program. Eur J Oper Res 96(1):205–216

    Article  Google Scholar 

  22. Mazzucco M, Dumas M (2011) Reserved or on-demand instances? a revenue maximization model for cloud providers. In: IEEE International Conference on Cloud Computing (CLOUD), 2011. IEEE, pp. 428–435

  23. Mohring RH (1984) Minimizing costs of resource requirements in project networks subject to a fixed completion time. Oper Res 32(1):89–120

    Article  Google Scholar 

  24. Pelikan M, Goldberg D, Lobo F (2002) A survey of optimization by building and using probabilistic models. Comput Optim Appl 21(1):5–20

    Article  MathSciNet  Google Scholar 

  25. Radulescu A, Van Gemund AJ (2001) A low-cost approach towards mixed task and data parallel scheduling. In: International Conference on Parallel Processing (ICPP2001). IEEE, pp. 69–76

  26. Ranjbar M, Kianfar F, Shadrokh S (2008) Solving the resource availability cost problem in project scheduling by path relinking and genetic algorithm. Appl Math Comput 196:879–888

    MathSciNet  MATH  Google Scholar 

  27. Rodrigues SB, Yamashita DS (2010) An exact algorithm for minimizing resource availability costs in project scheduling. Eur J Oper Res 206:562–568

    Article  MathSciNet  Google Scholar 

  28. Shadrokh S, Kianfar F (2007) A genetic algorithm for resource investment project scheduling problem, tardiness permitted with penalty. Eur J Oper Res 181:86–101

    Article  MathSciNet  Google Scholar 

  29. Tang S, Yuan J, Wang C, Li XY (2014) A framework for amazon ec2 bidding strategy under sla constraints. IEEE Trans Parallel Distrib Syst 25(1):2–11

    Article  Google Scholar 

  30. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  31. Vanhoucke M, Coelho J, Debels D, Maenhout B, Tavares LV (2008) An evaluation of the adequacy of project network generators with systematically sampled networks. Eur J Oper Res 187(2):511–524

    Article  Google Scholar 

  32. Van den Bossche R, Vanmechelen K, Broeckhove J (2015) Iaas reserved contract procurement optimisation with load prediction. Future Gener Comput Syst 53:13–24

    Article  Google Scholar 

  33. Wieczorek M, Prodan R, Fahringer T (2005) Scheduling of scientific workflows in the ASKALON grid environment. ACM SIGMOD Rec 34(3):56–62

    Article  Google Scholar 

  34. Yamashita DS, Armentano VA, Laguna M (2006) Scatter search for project scheduling with resource availability cost. Eur J Oper Res 169:623–637

    Article  MathSciNet  Google Scholar 

  35. Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3):217–230

    Google Scholar 

  36. Yu J, Buyya R, Tham CK (2005) Cost-based scheduling of scientific workflow applications on utility grids. In: First International Conference on e-Science and Grid Computing (e-Science 2005). IEEE, p 8

  37. Yuan Y, Li X, Wang Q, Zhu X (2009) Deadline division-based heuristic for cost optimization in workflow scheduling. Inf Sci 179(15):2562–2575

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61572127), the Key Research & Development program in Jiangsu Province (No. BE2015728) and Collaborative Innovation Center of Wireless Communications Technology.

Author information

Authors and Affiliations

  1. Key Laboratory of Computer Network and Information Integration (Ministry of Education), Southeast University, Nanjing, 211189, China

    Long Chen & Xiaoping Li

Authors
  1. Long Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoping Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Li, X. Cloud workflow scheduling with hybrid resource provisioning. J Supercomput 74, 6529–6553 (2018). https://doi.org/10.1007/s11227-017-2043-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-017-2043-5

Keywords

Search

Navigation