Abstract
A cloud workflow system is a type of platform service which facilitates the automation of distributed applications based on the novel cloud infrastructure. One of the most important aspects which differentiate a cloud workflow system from its other counterparts is the market-oriented business model. This is a significant innovation which brings many challenges to conventional workflow scheduling strategies. To investigate such an issue, this paper proposes a market-oriented hierarchical scheduling strategy in cloud workflow systems. Specifically, the service-level scheduling deals with the Task-to-Service assignment where tasks of individual workflow instances are mapped to cloud services in the global cloud markets based on their functional and non-functional QoS requirements; the task-level scheduling deals with the optimisation of the Task-to-VM (virtual machine) assignment in local cloud data centres where the overall running cost of cloud workflow systems will be minimised given the satisfaction of QoS constraints for individual tasks. Based on our hierarchical scheduling strategy, a package based random scheduling algorithm is presented as the candidate service-level scheduling algorithm and three representative metaheuristic based scheduling algorithms including genetic algorithm (GA), ant colony optimisation (ACO), and particle swarm optimisation (PSO) are adapted, implemented and analysed as the candidate task-level scheduling algorithms. The hierarchical scheduling strategy is being implemented in our SwinDeW-C cloud workflow system and demonstrating satisfactory performance. Meanwhile, the experimental results show that the overall performance of ACO based scheduling algorithm is better than others on three basic measurements: the optimisation rate on makespan, the optimisation rate on cost and the CPU time.
Similar content being viewed by others
References
Ardagna D, Pernici B (2007) Adaptive service composition in flexible processes. IEEE Trans Softw Eng 33(6):369–384
Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616
Cao H, Jin H, Wu X, Wu S (2009) ServiceFlow: QoS-based hybrid service-oriented grid workflow system. J Supercomput 53(3):371–393
Cao H, Jin H, Wu X, Wu S, Shi X (2009) DAGMap: efficient and dependable scheduling of DAG workflow job in grid. J Supercomput 51(2):201–223
Chang R, Chang J, Lin P (2009) An ant algorithm for balanced job scheduling in grids. Future Gener Comput Syst 25(1):20–27
Chen J, Yang Y (2007) Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Trans Auton Adaptive Syst 2(6)
Chen J, Yang Y (2007) Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurr Comput, Pract Exp 19(7):965–982
Chen J, Yang Y (2010) Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. In: ACM trans on software engineering and methodology, in press. http://www.ict.swin.edu.au/personal/yyang/papers/ACM-TOSEM-checkpoint.pdf. Accessed on 1st August 2010
Chen W, Zhang J, Chung HSH, Zhong W, Wu W, Shi Y (2009) A novel set-based particle swarm optimization method for discrete optimization problems. IEEE Trans Evol Comput 14(2):278–300
Chen W, Zhang J (2009) An ant colony optimization approach to a grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern, Part C, Appl Rev 39(1):29–43
Chen W, Zhang J, Yu Y (2007) Workflow scheduling in grids: an ant colony optimization approach. In: Proc 2007 IEEE congress on evolutionary computation, pp 3308–3315
Chin S, Suh T, Yu H (2009) Adaptive service scheduling for workflow applications in service-oriented grid. J Supercomput 52(3):253–283
Deelman E, Gannon D, Shields M, Taylor I (2008) Workflows and e-science: an overview of workflow system features and capabilities. Future Gener Comput Syst 25(6):528–540
Dou WC, Chen JJ, Liu JX, Cheung SC, Chen GH, Fan SK (2008) A workflow engine-driven SOA-based cooperative computing paradigm in grid environments. Int J High Perform Comput Appl 22(3):284–300
Thomas E (2008) SOA: principles of service design. Prentice Hall, New York
Foster I, Kesselman C (2004) The grid: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann, San Mateo
Foster I, Yong Z, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. In: Proc 2008 grid computing environments workshop, pp 1–10
Hadoop. http://hadoop.apache.org/. Accessed on 1st August 2010
Han JW, Kamber M (2006) Data mining: concepts and techniques, 2nd edn. Elsevier, Amsterdam
Kao B, Molina HG (1997) Deadline assignment in a distributed soft real-time system. IEEE Trans Parallel Distrib Syst 8(12):1268–1274
Kuo I, Horng S, Kao T, Lin T, Lee C, Terano T, Pan Y (2009) An efficient flow-shop scheduling algorithm based on a hybrid particle swarm optimization model. Expert Syst Appl 36(3):7027–7032, Part 2
Kwok Y, Ahmad I (1999) Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput Surv 31(4):406–471
Law AM, Kelton WD (2007) Simulation modelling and analysis, 4th edn. McGraw-Hill, New York
Liu K, Chen J, Yang Y, Jin H (2008) A throughput maximization strategy for scheduling transaction-intensive workflows on SwinDeW-G. Concurr Comput, Pract Exp 20(15):1807–1820
Liu X, Chen J, Wu Z, Ni Z, Yuan D, Yang Y (2010) Handling recoverable temporal violations in scientific workflow systems: a workflow rescheduling based strategy. In: Proc 10th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGrid10), Melbourne, Australia, pp 534–537
Liu X, Chen J, Yang Y (2008) A probabilistic strategy for setting temporal constraints in scientific workflows. In: Proc 6th international conference on business process management (BPM2008), Milan, Italy, vol 5204, pp 180–195
Liu X, Yang Y, Chen J, Wang Q, Li M (2009) Achieving on-time delivery: a two-stage probabilistic scheduling strategy for software projects. In: Proc the 2009 international conference on software process: trustworthy software development processes, Vancouver, BC, Canada. Springer, Berlin
Liu X, Yang Y, Jiang Y, Chen J (2010) Preventing temporal violations in scientific workflows: where and how. IEEE Trans Softw Eng. doi:10.1109/TSE.2010.99. http://www.ict.swin.edu.au/personal/xliu/papers/TSE-2010-01-0018.pdf. Accessed on 1st August 2010
Martinez A, Alfaro FJ, Sanchez JL, Quiles FJ, Duato J (2007) A new cost-effective technique for QoS support in clusters. IEEE Trans Parallel Distrib Syst 18(12):1714–1726
Moore M (2004) An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Comput 30(1):567–583
Mullen RJ, Monekosso D, Barman S, Remagnino P (2009) A review of ant algorithms. Expert Syst Appl 36(6):9608–9617
Oh J, Wu C (2004) Genetic-algorithm-based real-time task scheduling with multiple goals. J Syst Softw 71:245–258
Raghavan B, Ramabhadran S, Yocum K, Snoeren AC (2007) Cloud control with distributed rate limiting. In: Proc 2007 ACM SIGCOMM, Kyoto, Japan, pp 337–348
Buyya R, Bubendorfer K (2009) Market oriented grid and utility computing. Wiley, New York
Russell N, van der Aalst WMP, ter Hofstede AHM (2006) Exception handling patterns in process-aware information systems. Technical Report BPM-06-04, BPMcenter.org
Sahai A, Durante A, Machiraju V (2002) Towards automated SLA management for web services. Technical Report HPL-2001-310 (R.1), HP Laboratories Palo Alto
SECES (2008) Proc first international workshop on software engineering for computational science and engineering, in conjunction with the 30th international conference on software engineering (ICSE2008), Leipzig, Germany, May 2008
Taylor IJ, Deelman E, Gannon DB, Shields M (2007) Workflows for e-science: scientific workflows for grids. Springer, Berlin
Tracy DB, Howard JS, Noah B (2001) Comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J Parallel Distrib Comput 61(6):810–837
van der Aalst WMP, Hee KMV (2002) Workflow management: models, methods, and systems. MIT Press, Cambridge
VMware. http://www.vmware.com/. Accessed on 1st August 2010
Wang M, Kotagiri R, Chen J (2009) Trust-based robust scheduling and runtime adaptation of scientific workflow. Concurr Comput, Pract Exp 21(16):1982–1998
Wieczorek M, Prodan R, Hoheisel A (2007) Taxonomies of the multi-criteria gridworkflow scheduling problem. CoreGRID Technical Report Number TR-0106, August 30
Yan J, Yang Y, Raikundalia GK (2006) SwinDeW—a peer-to-peer based decentralized workflow management system. IEEE Trans Syst Man Cybern, Part A, Syst Hum 36(5):922–935
Yang Y, Liu K, Chen J, Lignier J, Jin H (2007) Peer-to-peer based grid workflow runtime environment of SwinDeW-G. In: Proc 3rd international conference on e-science and grid computing (e-Science07), Bangalore, India, pp 51–58
Yang Y, Liu K, Chen J, Liu X, Yuan D, Jin H (2008) An algorithm in SwinDeW-C for scheduling transaction-intensive cost-constrained cloud workflows. In: Proc 4th IEEE international conference on e-science (e-Science08), Indianapolis, USA, pp 374–375
Yang Z, Koelbel C, Cooper K (2009) Hybrid Re-scheduling mechanisms for workflow applications on multi-cluster grid. In: Proc 9th IEEE/ACM international symposium on cluster computing and the grid, pp 116–123
Yu J, Buyya R (2005) A taxonomy of workflow management systems for grid computing. J Grid Comput 3:171–200
Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3–4):217–230
Yu J, Buyya R (2008) Workflow scheduling algorithms for grid computing. In: Xhafa F, Abraham A (eds) Metaheuristics for scheduling in distributed computing environments. ISBN:978-3-540-69260-7. Springer, Berlin
Zhang L, Chen Y, Yang B (2006) Task scheduling based on PSO algorithm in computational grid. In: Proceedings—ISDA 2006: Sixth international conference on intelligent systems design and applications, Jinan, vol 2, pp 696–701
Zhang XD, Li XP, Wang Q, Yuan YC (2008) Hybrid particle swarm optimization algorithm for cost minimization in service-workflows with due dates. Tongxin Xuebao/Commun 29(8)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wu, Z., Liu, X., Ni, Z. et al. A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomput 63, 256–293 (2013). https://doi.org/10.1007/s11227-011-0578-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-011-0578-4