Abstract
One of the main challenges in volunteer computing systems is scheduling large-scale applications expressed as scientific workflows. This work aims to integrate partitioning scientific workflows and proximity-aware resource provisioning to increase the percentage of workflows that meet the deadline in peer-to-peer based volunteer computing systems. In the partitioning phase, a scientific workflow is partitioned into sub-workflows in order to minimize data dependencies among them. We utilize knowledge-free load balancing policy and proximity of resources to distribute sub-workflows on volunteer resources. Simulation results show that the proposed workflow scheduling system improves the percentage of scientific workflows that meet the deadline with average of 18% under a moderate workload.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@home: An experiment in public-resource computing. Commun. ACM 45, 56–61 (2002)
EDGeS@Home project, http://home.edges-grid.eu
Christensen, C., Aina, T., Stainforth, D.: The challenge of volunteer computing with lengthy climate model simulation. In: The 1st IEEE International Conference on e-Science and Grid Computing, pp. 8–15. IEEE Press, New York (2005)
Berriman, G.B., Deelman, E., et al.: Montage: a grid-enabled engine for delivering custom science grade mosaics on demand. In: 16th Annual Symposium Electronic Imaging Sci-ence and Technology, pp. 221–232. SPIE Press, California (2004)
Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of Scientific Workflows. In: The Third Workshop on Workflows in Support of Large-scale Science, pp. 1–10. IEEE Press, New York (2008)
Basher, N., Mahanti, A., Williamson, C., Arlitt, M.: A comparative analysis of web and peer-to-peer traffic. In: The 17th International Conference on World Wide Web, pp. 287–296. ACM Press, New York (2008)
Livny, J., Teonadi, H., Livny, M., Waldor, M.K.: High-throughput, kingdom-wide prediction and annotation of bacterial Non-Coding RNAs. PLoS ONE 3, e3197 (2008)
Ghafarian, T., Deldari, H., Javadi, B., Yaghmaee, M.H., Buyya, R.: CycloidGrid: A prox-imity-aware P2P-based resource discovery architecture in volunteer computing systems. Future Gener. Comp. Sy. 29, 1583–1595 (2013)
Di Gaspero, L., Schaerf, A.: Neighborhood portfolio approach for local search applied to timetabling problems. Journal of Mathematical Modeling and Algorithms 5, 65–89 (2006)
Topcuouglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13, 260–274 (2002)
Hordijk, A., der Laan, D.V.: Periodic routing to parallel queues and billiard sequences. Math. Method Oper. Res. 59, 173–192 (2004)
Medina, A., Lakhina, A., Matta, I., Byers, J.: BRITE: an approach to universal topology generation. In: IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pp. 346–353. IEEE Press, New York (2001)
Chen, W., Deelman, E.: Integration of workflow partitioning and resource provisioning. In: 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 764–768. IEEE Press, New York (2012)
Blythe, J., Jain, S., Deelman, E., Gil, Y., Vahi, K., Mandal, A., Kennedy, K.: Task scheduling strategies for workflow-based applications in Grids. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 759–767. IEEE Press, New York (2005)
Braun, T.D., Siegel, H.J., et al.: A Comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distr. Com. 61, 810–837 (2001)
Duan, R., Prodan, R., Fahringer, T.: Run-time Optimization of grid workflow applications. In: 7th IEEE/ACM International Conference on Grid Computing, pp. 33–40. IEEE Press, New York (2005)
Wieczorek, M., Prodan, R., Fahringer, T.: Scheduling of scientific workflows in the ASKALON grid environment. SIGMOND Record 34, 56–62 (2005)
Dong, F., Akl, S.: Two-phase computation and data scheduling algorithms for workflows in the grid. In: International Conference on Parallel Processing, p. 66. IEEE Press, New York (2007)
Kalayci, S., Dasgupta, G., Fong, L., Ezenwoye, O., Sadjadi, S.: Distributed and adaptive execution of condor DAGman workflows. In: The 23rd International Conference on Software Engineering and Knowledge Engineering, pp. 587–590. Knowledge Systems Institute, Illinois (2010)
Lin, C., Shih, C., Hsu, C.: Adaptive dynamic scheduling algorithms for mapping ongoing m-tasks to pr 2 grid. J. Inf. Sci. Eng. 26, 2107–2125 (2010)
Kumar, S., Das, S., Biswas, R.: Graph partitioning for parallel applications in heterogene-ous grid environments. In: International Parallel and Distributed Processing Symposium, pp. 66–72. IEEE Press, New York (2002)
Sanchis, L.A.: Multiple-way network partitioning. IEEE Trans. on Computers 38, 62–81 (1989)
Sanchis, L.A.: Multiple-way network partitioning with different cost functions. IEEE Trans. on Computers 42, 1500–1504 (1993)
Benlic, U., Hao, J.K.: An effective multilevel tabu search approach for balanced graph partitioning. Comput. Oper. Res. 38, 1066–1075 (2010)
Ghafarian, T., Deldari, H., Javadi, B., Buyya, R.: A proximity-aware load balancing in peer-to-peer-based volunteer computing systems. J. Supercomput. 65, 797–822 (2013)
Malawski, M., Juve, G., Deelman, E., Nabrzyskiz, J.: Cost and Deadline-Constrained Pro-visioning for Scientific Workflow Ensembles in IaaS Clouds. In: The International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–11. IEEE Press, New York (2012)
Deelman, E., Callaghan, S., Field, E., Francoeur, H., Graves, R., Gupta, V., Jordan, T.H., Kesselman, C., Maechling, P., Mehta, G., Kaya, D.O., Vahi, K., Zhao, L.: Managing large-scale Workflow Execution from resource provisioning to provenance tracking: the CyberShake example. In: Proceedings of the Second IEEE international Conference on E-Science and Grid Computing, p. 14. IEEE Press, New York (2006)
Workflow Generator, https://conflence.pegasus.isi.edu/display/Pegasus/WorkflowGenerator
Iosup, A., Sonmez, O., Anoep, S., Epema, D.: The performance of Bags-of-Tasks in large-scale distributed systems. In: The 17th International Symposium on High Performance Distributed Computing, pp. 97–108. ACM Press, New York (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Ghafarian, T., Javadi, B. (2013). Deadline-Constrained Workflow Scheduling in Volunteer Computing Systems. In: Kołodziej, J., Di Martino, B., Talia, D., Xiong, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2013. Lecture Notes in Computer Science, vol 8285. Springer, Cham. https://doi.org/10.1007/978-3-319-03859-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-03859-9_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03858-2
Online ISBN: 978-3-319-03859-9
eBook Packages: Computer ScienceComputer Science (R0)