Abstract
Controlling the granularity of workflow activities executed on widely distributed computing platforms such as grids is required to reduce the impact of task queuing and data transfer time. Most existing granularity control approaches assume extensive knowledge about the applications and resources (e.g. task duration on each resource), and that both the workload and available resources do not change over time. We propose a granularity control algorithm for platforms where such clairvoyant and offline conditions are not realistic. Our method groups tasks when the fineness degree of the application, which takes into account the ratio of shared data and the queuing/round-trip time ratio, becomes higher than a threshold determined from execution traces. The algorithm also de-groups task groups when new resources arrive. The application’s behavior is constantly monitored so that the characteristics useful for the optimization are progressively discovered. Experimental results, obtained with 3 workflow activities deployed on the European Grid Infrastructure, show that (i) the grouping process yields speed-ups of about 2.5 when the amount of available resources is constant and that (ii) the use of de-grouping yields speed-ups of 2 when resources progressively appear.
Chapter PDF
References
Glatard, T., et al.: A virtual imaging platform for multi-modality medical image simulation. IEEE Transactions on Medical Imaging 32, 110–118 (2013)
Shahand, S., et al.: Front-ends to Biomedical Data Analysis on Grids. In: Proceedings of HealthGrid 2011, Bristol, UK (June 2011)
Kacsuk, P.: P-GRADE Portal Family for Grid Infrastructures. Concurrency and Computation: Practice and Experience 23(3), 235–245 (2011)
Barbera, R., et al.: Supporting e-science applications on e-infrastructures: Some use cases from latin america. In: Grid Computing, pp. 33–55 (2011)
Ferreira da Silva, R., Glatard, T., Desprez, F.: Self-healing of operational workflow incidents on distributed computing infrastructures. In: CCGrid 2012, pp. 318–325 (2012)
da Silva, R.F., Glatard, T., Desprez, F.: Workflow fairness control on online and non-clairvoyant distributed computing platforms. In: Wolf, F., Mohr, B., an Mey, D. (eds.) Euro-Par 2013. LNCS, vol. 8097, pp. 102–113. Springer, Heidelberg (2013)
Muthuvelu, N., et al.: Task granularity policies for deploying bag-of-task applications on global grids. FGCS 29(1), 170–181 (2012)
Singh, G., et al.: Workflow task clustering for best effort systems with pegasus. In: Mardi Gras 2008, pp. 9:1–9:8. ACM, New York (2008)
Muthuvelu, N., et al.: A dynamic job grouping-based scheduling for deploying applications with fine-grained tasks on global grids. In: Proceedings of the 2005 Australasian Workshop on Grid Computing and E-Research, ACSW Frontiers 2005, vol. 44, pp. 41–48. Australian Computer Society, Inc. (2005)
Keat, N.W., et al.: Scheduling framework for bandwidth-aware job grouping-based scheduling in grid computing. Malaysian Journal of Computer Science 19 (2006)
Ang, T., et al.: A bandwidth-aware job grouping-based scheduling on grid environment. Information Technology Journal 8, 372–377 (2009)
Muthuvelu, N., Chai, I., Eswaran, C.: An adaptive and parameterized job grouping algorithm for scheduling grid jobs. In: 10th International Conference on Advanced Communication Technology, ICACT 2008, vol. 2, pp. 975–980 (2008)
Liu, Q., Liao, Y.: Grouping-based fine-grained job scheduling in grid computing. In: ETCS 2009, vol. 1, pp. 556–559 (2009)
Soni, V.K., et al.: Grouping-based job scheduling model in grid computing. World Academy of Science, Engineering and Technology 41, 781–784 (2010)
Zomaya, A.Y., Chan, G.: Efficient clustering for parallel tasks execution in distributed systems. In: 18th IPDPS, pp. 167–174 (2004)
Muthuvelu, N., Chai, I., Chikkannan, E., Buyya, R.: On-line task granularity adaptation for dynamic grid applications. In: Hsu, C.-H., Yang, L.T., Park, J.H., Yeo, S.-S. (eds.) ICA3PP 2010, Part I. LNCS, vol. 6081, pp. 266–277. Springer, Heidelberg (2010)
Ferreira da Silva, R., Glatard, T.: A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-Steps, and Workflow Executions. In: CoreGRID, Rhodes, GR (2012)
Glatard, T., Montagnat, J., Lingrand, D., Pennec, X.: Flexible and Efficient Workflow Deployment of Data-Intensive Applications on Grids with MOTEUR. IJHPCA 22(3), 347–360 (2008)
Tsaregorodtsev, A., et al.: DIRAC3. The New Generation of the LHCb Grid Software. Journal of Physics: Conference Series 219(6), 062029 (2009)
Cao, F., Commowick, O., Bannier, E., Ferré, J.-C., Edan, G., Barillot, C.: MRI estimation of t 1 relaxation time using a constrained optimization algorithm. In: Yap, P.-T., Liu, T., Shen, D., Westin, C.-F., Shen, L. (eds.) MBIA 2012. LNCS, vol. 7509, pp. 203–214. Springer, Heidelberg (2012)
Jensen, J., Svendsen, N.: Calculation of pressure fields from arbitrarily shaped, apodized and excited ultrasound transducers. IEEE T-UFFC 39(2), 262–267 (1992)
Reilhac, A., et al.: PET-SORTEO: Validation and Development of Database of Simulated PET Volumes. IEEE Trans. on Nuclear Science 52, 1321–1328 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
da Silva, R.F., Glatard, T., Desprez, F. (2013). On-Line, Non-clairvoyant Optimization of Workflow Activity Granularity on Grids. In: Wolf, F., Mohr, B., an Mey, D. (eds) Euro-Par 2013 Parallel Processing. Euro-Par 2013. Lecture Notes in Computer Science, vol 8097. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40047-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-40047-6_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40046-9
Online ISBN: 978-3-642-40047-6
eBook Packages: Computer ScienceComputer Science (R0)