On-Line, Non-clairvoyant Optimization of Workflow Activity Granularity on Grids

  • Rafael Ferreira da Silva
  • Tristan Glatard
  • Frédéric Desprez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8097)


Controlling the granularity of workflow activities executed on widely distributed computing platforms such as grids is required to reduce the impact of task queuing and data transfer time. Most existing granularity control approaches assume extensive knowledge about the applications and resources (e.g. task duration on each resource), and that both the workload and available resources do not change over time. We propose a granularity control algorithm for platforms where such clairvoyant and offline conditions are not realistic. Our method groups tasks when the fineness degree of the application, which takes into account the ratio of shared data and the queuing/round-trip time ratio, becomes higher than a threshold determined from execution traces. The algorithm also de-groups task groups when new resources arrive. The application’s behavior is constantly monitored so that the characteristics useful for the optimization are progressively discovered. Experimental results, obtained with 3 workflow activities deployed on the European Grid Infrastructure, show that (i) the grouping process yields speed-ups of about 2.5 when the amount of available resources is constant and that (ii) the use of de-grouping yields speed-ups of 2 when resources progressively appear.


Task Group Execution Trace Task Duration Data Transfer Time Granularity Control 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Glatard, T., et al.: A virtual imaging platform for multi-modality medical image simulation. IEEE Transactions on Medical Imaging 32, 110–118 (2013)CrossRefGoogle Scholar
  2. 2.
    Shahand, S., et al.: Front-ends to Biomedical Data Analysis on Grids. In: Proceedings of HealthGrid 2011, Bristol, UK (June 2011)Google Scholar
  3. 3.
    Kacsuk, P.: P-GRADE Portal Family for Grid Infrastructures. Concurrency and Computation: Practice and Experience 23(3), 235–245 (2011)CrossRefGoogle Scholar
  4. 4.
    Barbera, R., et al.: Supporting e-science applications on e-infrastructures: Some use cases from latin america. In: Grid Computing, pp. 33–55 (2011)Google Scholar
  5. 5.
    Ferreira da Silva, R., Glatard, T., Desprez, F.: Self-healing of operational workflow incidents on distributed computing infrastructures. In: CCGrid 2012, pp. 318–325 (2012)Google Scholar
  6. 6.
    da Silva, R.F., Glatard, T., Desprez, F.: Workflow fairness control on online and non-clairvoyant distributed computing platforms. In: Wolf, F., Mohr, B., an Mey, D. (eds.) Euro-Par 2013. LNCS, vol. 8097, pp. 102–113. Springer, Heidelberg (2013)Google Scholar
  7. 7.
    Muthuvelu, N., et al.: Task granularity policies for deploying bag-of-task applications on global grids. FGCS 29(1), 170–181 (2012)CrossRefGoogle Scholar
  8. 8.
    Singh, G., et al.: Workflow task clustering for best effort systems with pegasus. In: Mardi Gras 2008, pp. 9:1–9:8. ACM, New York (2008)Google Scholar
  9. 9.
    Muthuvelu, N., et al.: A dynamic job grouping-based scheduling for deploying applications with fine-grained tasks on global grids. In: Proceedings of the 2005 Australasian Workshop on Grid Computing and E-Research, ACSW Frontiers 2005, vol. 44, pp. 41–48. Australian Computer Society, Inc. (2005)Google Scholar
  10. 10.
    Keat, N.W., et al.: Scheduling framework for bandwidth-aware job grouping-based scheduling in grid computing. Malaysian Journal of Computer Science 19 (2006)Google Scholar
  11. 11.
    Ang, T., et al.: A bandwidth-aware job grouping-based scheduling on grid environment. Information Technology Journal 8, 372–377 (2009)CrossRefGoogle Scholar
  12. 12.
    Muthuvelu, N., Chai, I., Eswaran, C.: An adaptive and parameterized job grouping algorithm for scheduling grid jobs. In: 10th International Conference on Advanced Communication Technology, ICACT 2008, vol. 2, pp. 975–980 (2008)Google Scholar
  13. 13.
    Liu, Q., Liao, Y.: Grouping-based fine-grained job scheduling in grid computing. In: ETCS 2009, vol. 1, pp. 556–559 (2009)Google Scholar
  14. 14.
    Soni, V.K., et al.: Grouping-based job scheduling model in grid computing. World Academy of Science, Engineering and Technology 41, 781–784 (2010)Google Scholar
  15. 15.
    Zomaya, A.Y., Chan, G.: Efficient clustering for parallel tasks execution in distributed systems. In: 18th IPDPS, pp. 167–174 (2004)Google Scholar
  16. 16.
    Muthuvelu, N., Chai, I., Chikkannan, E., Buyya, R.: On-line task granularity adaptation for dynamic grid applications. In: Hsu, C.-H., Yang, L.T., Park, J.H., Yeo, S.-S. (eds.) ICA3PP 2010, Part I. LNCS, vol. 6081, pp. 266–277. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Ferreira da Silva, R., Glatard, T.: A Science-Gateway Workload Archive to Study Pilot Jobs, User Activity, Bag of Tasks, Task Sub-Steps, and Workflow Executions. In: CoreGRID, Rhodes, GR (2012)Google Scholar
  18. 18.
    Glatard, T., Montagnat, J., Lingrand, D., Pennec, X.: Flexible and Efficient Workflow Deployment of Data-Intensive Applications on Grids with MOTEUR. IJHPCA 22(3), 347–360 (2008)Google Scholar
  19. 19.
    Tsaregorodtsev, A., et al.: DIRAC3. The New Generation of the LHCb Grid Software. Journal of Physics: Conference Series 219(6), 062029 (2009)Google Scholar
  20. 20.
    Cao, F., Commowick, O., Bannier, E., Ferré, J.-C., Edan, G., Barillot, C.: MRI estimation of t 1 relaxation time using a constrained optimization algorithm. In: Yap, P.-T., Liu, T., Shen, D., Westin, C.-F., Shen, L. (eds.) MBIA 2012. LNCS, vol. 7509, pp. 203–214. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Jensen, J., Svendsen, N.: Calculation of pressure fields from arbitrarily shaped, apodized and excited ultrasound transducers. IEEE T-UFFC 39(2), 262–267 (1992)CrossRefGoogle Scholar
  22. 22.
    Reilhac, A., et al.: PET-SORTEO: Validation and Development of Database of Simulated PET Volumes. IEEE Trans. on Nuclear Science 52, 1321–1328 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Rafael Ferreira da Silva
    • 1
  • Tristan Glatard
    • 1
  • Frédéric Desprez
    • 2
  1. 1.CNRS, INSERM, CREATISUniversity of LyonVilleurbanneFrance
  2. 2.LIP, ENS LyonINRIA, University of LyonLyonFrance

Personalised recommendations