Journal of Grid Computing

, Volume 12, Issue 1, pp 169–186 | Cite as

On Efficiency of Multi-job Grid Allocation Based on Statistical Trace Data

  • Gábor Bacsó
  • Ádám Visegrádi
  • Attila Kertesz
  • Zsolt Németh
Article

Abstract

The ever growing number of computation-intensive applications calls for utilizing large-scale, potentially interoperable distributed infrastructures. Nowadays, such distributed systems enable the management of heterogeneous scientific workflows of considerable sizes, where job scheduling and resource management is a crucial issue. In this paper we focus on the challenges of scheduling parameter sweep applications, a specific and commonly used type of workflows where ordering of job executions is irrelevant. A parameter sweep has a large set of independent job instances, called a multi-job, submitted for execution in a single step. In order to cope with the high uncertainty and unpredictable load of resources, and the simultaneous submissions of multi-job instances, we propose a statistics-based brokering approach for allocating jobs to resources so that the makespan is minimised. Earlier studies claim that users’ predictions on job runtime are inaccurate and unusable for scheduling. Our aim is to examine, whether statistical trace data for the same purpose is efficient compared to randomized allocation.

Keywords

Grid computing Grid scheduling Allocation Grid brokering Workload traces 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Constantini, A.: RWavePR workflow at GASuC. Online: http://www.lpds.sztaki.hu/gasuc/index.php?-m=7&s=12 (2012). Accessed 1 Oct 2012
  2. 2.
    Wiggins, A.: Success-Abandonment-Classification workflow at myExperiment. Online: http://www.myexperiment.org/workflows/140.html (2012). Accessed 1 Oct 2012
  3. 3.
    Buyya, R., Murshed, M., Abramson, D.: Gridsim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing. In: Journal of Concurrency and Computation: Practice and Experience, pp. 1175–1220 (2002)Google Scholar
  4. 4.
    Casanova, H., et al.: Heuristics for scheduling parameter sweep applications in Grid environments. In: Proceedings 9th Heterogeneous Computing Workshop, (HCW 2000). IEEE, Press, Piscataway (2000)Google Scholar
  5. 5.
    Cirne, W., Paranhos, D., Costa, L., Santos-Neto, E., Brasileiro, F., Sauve, J., Silva, F.A.B., Barros, C.O., Silveira, C.: Running bag-of-tasks applications on computational Grids: the mygrid approach. In: International Conference on Parallel Processing, pp. 407–416. IEEE Press, Piscataway (2003)Google Scholar
  6. 6.
    Da Silva, D.P., Cirne, W., Vilar Brasileiro F.: Trading cycles for information: using replication to schedule bag-of-tasks applications on computational Grids. Euro-Par 2003 Parallel Processing, pp. 169–180. Springer Berlin Heidelberg (2003)Google Scholar
  7. 7.
    European Grid Infrastructure. Online: http://www.egi.eu/ (2012). Accessed 1 Oct 2012
  8. 8.
    Garey, M.R., Johnson D.S.: Computers and Intractability; a Guide to the Theory of Np-Completeness. W. H. Freeman & Co., New York (1979)MATHGoogle Scholar
  9. 9.
    Goble, C.A., Bhagat, J., Aleksejevs, S., Cruickshank, D., Michaelides, D., Newman, D., Borkum, M., Bechhofer, S., Roos, M., Li, P., De Roure, D.: myExperiment: a repository and social network for the sharing of bioinformatics workflows. Nucleic. Acids Res. 38(suppl 2), W677–W682 (2010)CrossRefGoogle Scholar
  10. 10.
    The Grid Workloads Archive website. Online: http://gwa.ewi.tudelft.nl (2010). Accessed 1 Oct 2012
  11. 11.
    Hirales-Carbajal, A., Tchernykh, A., Yahyapour, R., Gonzalez-Garcia, J.L., Roblitz, T., Ramirez-Alcaraz, J.M.: Multiple workflow scheduling strategies with user run time estimates on a Grid. J. Grid Comput. 10(2), 325–346 (2012)CrossRefGoogle Scholar
  12. 12.
    Howell, F., McNab, R.: SimJava: a discrete event simulation library for Java. In: Proc. of the International Conference on Web-Based Modeling and Simulation, San Diego, USA (1998)Google Scholar
  13. 13.
    Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L., Epema, D.H.J.: The Grid workloads archive. Futur. Gener. Comput. Syst. 24(7), 672–686 (2008)CrossRefGoogle Scholar
  14. 14.
    Kacsuk, P., Farkas, Z., Kozlovszky, M., Hermann, G., Balasko, A., Karoczkai, K., Marton, I.: WS-PGRADE/gUSE Generic DCI gateway framework for a large variety of user communities. J. Grid Comput. 9(4), 479–499 (2012)Google Scholar
  15. 15.
    Kwok, Y-K., Ahmad. I.: Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput. Surv. (CSUR) 31(4), 406–471 (1999)CrossRefGoogle Scholar
  16. 16.
    Lee, C.B., Schwartzman, Y., Hardy, J., Snavely, A.: Are user runtime estimates inherently inaccurate? Springer LNCS, vol. 3277, pp. 253–263 (2005)Google Scholar
  17. 17.
    Maheswaran, M., Ali, S., Siegal, H.J., Hensgen, D., Freund, RF.: Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In: Proceedings Heterogeneous Computing Workshop, (HCW’99), pp. 30–44. IEEE (1999)Google Scholar
  18. 18.
    Parallel workloads archive website. Online: http://www.cs.huji.ac.il/labs/parallel/workload (2009). Accessed 1 Oct 2012
  19. 19.
    Ramirez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., Gonzalez-Garcia, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run-time estimates for online scheduling in hierarchical Grids. J. Grid Computing 9(1), 95–116 (2011)CrossRefGoogle Scholar
  20. 20.
    Oprescu, A., Kielmann, T.: Bag-of-Tasks Scheduling under Budget Constraints. CloudCom, pp. 351–359 (2010)Google Scholar
  21. 21.
    Saha, D., Menasce, D., Porto, S.: Static and dynamic processor scheduling disciplines in heterogeneous parallel architectures. J. Parallel Distrib. Comput. 28.1, 1–18 (1995)Google Scholar
  22. 22.
    Schwiegelshohn, U., Tchernykh, A., Yahyapour, R.: Online scheduling in Grids. In: 22nd IEEE International Symposium on Parallel and Distributed Processing (IPDPS 2008), pp. 1–10 (2008)Google Scholar
  23. 23.
    SHaring Interoperable Workflows for large-scale scientific simulations on Available DCIs (SHIWA) Eu FP7 project. Online: http://www.shiwa-workflow.eu/ (2012). Accessed 1 Oct 2012
  24. 24.
    Building a European Research Community through Interoperable Workflows and Data (ER-flow) Eu FP7 project. Online: http://www.erflow.eu/ (2013). Accessed 1 Oct 2012
  25. 25.
    Silberstein, M., Sharov, A., Geiger, D., Schuster, A.: GridBot, execution of bags of tasks in multiple Grids. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC ’09) (2009)Google Scholar
  26. 26.
    Ullman, J.D.: NP-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Gábor Bacsó
    • 1
  • Ádám Visegrádi
    • 1
  • Attila Kertesz
    • 1
  • Zsolt Németh
    • 1
  1. 1.MTA SZTAKIBudapestHungary

Personalised recommendations