Advertisement

Journal of Grid Computing

, Volume 14, Issue 2, pp 347–358 | Cite as

A Set of Successive Job Allocation Models in Distributed Computing Infrastructures

  • Gábor Bacsó
  • Tamás Kis
  • Ádám Visegrádi
  • Attila Kertész
  • Zsolt Németh
Article

Abstract

The growing number of scientific computation-intensive applications calls for an efficient utilization of large-scale, potentially interoperable distributed infrastructures. Parameter sweep applications represent a large body of workflows. While the principle of workflows is easy to conceive, their execution is very complex and no universally accepted solution exists. In this paper we focus on the resource allocation challenges of parameter study jobs in distributed computing infrastructures. To cope with this NP-hard problem and the high uncertainty present in these systems, we propose a series of job allocation models that helps refining and simplifying the problem complexity. In this way we present some special cases that are polynomial and show how more complex scenarios can be reduced to these models. It is known from practice that a small number of job sizes improves the result of job allocation, therefore we state a hypothesis relying on this fact in one of our models. Unfortunately, the reduction of the general problem (using K-means clustering) did not help, and thus the hypothesis has proved to be false. In the future, we shall look for clustering techniques which fit this goal better.

Keywords

Distributed computing infrastructures Job allocation Parameter sweep applications 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Gil, Y., Deelman, E., Ellisman, M., Fahringer, T., Fox, G., Gannon, D., Goble, C., Livny, M., Moreau, L., Myers, J.: Examining the challenges of scientific workflows. IEEE Comput. 40(12), 26–34 (2007)CrossRefGoogle Scholar
  2. 2.
    Bacsó, G., Visegrádi, A., Kertesz, A., Németh, Z.: On efficiency of multi-job grid allocation based on statistical trace data. J. Grid Comput. (2013)Google Scholar
  3. 3.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)MATHGoogle Scholar
  4. 4.
    Lee, C.B., Schwartzman, Y., Hardy, J., Snavely, A.: Are User Runtime Estimates Inherently Inaccurate? vol. 3277, pp 253–263. Springer LNCS (2005)Google Scholar
  5. 5.
    Ramírez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., Gonzalez-García, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run time estimates for online scheduling in hierarchical grids. J. Grid Comput. 9(1), 95–116 (2011)Google Scholar
  6. 6.
    Schwiegelshohn, U., Tchernykh, A., Yahyapour, R.: Online scheduling in grids. In: 22nd IEEE International Symposium on Parallel and Distributed Processing (IPDPS), vol. 2008, pp 1–10 (2008)Google Scholar
  7. 7.
    Oprescu, A., Kielmann, T.: Bag-of-Tasks Scheduling under Budget Constraints. CloudCom, pp. 351–359 (2010)Google Scholar
  8. 8.
    Sgall, J.: On-line Scheduling - A Survey. Dagstuhl Seminar on On-Line Algorithms (Schloss Dagstuhl, Wadern, Germany, June 24-28, 1996), LNCS. Springer (1998)Google Scholar
  9. 9.
    Schuurman, P., Woeginger, G.: Approximation schemes – a tutorial. In: Möhring, R.H., Potts, C.N., Schulz, A.S., Woeginger, G.J., Wolsey, L.A. (eds.) Preliminary version of a chapter of the book ”Lectures on Scheduling” (to appear)Google Scholar
  10. 10.
    Arndt, O., Freisleben, B., Kielmann, T., Thilo, F.: A comparative study of on-line scheduling algorithms for networks of workstation. Clust. Comput. 3(2), 95–112 (2000)CrossRefGoogle Scholar
  11. 11.
    Braun, T.D., Siegel, H.J., Beck, N., Bölöni, L.L., Maheswaran, M., Reuther, A.I., Robertson, J.P., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 61(6), 810–837 (2001)CrossRefMATHGoogle Scholar
  12. 12.
    Ullman, J.D.: NP-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Xu, C., Lau, F.C.M.: Load Balancing in Parallel Computers: Theory and Practice. Springer Science & Business Media (1997)Google Scholar
  14. 14.
    Graham, R.L.: Bounds on multiprocessing timing anomalies. SIAM J. Appl. Math. 17(2), 416–429 (1969)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Smith, W.E.: Various optimizers for single-stage production. Nav. Res. Logist. Q. 3, 5917 (1956)MathSciNetGoogle Scholar
  16. 16.
    Schrage, L.: A proof of the shortest remaining processing time processing discipline. Oper. Res. 16, 687170 (1968)CrossRefMATHGoogle Scholar
  17. 17.
    Hirales-Carbajal, A., Tchernykh, A., Yahyapour, R., Gonzalez-Garcia, J.L., Roblitz, T., Ramirez-Alcaraz, J.M.: Multiple workflow scheduling strategies with user run time estimates on a grid. J. Grid Comput. (2012)Google Scholar
  18. 18.
    Rényi, A.: Probability Theory. Elsevier (1970)Google Scholar
  19. 19.
    Silberstein, M., Sharov, A., Geiger, D., Schuster, A.: GridBot, execution of bags of tasks in multiple grids. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC ’09) (2009)Google Scholar
  20. 20.
    Saha, D., Menasce, D., Porto, S.: Static and dynamic processor scheduling disciplines in heterogeneous parallel architectures. J. Parallel Distrib. Comput. 28.1, 1–18 (1995)MATHGoogle Scholar
  21. 21.
    Maheswaran, M., Ali, S., Siegal, H.J., Hensgen, D., Freund, R.F.: Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In: Proceedings of Heterogeneous Computing Workshop, 1999. (HCW’99), pp 30–44. IEEE (1999)Google Scholar
  22. 22.
    Casanova, H., et al.: Heuristics for scheduling parameter sweep applications in grid environments. In: Proceedings of 9th Heterogeneous Computing Workshop (HCW 2000). IEEE (2000)Google Scholar
  23. 23.
    Cirne, W., Paranhos, D., Costa, L., Santos-Neto, E., Brasileiro, F., Sauve, J., Silva, F.A.B., Barros, C.O., Silveira, C.: Running bag-of-tasks applications on computational grids: The mygrid approach, pp 407–416. IEEE (2003)Google Scholar
  24. 24.
    Da Silva, D.P., Cirne, W., Vilar Brasileiro, F.: Trading cycles for information: Using replication to schedule bag-of-tasks applications on computational grids. Euro-Par 2003 Parallel Processing, pp 169–180. Springer Berlin Heidelberg (2003)Google Scholar
  25. 25.
    Nielsen, F.: Chernoff information of exponential families. CoRR, arXiv: 1102.2684(2011)
  26. 26.
    SZTAKI Desktop Grid. http://szdg.lpds.sztaki.hu/szdg/ (2014)

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Gábor Bacsó
    • 1
  • Tamás Kis
    • 1
  • Ádám Visegrádi
    • 1
  • Attila Kertész
    • 1
  • Zsolt Németh
    • 1
  1. 1.MTA SZTAKI Computer and Automation Research InstituteBudapestHungary

Personalised recommendations