Advertisement

Journal of Grid Computing

, Volume 9, Issue 1, pp 95–116 | Cite as

Job Allocation Strategies with User Run Time Estimates for Online Scheduling in Hierarchical Grids

  • Juan Manuel Ramírez-Alcaraz
  • Andrei Tchernykh
  • Ramin Yahyapour
  • Uwe Schwiegelshohn
  • Ariel Quezada-Pina
  • José Luis González-García
  • Adán Hirales-Carbajal
Article

Abstract

We address non-preemptive non-clairvoyant online scheduling of parallel jobs on a Grid. We consider a Grid scheduling model with two stages. At the first stage, jobs are allocated to a suitable Grid site, while at the second stage, local scheduling is independently applied to each site. We analyze allocation strategies depending on the type and amount of information they require. We conduct a comprehensive performance evaluation study using simulation and demonstrate that our strategies perform well with respect to several metrics that reflect both user- and system-centric goals. Unfortunately, user run time estimates and information on local schedules does not help to significantly improve the outcome of the allocation strategies. When examining the overall Grid performance based on real data, we determined that an appropriate distribution of job processor requirements over the Grid has a higher performance than an allocation of jobs based on user run time estimates and information on local schedules. In general, our experiments showed that rather simple schedulers with minimal information requirements can provide a good performance.

Keywords

Grid computing Online scheduling Resource management Job allocation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. International Journal of Software: Practice and Experience (SPE) 32, 135–164 (2002)zbMATHCrossRefGoogle Scholar
  2. 2.
    Rodero, I., Corbalan, J., Badía, R.M., Labarta, J.: eNANOS Grid resource broker. In: Advances in Grid Computing. European Grid Conference (EGC 2005), pp. 111–121. Springer, Amsterdam (2005)Google Scholar
  3. 3.
    Rodero, I., Guim, F., Corbalan, J., Goyeneche, A.: The Grid backfilling: a multi-site scheduling architecture with data mining prediction techniques. In: Talia, D., Yahyapour, R., Ziegler, W. (eds.) Grid Middleware and Services. Challenges and Solutions, vol. 8, pp. 137–152. Springer, New York (2008)CrossRefGoogle Scholar
  4. 4.
    Elmroth, E., Tordsson, J.: An interoperable, standards-based Grid resource broker and job submission service. In: First International Conference on e-Science and Grid Computing, 2005, pp. 212–220. IEEE Computer Society, Melbourne, Vic. (2005)Google Scholar
  5. 5.
    Avellino, G., Beco, S., Cantalupo, B., Maraschini, A., Pacini, F., Terracina, A., Barale, S., Guarise, A., Werbrouck, A., Sezione Di Torino, Colling, D., Giacomini, F., Ronchieri, E., Gianelle, A., Peluso, R., Sgaravatto, M., Mezzadri, M., Prelz, F., Salconi, L.: The EU datagrid workload management system: towards the second major release. In: 2003 Conference for Computing in High Energy and Nuclear Physics. University of California, La Jolla, California, USA (2003)Google Scholar
  6. 6.
    Ranganathan, K., Foster, I.: Simulation studies of computation and data scheduling algorithms for data Grids. Journal Grid Computing 1, 53–62 (2003)CrossRefGoogle Scholar
  7. 7.
    Derbal, Y.: Entropic Grid scheduling. Journal Grid Computing 4, 373–394 (2006)zbMATHCrossRefGoogle Scholar
  8. 8.
    de Lucchese, O.F., Huerta Yero, E., Sambatti, F., Henriques, M.: An adaptive scheduler for Grids. Journal Grid Computing 4, 1–17 (2006)CrossRefGoogle Scholar
  9. 9.
    Ernemann, C., Yahyapour, R.: Applying economic scheduling methods to Grid environments. In: Grid Resource Management: State of the Art and Future Trends, pp. 491–506. Kluwer, Dordrecht (2004)Google Scholar
  10. 10.
    Ernemann, C., Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On advantages of Grid computing for parallel job scheduling. In: 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 39. IEEE Computer Society (2002)Google Scholar
  11. 11.
    Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global Grid computing for job scheduling. In: Fifth IEEE/ACM International Workshop on Grid Computing (Grid ’04), in Conjunction with SuperComputing 2004, pp. 374–379. IEEE Computer Society, Pittsburgh (2004)Google Scholar
  12. 12.
    Vázquez-Poletti, J.L., Huedo, E., Montero, R.S., Llorente, I.M.: A comparison between two Grid scheduling philosophies: EGEE WMS and Grid way. Multiagent and Grid System. Grid Computing, High Performance and Distributed Applications 3, 429–439 (2007)zbMATHGoogle Scholar
  13. 13.
    Schwiegelshohn, U., Yahyapour, R.: Attributes for communication between Grid scheduling instances. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management: State of the Art and Future Trends, pp. 41–52. Kluwer, Norwell (2004)Google Scholar
  14. 14.
    Kurowski, K., Nabrzyski, J., Oleksiak, A., Weglarz, J.: A multicriteria approach to two-level hierarchy scheduling in Grids. J. Sched. 11, 371–379 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Zikos, S., Karatza, H.D.: Resource allocation strategies in a 2-level hierarchical Grid system. In: Simulation Symposium, 2008. ANSS 2008. 41st Annual, pp. 157–164. Ottawa, Ont. (2008)Google Scholar
  16. 16.
    Chunlin, L., Layuan, L.: Multi-level scheduling for global optimization in Grid computing. Comput. Electr. Eng. 34, 202–221 (2008)zbMATHCrossRefGoogle Scholar
  17. 17.
    Wäldrich, O., Wieder, P., Ziegler, W.: A meta-scheduling service for co-allocating arbitrary types of resources. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) Parallel Processing and Applied Mathematics, vol. 3911, pp. 782–791. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Tchernykh, A., Ramírez, J., Avetisyan, A., Kuzjurin, N., Grushin, D., Zhuk, S.: Two level job-scheduling strategies for a computational Grid. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) 6th International Conference on Parallel Processing and Applied Mathematics PPAM 2005, LNCS, vol. 3911, pp. 774–781. Springer, Heidelberg (2006)Google Scholar
  19. 19.
    Zhuk, S., Chernykh, A., Avetisyan, A., Gaissaryan, S., Grushin, D., Kuzjurin, N., Pospelov, A., Shokurov, A.: Comparison of scheduling heuristics for Grid resource broker. In: Third International IEEE Conference on Parallel Computing Systems (PCS 2004), pp. 388–392. IEEE, Colima, Colima, México (2004)Google Scholar
  20. 20.
    Pugliese, A., Talia, D., Yahyapour, R.: Modeling and supporting Grid scheduling. Journal of Grid Computing 6, 195–213 (2008)CrossRefGoogle Scholar
  21. 21.
    Schwiegelshohn, U.: An owner-centric metric for the evaluation of online job schedules. In: Multidisciplinary International Conference on Scheduling. Theory and Applications (MISTA 2009), pp. 557–569. Dublin, Ireland (2009)Google Scholar
  22. 22.
    Graham, R.L., Lawler, E.L., Lenstra, J.K., Rinnooy Kan, A.H.G.: Optimization and approximation in deterministic sequencing and scheduling: a survey. In: Hammer, P.L., Johnson, E.L., Korte, B.H. (eds.) Annals of Discrete Mathematics 5. Discrete Optimization II, pp. 287–326. North-Holland, Amsterdam (1979)Google Scholar
  23. 23.
    Naroska, E., Schwiegelshohn, U.: On an on-line scheduling problem for parallel jobs. Inf. Process. Lett. 81, 297–304 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Schwiegelshohn, U., Tchernykh, A., Yahyapour, R.: Online scheduling in Grids. In: IEEE International Symposium on Parallel and Distributed Processing 2008 (IPDPS 2008), pp. 1–10. Miami, FL, USA (2008)Google Scholar
  25. 25.
    Garey, M.R., Graham, R.L.: Bounds for multiprocessor scheduling with resource constraints. SIAM J. Comput. 4, 187–200 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids. In: Priol, T., Vanneschi, M. (eds.) From Grids to Service and Pervasive Computing, pp. 77–91. Springer, New York (2008)CrossRefGoogle Scholar
  27. 27.
    Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids with admissible allocation. J. Sched. 13, 545–552 (2010). doi: 10.1007/s10951-010-0169-x MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Bar-Noy, A., Freund, A.: On-line load balancing in a hierarchical server topology. SIAM J. Comput. 31, 527–549 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Pascual, F., Rzadca, K., Trystram, D.: Cooperation in multi-organization scheduling. Concurr. Comput.: Practice and Experience 21, 905–921 (2009)CrossRefGoogle Scholar
  30. 30.
    Zhuk, S.: Approximate algorithms to pack rectangles into several strips. Discrete Math. Appl. 16, 73–85 (2007)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Bougeret, M., Dutot, P.-F., Jansen, K., Otte, C., Trystram, D.: A fast 5/2 approximation algorithm for hierarchical scheduling. In: 16th International European Conference on Parallel and Distributed Computing, Euro-Par 2010. Ischia, Italy (2010)Google Scholar
  32. 32.
    Tsafrir, D., Etsion, Y., Feitelson, D.G.: Modeling user runtime estimates. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) 11th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2005). LNCS, vol. 3834, pp. 1–35. Springer, Cambridge (2006)CrossRefGoogle Scholar
  33. 33.
    Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18, 789–803 (2007)CrossRefGoogle Scholar
  34. 34.
    Chiang, S.-H., Arpaci-Dusseau, A.C., Vernon, M.K.: The impact of more accurate requested runtimes on production job scheduling performance. In: 8th International Workshop on Job Scheduling Strategies for Parallel Processing, pp. 103–127. Springer Verlang (2002)Google Scholar
  35. 35.
    Bailey Lee, C., Schwartzman, Y., Hardy, J., Snavely, A.: Are user runtime estimates inherently inaccurate? In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) Job Scheduling Strategies for Parallel Processing. Springer, New York (2004)Google Scholar
  36. 36.
    Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12, 529–543 (2001)CrossRefGoogle Scholar
  37. 37.
    Guim, F., Corbalan, J., Labarta, J.: Prediction of based models for evaluating backfilling scheduling policies. In: Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 9–17. IEEE Computer Society (2007)Google Scholar
  38. 38.
    Goyeneche, A., Guim, F., Rodero, I., Terstyanszky, G., Corbalan, J.: Extracting performance hints for Grid Users using data mining techniques: a case study in the NGS. The Mediterranean Journal of Computers and Networks (MEDJCN). SPECIAL ISSUE on Data Mining Applications on Supercomputing and Grid Environments 3(2), 52–61 (2007)Google Scholar
  39. 39.
    Smith, W.: Improving resource selection and scheduling using predictions. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid resource management: state of the art and future trends, pp. 237–253. Kluwer, Dordrecht (2004)Google Scholar
  40. 40.
    Tsafrir, D., Feitelson, D.G.: The dynamics of backfilling: solving the mystery of why increased inaccuracy may help. In: IEEE International Symposium on Workload Characterization (IISWC 2006), pp. 131–141. IEEE, San Jose, California (2006)Google Scholar
  41. 41.
    Zotkin, D., Keleher, P.J.: Job-length estimation and performance in backfilling schedulers. In: Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8 ’99), pp. 39–46. IEEE Computer Society (1999)Google Scholar
  42. 42.
    Talby, D., Tsafrir, D., Goldberg, Z., Feitelson, D.G.: Session-based, estimation-less, and information-less runtime prediction algorithms for parallel and Grid job scheduling. Technical report, School of Computer Science and Engineering, Hebrew University of Jerusalem (2006)Google Scholar
  43. 43.
  44. 44.
    Grid Workloads Archive, TU Delft. http://gwa.ewi.tudelft.nl

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Juan Manuel Ramírez-Alcaraz
    • 1
    • 2
  • Andrei Tchernykh
    • 1
  • Ramin Yahyapour
    • 3
  • Uwe Schwiegelshohn
    • 4
  • Ariel Quezada-Pina
    • 1
  • José Luis González-García
    • 1
  • Adán Hirales-Carbajal
    • 1
    • 5
  1. 1.Computer Science DepartmentCICESE Research CenterEnsenadaMéxico
  2. 2.Telematics FacultyColima UniversityColimaMéxico
  3. 3.IT and Media CenterTechnische Universität DortmundDortmundGermany
  4. 4.Robotics Research InstituteTechnische Universität DortmundDortmundGermany
  5. 5.Science FacultyAutonomous University of Baja CaliforniaEnsenadaMéxico

Personalised recommendations