Advertisement

The Characteristics and Performance of Groups of Jobs in Grids

  • Alexandru Iosup
  • Mathieu Jan
  • Ozan Sonmez
  • Dick Epema
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4641)

Abstract

Even though with few exceptions, grid workloads are dominated by single-node jobs, not all of these jobs are necessarily independent or unrelated. For instance, sets of jobs may be grouped because they are submitted by users in batches, e.g., to perform parameter sweeps. However, there is no reported data to confirm the presence and structure of these groupings, despite the large potential impact of such information. To address this lack of information, in this work we present a first investigation into the characteristics of groups of jobs present in grid workloads. First, we define three types of job groupings: batch, continued, and bursty submissions. Then, we analyze the characteristics of these groupings for three long-term traces from currently deployed grid environments. Notably, our results show that the various groupings are responsible for up to 96% of the total CPU time consumption. Finally, we present insights into the performance of real grids in dealing with grouped jobs.

Keywords

Cumulative Distribution Function Batch Size Interarrival Time Grid Environment Arrival Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bolze, R., Cappello, F., Caron, E., Daydè, M., Desprez, F., Jeannot, E., Jègou, Y., Lanteri, S., Leduc, J., Melab, N., Mornet, G., Namyst, R., Primet, P., Quetier, B., Richard, O., Talbi, E.G., Irena, T.: Grid’5000: a large scale and highly reconfigurable experimental grid testbed. International Journal of High Performance Computing Applications 20(4), 481–494 (2006)Google Scholar
  2. 2.
    Eerola, P., Kónya, B., Smirnova, O., Ekelöf, T., Ellert, M., Hansen, J.R., Nielsen, J.L., Wäänänen, A., Konstantinov, A., Herrala, J., Tuisku, M., Myklebust, T., Ould-Saada, F., Vinter, B.: The nordugrid production grid infrastructure, status and plans. In: The 4th International Workshop on Grid Computing (Grid2003), Phoenix, AZ, USA, pp. 158–165 (2003)Google Scholar
  3. 3.
    Iosup, A., Dumitrescu, C., Epema, D., Li, H., Wolters, L.: How are real grids used? the analysis of four grid traces and its implications. In: IEEE/ACM, pp. 262–269. IEEE Computer Society Press, Los Alamitos (2006)Google Scholar
  4. 4.
    Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2005)CrossRefGoogle Scholar
  5. 5.
    Karonis, N.T., Toonen, B.R., Foster, I.T.: MPICH-G2: A grid-enabled implementation of the message passing interface. J. Parallel Distrib. Comput. 63(5), 551–563 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    van Reeuwijk, K., van Niewpoort, R.V., Bal, H.E.: Developing Java Grid applications with Ibis. In: Cunha, J.C., Medeiros, P.D. (eds.) Euro-Par 2005. LNCS, vol. 3648, pp. 411–420. Springer, Heidelberg (2005)Google Scholar
  7. 7.
    Feitelson, D.G., Rudolph, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and Practice in Parallel Job Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)Google Scholar
  8. 8.
    Feitelson, D.G.: Packing schemes for gang scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP. LNCS, vol. 1162, pp. 89–110. Springer, Heidelberg (1996)Google Scholar
  9. 9.
    Squillante, M., Yao, D., Zhang, L.: Analysis of job arrival patterns and parallel scheduling performance. Perform. Eval. 36-37(1-4), 137–163 (1999)CrossRefGoogle Scholar
  10. 10.
    Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing 63(11), 1105–1122 (2003)zbMATHCrossRefGoogle Scholar
  11. 11.
    Costa, G.D., Georgiou, Y., Huard, G., Martin, C., Mouniè, G., Neyron, P., Richard, O.: A batch scheduler with high level components. In: CCGrid 2005. Proceedings of the 5th IEEE/ACM International Symposium on Cluster Computing and the Grid, Cardiff, UK, pp. 776–783. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  12. 12.
    Ellert, M., et al.: Advanced Resource Connector middleware for lightweight computational Grids. FGCS 23(2), 219–240 (2007)Google Scholar
  13. 13.
    Dumitrescu, C., Raicu, I., Ripeanu, M., Foster, I.: DiPerF: An automated distributed performance testing framework. In: GRID 2004. Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, Pittsburgh, PA, USA, pp. 289–296. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  14. 14.
    Dumitrescu, C., Raicu, I., Foster, I.: Experiences in Running Workloads over Grid3. In: Zhuge, H., Fox, G.C. (eds.) GCC 2005. LNCS, vol. 3795, pp. 274–286. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Thain, D., Bent, J., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H., Livny, M.: Pipeline and batch sharing in grid workloads. In: HPDC, pp. 152–161. IEEE Computer Society, Los Alamitos (2003)Google Scholar
  16. 16.
    Bonacorsi, D., et al.: Running CMS software on GRID Testbeds. Computing in High Energy and Nuclear Physics, 24-28, March 2003, La Jolla, CA, USA (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Alexandru Iosup
    • 1
  • Mathieu Jan
    • 1
  • Ozan Sonmez
    • 1
  • Dick Epema
    • 1
  1. 1.Department of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, The Netherlands, Members of the CoreGRID European Virtual Institute on Grid Scheduling 

Personalised recommendations