Abstract
Pilot-job systems have emerged as a computation paradigm to cope with heterogeneity of production grids, greatly improving fault ratios and latency. Tools like DIANE, WISDOM-II, ToPoS and Condor glideIns are now being widely adopted to conduct large-scale experiments on such platforms. However, a model of pilot-job applications is still lacking, making it difficult to determine submission parameters such as the number of pilots to submit to achieve a given performance level. The variability of production conditions and the heterogeneity of the underlying middleware and infrastructure further complicates this issue. This paper presents a performance model for pilot-job applications running on production grids. Based on a probabilistic modelling, we derive statistics about the number of available pilots along time and the makespan of the application given the number of submitted pilots. Results obtained on a radiotherapy application running on the EGEE production grid show that the model is accurate enough to correctly describe the behavior of the application, setting the basis for further optimization strategies.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Ahn, S., Namgyu, K., Seehoon, L., Soonwook, H., Dukyun, N., Koblitz, B., Breton, V., Sangyong, H.: Improvement of Task Retrieval Performance Using AMGA in a Large-Scale Virtual Screening. In: NCM’08, pp. 456–463 (September 2008)
Fu, S., Xu, C.-Z.: Exploring event correlation for failure prediction in coalition of clusters. In: Supercomputing (2007)
Germain, C., Loomis, C., Mosciki, J.T., Texier, R.: Scheduling for Responsive Grids. JGC 6(1), 15–27 (2008)
Glatard, T., Lingrand, D., Montagnat, J., Riveill, M.: Impact of the execution context on Grid job performances. In: WCAMG’07 (CCGrid’07), pp. 713–718 (May 2007)
Glatard, T., Montagnat, J., Pennec, X.: Probabilistic and dynamic optimization of job partitioning on a grid infrastructure. In: PDP’06, pp. 231–238 (February 2006)
Glatard, T., Montagnat, J., Pennec, X.: Optimizing jobs timeouts on clusters and production grids. In: CCGrid’07, pp. 100–107 (May 2007)
Jacq, N., Salzeman, J., Jacq, F., Legre, Y., Medernach, E., Montagnat, J., Maass, J., Reichstadt, M., Schwichtenberg, H., Sridhar, M., Kasam, V., Zimmermann, M., Hofmann, M., Breton, V.: Grid-enabled Virtual Screening against malaria. JGC 6, 29–43 (2008)
Juve, G., Deelman, E.: Resource Provisioning Options for Large-Scale Scientific Workflows. In: eScience’08, pp. 608–613 (December 2008)
Kacsuk, P., Farkas, Z., Fedak, G.: Towards making BOINC and EGEE interoperable. In: eScience’08, pp. 478–484 (December 2008)
Lingrand, D., Montagnat, J., Martyniak, J., Colling, D.: Analyzing the EGEE production grid workload: Application to jobs submission optimization. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2009. LNCS, vol. 5798, pp. 37–58. Springer, Heidelberg (2009)
Mosciki, J.T.: Distributed analysis environment for HEP and interdisciplinary applications. Nuclear Instruments and Methods in Physics Research A 502, 426–429 (2003)
Ngoc Minh, T., Wolters, L.: Modeling Job Arrival Process with Long Range Dependence and Burstiness Characteristics. In: CCGrid’09, pp. 324–330 (May 2009)
Sarrut, D., Guigues, L.: Region-oriented ct image representation for reducing computing time of monte carlo simulations. Med. Phys. 35(4) (2008)
Schroeder, B., Gibson, G.A.: A large-scale study of failures in high-performance computing systems. In: DSN, pp. 249–258 (2006)
Sfiligoi, I.: glideInWMS: a generic pilot-based workload management system. Journal of Physics: Conference Series 119(6) (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Glatard, T., Camarasu-Pop, S. (2010). Modelling Pilot-Job Applications on Production Grids. In: Lin, HX., et al. Euro-Par 2009 – Parallel Processing Workshops. Euro-Par 2009. Lecture Notes in Computer Science, vol 6043. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14122-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-14122-5_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14121-8
Online ISBN: 978-3-642-14122-5
eBook Packages: Computer ScienceComputer Science (R0)