Abstract
Grid computing is becoming a common platform for solving large scale computing tasks. However, a number of major technical issues, including the lack of adequate performance evaluation approaches, hinder the grid computing’s further development. The requirements herefore are manifold; adequate approaches must combine appropriate performance metrics, realistic workload models, and flexible tools for workload generation, submission, and analysis. In this paper we present an approach to tackle this complex problem. First, we introduce a set of grid performance objectives based on traditional and grid-specific performance metrics. Second, we synthesize the requirements for realistic grid workload modeling, e.g. co-allocation, data and network management, and failure modeling. Third, we show how GrenchMark, an existing framework for generating, running, and analyzing grid workloads, can be extended to implement the proposed modeling techniques. Our approach aims to be an initial and necessary step towards a common performance evaluation framework for grid environments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
The Parallel Workloads Archive Team. The parallel workloads archive logs (June 2006), Available at http://www.cs.huji.ac.il/labs/parallel/workload/logs.html
Berman, F., et al.: Adaptive computing on the grid using apples. IEEE Trans. Parallel Distrib. Syst. 14(4), 369–382 (2003)
Beume, N., et al.: Scheduling Algorithm Development based on Complex Owner Defined Objectives. Technical Report CI-190/05, University of Dortmund (January 2005)
Bucur, A.I.D., Epema, D.H.J.: The performance of processor co-allocation in multicluster systems. In: Proc. of the 3rd IEEE Int’l. Symp. on Cluster Computing and the Grid (CCGrid), pp. 302–309. IEEE Computer Society Press, Los Alamitos (2003)
Bucur, A.I.D., Epema, D.H.J.: Trace-based simulations of processor co-allocation policies in multiclusters. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 70–79 (2003)
Buyya, R., Abramson, D., Venugopal, S.: The grid economy. In: Special Issue of the Proceedings of the IEEE on Grid Computing. IEEE Press, Los Alamitos (To appear, 2005)
Calzarossa, M., Serazzi, G.: A characterization of the variation in time of workload arrival patterns. IEEE Trans. Comput. 34(2), 156–162 (1985)
Chapin, S.J., et al.: Benchmarks and standards for the evaluation of parallel job schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1999, IPPS-WS 1999, and SPDP-WS 1999. LNCS, vol. 1659, pp. 67–90. Springer, Heidelberg (1999)
Chun, B.N., et al.: Mirage: A microeconomic resource allocation system for sensornet testbeds. In: Proc. of 2nd IEEE Workshop on Embedded Networked Sensors (EmNetsII), IEEE Computer Society Press, Los Alamitos (2005)
Chun, G., et al.: Benchmark probes for grid assessment. In: Proc. of the 18th International Parallel and Distributed Processing Symposium (IPDPS) (2004)
Cirne, W., Berman, F.: A Comprehensive Model of the Supercomputer Workload. In: 4th Workshop on Workload Characterization (December 2001)
Cirne, W., Berman, F.: A model for moldable supercomputer jobs. In: Proc. of the 15th International Parallel and Distributed Processing Symposium (IPDPS), pp. 59–79 (2001)
Downey, A.B.: A parallel workload model and its implications for processor allocation. Cluster Computing 1(1), 133–145 (1998)
Dumitrescu, C., Raicu, I., Foster, I.T.: Experiences in running workloads over grid3. In: Zhuge, H., Fox, G.C. (eds.) GCC 2005. LNCS, vol. 3795, pp. 274–286. Springer, Heidelberg (2005)
Ernemann, C., Yahyapour, R.: Applying Economic Scheduling Methods to Grid Environments. In: Ernemann, C., Yahyapour, R. (eds.) Grid Resource Management - State of the Art and Future Trends, pp. 491–506. Kluwer Academic Publishers, Dordrecht (2003)
Ernemann, C., et al.: On advantages of grid computing for parallel job scheduling. In: Proc. of the 2nd IEEE Int’l. Symp. on Cluster Computing and the Grid (CCGrid), pp. 39–49. IEEE Computer Society Press, Los Alamitos (2002)
Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global grid computing for job scheduling. In: Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, Pittsburgh, November 2004, IEEE Computer Society, Los Alamitos (2004)
Ernemann, C., Song, B., Yahyapour, R.: Scaling of Workload Traces. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 166–183. Springer, Heidelberg (2003)
Feitelson, D.G.: Packing Schemes for Gang Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1162, pp. 89–110. Springer, Heidelberg (1996)
Feitelson, D.G.: The forgotten factor: facts on performance evaluation and its dependence on workloads. In: Monien, B., Feldmann, R.L. (eds.) Euro-Par 2002. LNCS, vol. 2400, pp. 49–60. Springer, Heidelberg (2002)
Feitelson, D.G.: Workload Modeling for Performance Evaluation. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 114–141. Springer, Heidelberg (2002)
Feitelson, D.G.: Metric and workload effects on computer systems evaluation. IEEE Computer 36(9), 18–25 (2003)
Feitelson, D.G.: Experimental analysis of the root causes of performance evaluation results: a backfilling case study. IEEE Transactions on Parallel and Distributed Systems 16(2), 175–182 (2005)
Feitelson, D.G., Rudolph, L.: Metrics and benchmarking for parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1459, pp. 1–24. Springer, Heidelberg (1998)
Feitelson, D.G., et al.: Theory and Practice in Parallel Job Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)
Foster, I., et al.: End-to-end quality of service for high-end applications. Computer Communications 27(14), 1375–1388 (2004)
Frumkin, M.A., Van der Wijngaart, R.F.: Nas grid benchmarks: A tool for grid space exploration. In: Proc. of the 10th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 315–326 (2001)
Graham, R.L., et al.: Optimization and approximation in deterministic sequencing and scheduling: A survey. Annals of Discrete Mathematics 15, 287–326 (1979)
Hwang, S., Kesselman, C.: Gridworkflow: A flexible failure handling framework for the grid. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 126–137 (2003)
Iosup, A., et al.: How are real grids used? the analysis of four grid traces and its implications. In: The 7th IEEE/ACM International Conference on Grid Computing (Grid), Barcelona, ES, Sept. 28-29 (accepted, 2006)
Iosup, A., Epema, D.H.J.: GrenchMark: A framework for analyzing, testing, and comparing grids. In: Proc. of the 6th IEEE/ACM Int’l. Symp. on Cluster Computing and the GRID (CCGrid), May (accepted, 2006)
Iosup, A., et al.: Synthetic grid workloads with Ibis, KOALA, and GrenchMark. In: Dumke, R.R., Abran, A. (eds.) IWSM 2000. LNCS, vol. 2006, Springer, Heidelberg (2001)
Jackson, D., Snell, Q., Clement, M.: Core algorithms of the Maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)
Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling (Winner of “1991 Best Advanced How-To Book, Systems” award from the Computer Press Association). Wiley-Interscience, New York (May 1991)
Kenyon, C., Cheliotis, G.: Architecture requirements for commercializing grid resources. In: Proc. of the 11th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 215–224 (2002)
Kola, G., Kosar, T., Livny, M.: Phoenix: Making data-intensive grid applications fault-tolerant. In: Buyya, R. (ed.) GRID, pp. 251–258. IEEE Computer Society Press, Los Alamitos (2004)
Kondo, D., et al.: Characterizing and evaluating desktop grids: An empirical study. In: Proc. of the 18th International Parallel and Distributed Processing Symposium (IPDPS) (2004)
Li, H., Groep, D., Wolters, L.: Workload characteristics of a multi-cluster supercomputer. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 176–194. Springer, Heidelberg (2005)
Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing 63(20), 1105–1122 (2003)
Medeiros, R., et al.: Faults in grids: Why are they so bad and what can be done about it? In: Stockinger, H. (ed.) GRID, pp. 18–24. IEEE Computer Society Press, Los Alamitos (2003)
Medernach, E.: Workload analysis of a cluster in a grid environment. In: Feitelson, D.G., et al. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 36–61. Springer, Heidelberg (2005)
Mohamed, H.H., Epema, D.H.J.: Experiences with the koala co-allocating scheduler in multiclusters. In: Proc. of the 5th IEEE/ACM Int’l Symp. on Cluster Computing and the GRID (CCGrid), May 2005, ACM Press, New York (2005)
Pinedo, M.: Scheduling: Theory, Algorithms, and Systems, 2nd edn. Prentice-Hall, Englewood Cliffs (2002)
Schroeder, W., et al.: Analysis of HPSS Performance Based on Per-file Transfer Logs. In: Proc. of the 16th IEEE Mass Storage Systems Symposium, San Diego, March 1999, pp. 103–115. IEEE Computer Society Press, Los Alamitos (1999)
Schwiegelshohn, U., Yahyapour, R.: Fairness in parallel job scheduling. Journal of Scheduling 3(5), 297–320 (2000)
Shan, H., Oliker, L., Biswas, R.: Job superscheduler architecture and performance in computational grid environments. In: SC, pp. 44–54. ACM Press, New York (2003)
Song, B., Ernemann, C., Yahyapour, R.: Parallel Computer Workload Modeling with Markov Chains. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 47–62. Springer, Heidelberg (2005)
Song, B., Ernemann, C., Yahyapour, R.: User Group-based Workload Analysis and Modeling. In: Proc. of the 5th Int’l. Symp. on Cluster Computing and the Grid (CCGrid), IEEE Computer Society Press, Los Alamitos (2005)
Thain, D., et al.: Pipeline and batch sharing in grid workloads. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 152–161 (2003)
Theys, M.D., et al.: A Mathematical Model, Heuristic and Simulation Study for a Basic Data Staging Problem in a Heterogenous Networking Environment. In: Proc. of the 7th Heterogeneous Computing Workshop, Orlando, March 1998, pp. 115–122. IEEE, Los Alamitos (1998)
Tsouloupas, G., Dikaiakos, M.D.: GridBench: A workbench for grid benchmarking. In: Sloot, P.M.A., et al. (eds.) EGC 2005. LNCS, vol. 3470, pp. 211–225. Springer, Heidelberg (2005)
Wolski, R., Spring, N., Hayes, J.: The network weather service: A distributed resource performance forecasting service for metacomputing. Journal of Future Generation Computing Systems 15(5-6), 757–768 (1999)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Iosup, A. et al. (2007). On Grid Performance Evaluation Using Synthetic Workloads. In: Frachtenberg, E., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2006. Lecture Notes in Computer Science, vol 4376. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71035-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-540-71035-6_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71034-9
Online ISBN: 978-3-540-71035-6
eBook Packages: Computer ScienceComputer Science (R0)