SPEC HPG Benchmarks for Large Systems
Performance characteristics of application programs onlarge-scale systems are often significantly different from those on smaller systems. In this paper, we discuss such characteristics of the benchmark suites maintained by SPEC’s High-Performance Group (HPG). The Standard Performance Evaluation Corporation’s (SPEC) High-Performance Group (HPG) has developed a set of benchmarks for measuring performance of large-scale systems using both OpenMP and MPI parallel programming paradigms. Currently, SPEC HPG has two lines of benchmark suites: SPEC OMP and SPEC HPC2002. SPEC OMP uses the OpenMP API and includes benchmark suites intended for measuring performance of modern shared memory parallel systems. SPEC HPC2002 is based on both OpenMP and MPI, and thus it is suitable for distributed memory systems, shared memory systems, and hybrid systems. SPEC HPC2002 contains benchmarks from three popular application areas, Chemistry, Seismic, and Weather Forecasting. Each of the three benchmarks in HPC2002 has small and medium data sets in order to satisfy the need for benchmarking a wide range of high-performance systems. We present our experiences regarding the scalability of the benchmark suites. We also analyze published results of these benchmark suites based on application program behavior and systems’ architectural features.
KeywordsSPEC OMP SPEC HPC Benchmark High-Performance Computing Performance Evaluation OpenMP MPI
Unable to display preview. Download preview PDF.
- 2.Aslot, V., Eigenmann, R.: Performance characteristics of the SPEC OMP2001 benchmarks. In: 3rd European Workshop on OpenMP, EWOMP 2001, Barcelona, Spain (September 2001)Google Scholar
- 4.Barszcz, E., Fatoohi, R., Venkatkrishnan, V., Weeratunga, S.: Solution of regular sparse triangular systems on vector and distributed-memory multiprocessors. Technical Report Rept. No: RNR-93-007, NASA Ames Research Center (1993)Google Scholar
- 5.Domeika, M.J., Roberson, C.W., Page, E.W., Tagliarini, G.A.: Adaptive resonance theory 2 neural network approach to star field recognition. In: Rogers, S.K., Ruck, D.W. (eds.) Proc. SPIE Applications and Science of Artificial Neural Networks II, vol. 2760, International Society for Optical Engineering, pp. 589–596 (1996)Google Scholar
- 6.Iwashita, H., Yamanaka, E., Sueyasu, N., van Waveren, M., Miura, K.: The SPEC OMP 2001 benchmark on the Fujitsu PRIMEPOWER system. In: 3rd European Workshop on OpenMP, EWOMP 2001, Barcelona, Spain (September 2001)Google Scholar
- 8.OpenMP Architecture Review Board. OpenMP C and C++ Application Programming Interface Version 2.0 (March 2002), http://www.openmp.org/specs/mp-documents/cspec20.pdf
- 9.OpenMP Architecture Review Board. OpenMP Fortran Application Programming Interface Version 2.0 (November 2000), http://www.openmp.org/specs/mp-documents/fspec20.pdf
- 10.Saito, H., Gaertner, G., Jones, W., Eigenmann, R., Iwashita, H., Lieberman, R., van Waveren, M., Whitney, B.: Large system performance of SPEC OMP benchmark suites. Int’l Journal of Parallel Programming 31(3) (June 2003)Google Scholar