SPEC HPG Benchmarks for Large Systems

  • Matthias S. Müller
  • Kumaran Kalyanasundaram
  • Greg Gaertner
  • Wesley Jones
  • Rudolf Eigenmann
  • Ron Lieberman
  • Matthijs van Waveren
  • Brian Whitney
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2858)


Performance characteristics of application programs onlarge-scale systems are often significantly different from those on smaller systems. In this paper, we discuss such characteristics of the benchmark suites maintained by SPEC’s High-Performance Group (HPG). The Standard Performance Evaluation Corporation’s (SPEC) High-Performance Group (HPG) has developed a set of benchmarks for measuring performance of large-scale systems using both OpenMP and MPI parallel programming paradigms. Currently, SPEC HPG has two lines of benchmark suites: SPEC OMP and SPEC HPC2002. SPEC OMP uses the OpenMP API and includes benchmark suites intended for measuring performance of modern shared memory parallel systems. SPEC HPC2002 is based on both OpenMP and MPI, and thus it is suitable for distributed memory systems, shared memory systems, and hybrid systems. SPEC HPC2002 contains benchmarks from three popular application areas, Chemistry, Seismic, and Weather Forecasting. Each of the three benchmarks in HPC2002 has small and medium data sets in order to satisfy the need for benchmarking a wide range of high-performance systems. We present our experiences regarding the scalability of the benchmark suites. We also analyze published results of these benchmark suites based on application program behavior and systems’ architectural features.


SPEC OMP SPEC HPC Benchmark High-Performance Computing Performance Evaluation OpenMP MPI 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aslot, V., Domeika, M., Eigenmann, R., Gaertner, G., Jones, W.B., Parady, B.: SPEComp: a new benchmark suite for measuring parallel computer performance. In: Eigenmann, R., Voss, M.J. (eds.) WOMPAT 2001. LNCS, vol. 2104, pp. 1–10. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  2. 2.
    Aslot, V., Eigenmann, R.: Performance characteristics of the SPEC OMP2001 benchmarks. In: 3rd European Workshop on OpenMP, EWOMP 2001, Barcelona, Spain (September 2001)Google Scholar
  3. 3.
    Bao, H., Bielak, J., Ghattas, O., Kallivokas, L.F., O’Hallaron, D.R., Shewchuk, J.R., Xu, J.: Large-scale simulation of elastic wave propagation in heterogeneous media on parallel computers. Computer Methods in Applied Mechanics and Engineering 152(1-2), 85–102 (1998)zbMATHCrossRefGoogle Scholar
  4. 4.
    Barszcz, E., Fatoohi, R., Venkatkrishnan, V., Weeratunga, S.: Solution of regular sparse triangular systems on vector and distributed-memory multiprocessors. Technical Report Rept. No: RNR-93-007, NASA Ames Research Center (1993)Google Scholar
  5. 5.
    Domeika, M.J., Roberson, C.W., Page, E.W., Tagliarini, G.A.: Adaptive resonance theory 2 neural network approach to star field recognition. In: Rogers, S.K., Ruck, D.W. (eds.) Proc. SPIE Applications and Science of Artificial Neural Networks II, vol. 2760, International Society for Optical Engineering, pp. 589–596 (1996)Google Scholar
  6. 6.
    Iwashita, H., Yamanaka, E., Sueyasu, N., van Waveren, M., Miura, K.: The SPEC OMP 2001 benchmark on the Fujitsu PRIMEPOWER system. In: 3rd European Workshop on OpenMP, EWOMP 2001, Barcelona, Spain (September 2001)Google Scholar
  7. 7.
    Key, S.W., Hoff, C.C.: An improved constant membrane and bending stress shell element for explicit transient dynamics. Computer Methods in Applied Mechanics and Engineering 124, 33–47 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    OpenMP Architecture Review Board. OpenMP C and C++ Application Programming Interface Version 2.0 (March 2002),
  9. 9.
    OpenMP Architecture Review Board. OpenMP Fortran Application Programming Interface Version 2.0 (November 2000),
  10. 10.
    Saito, H., Gaertner, G., Jones, W., Eigenmann, R., Iwashita, H., Lieberman, R., van Waveren, M., Whitney, B.: Large system performance of SPEC OMP benchmark suites. Int’l Journal of Parallel Programming 31(3) (June 2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Matthias S. Müller
    • 1
  • Kumaran Kalyanasundaram
    • 2
  • Greg Gaertner
    • 3
  • Wesley Jones
    • 4
  • Rudolf Eigenmann
    • 5
  • Ron Lieberman
    • 6
  • Matthijs van Waveren
    • 7
  • Brian Whitney
    • 8
  1. 1.High Performance Computing Center Stuttgart (HLRS)StuttgartGermany
  2. 2.SPEC HPG ChairSilicon Graphics Inc. 
  3. 3.SPEC HPG Vice ChairHewlett-Packard Corporation 
  4. 4.National Renewable Energy Lab 
  5. 5.Purdue University 
  6. 6.Hewlett-Packard Corporation 
  7. 7.Fujitsu Systems Europe Ltd. 
  8. 8.Sun Microsystems 

Personalised recommendations