Benchmark Based on Application Signature to Analyze and Predict Their Behavior

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1050)


Currently, there are benchmark sets that measure the performance of HPC systems under specific computing and communication properties. These benchmarks represent the kernels of applications that measure specific hardware components. If the user’s application is not represented by any benchmark, it is not possible to obtain an equivalent performance metric. In this work, we propose a benchmark based on the signature of an MPI application obtained by the PAS2P method. PAS2P creates the application signature in order to predict the execution time, which we believe will be very adjusted in relation to the execution time of the full application. The signature has two performance qualities: the bounded time to execute it (a benchmark property) and the quality of prediction. Therefore, we propose to extend the signature by giving the benchmark capacities such as the efficiency of the application over the HPC system. The performance metrics will be performed by the benchmark proposed. The experimentation validates our proposal with an average error of prediction close to 7%.


High Performance Computing MPI application Performance Prediction Performance metrics 



This research has been supported by the Agencia Estatal de Investigación (AEI), Spain and the Fondo Europeo de Desarrollo Regional (FEDER) UE, under contract TIN2017-84875-P and partially funded by a research collaboration agreement with the Fundacion Escuelas Universitarias Gimbernat (EUG).


  1. 1.
    Adams, M., Brown, J., Shalf, J., Van Straalen, B., Strohmaier, E., Williams, S.: HPGMG 1.0: a benchmark for ranking high performance computing systems. Technical report, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA, United States (2014)Google Scholar
  2. 2.
    Bailey, D.H., et al.: The NAS parallel benchmarks. Int. J. Supercomput. Appl. 5, 63–73 (1991). Technical reportCrossRefGoogle Scholar
  3. 3.
    Brown, P.N., Falgout, R.D., Jones, J.E.: Semicoarsening multigrid on distributed memory machines. SIAM J. Sci. Comput. 21(5), 1823–1834 (2000)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Dongarra, J.J., Luszczek, P., Petitet, A.: The LINPACK benchmark: past, present and future. Concur. Comput. Pract. Exp. 15(9), 803–820 (2003)CrossRefGoogle Scholar
  5. 5.
    Heroux, M.A., et al.: Improving performance via mini-applications. Sandia National Laboratories, Technical report SAND2009-5574, 3 (2009)Google Scholar
  6. 6.
    Heroux, M.A., Dongarra, J.: Toward a new metric for ranking high performance computing systems. Sandia National Laboratories Report, SAND2013-4744 (2013)Google Scholar
  7. 7.
    Hoisie, A., Lubeck, O., Wasserman, H.: Performance and scalability analysis of teraflop-scale parallel architectures using multidimensional wavefront applications. Int. J. High Perform. Comput. Appl. 14(4), 330–346 (2000)CrossRefGoogle Scholar
  8. 8.
    Marjanović, V., Gracia, J., Glass, C.W.: Performance modeling of the HPCG benchmark. In: Jarvis, S.A., Wright, S.A., Hammond, S.D. (eds.) PMBS 2014. LNCS, vol. 8966, pp. 172–192. Springer, Cham (2015). Scholar
  9. 9.
    McCalpin, J., Oakland, C.A.: An industry perspective on performance characterization: applications vs benchmarks. In: Proceedings of the Third Annual IEEE Workshop Workload Characterization, Keynote Address, September 2000Google Scholar
  10. 10.
    Meuer, H., Strohmaier, E., Dongarra, J., Simon, H., Meuer, M.: Top 500 list (2012)Google Scholar
  11. 11.
    Terpstra, D., Jagode, H., You, H., Dongarra, J.: Collecting performance data with PAPI-C. In: Müller, M., Resch, M., Schulz, A., Nagel, W. (eds.) Tools for High Performance Computing 2009, pp. 157–173. Springer, Heidelberg (2010). Scholar
  12. 12.
    Wong, A., Rexachs, D., Luque, E.: Parallel application signature for performance analysis and prediction. IEEE Trans. Parallel Distrib. Syst. 26(7), 2009–2019 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Departamento de Computación e IndustriasUniversidad Católica del MauleTalcaChili
  2. 2.Computer Architecture and Operating System DepartmentUniversidad Autónoma de BarcelonaBarcelonaSpain

Personalised recommendations