Determine energy-saving potential in wait-states of large-scale parallel programs


Energy consumption is one of the major topics in high performance computing (HPC) in the last years. However, little effort is put into energy analysis by developers of HPC applications.

We present our approach of combined performance and energy analysis using the performance analysis tool-set Scalasca. Scalascas parallel wait-state analysis is extended by a calculation of the energy-saving potential if a lower power-state can be used.


  1. 1.

    Dong Y, Chen J, Yang X, Yang C, Peng L (2008) Low power optimization for MPI collective operations. In: Young computer scientists, International conference for, pp 1047–1052

    Google Scholar 

  2. 2.

    Feng X, Ge R, Cameron KW (2005) Power and energy profiling of scientific applications on distributed systems. In: Proceedings of the 19th IEEE international parallel and distributed processing symposium, IPDPS’05. IEEE Computer Society, Washington, vol 1, p 34. doi:10.1109/IPDPS.2005.346

    Google Scholar 

  3. 3.

    Freeh VW, Pan F, Kappiah N, Lowenthal DK, Springer R (2005) Exploring the energy-time tradeoff in MPI programs on a power-scalable cluster. In: Proceedings of the 19th IEEE international parallel and distributed processing symposium, IPDPS’05, vol 1. IEEE Computer Society, Washington. doi:10.1109/IPDPS.2005.214

    Google Scholar 

  4. 4.

    Ge R, Feng X, Cameron KW (2005) Performance-constrained distributed DVS scheduling for scientific applications on power-aware clusters. In: Proceedings of the 2005 ACM/IEEE conference on supercomputing, SC’05. IEEE Computer Society, Washington, p 34. doi:10.1109/SC.2005.57

    Google Scholar 

  5. 5.

    Ge R, Feng X, Song S, Chang HC, Li D, Cameron KW (2009) PowerPack: energy profiling and analysis of high-performance systems and applications. IEEE Trans Parallel Distrib Syst 99:658–671

    Google Scholar 

  6. 6.

    Geimer M, Wolf F, Wylie BJN, Abraham E, Becker D, Mohr B (2010) The Scalasca performance toolset architecture. Concurr Comput: Pract Exp 22(6):277–288

    Google Scholar 

  7. 7.

    Gibbon P (2003) PEPC: Pretty efficient parallel Coulomb-solver. Forschungszentrum Jülich

  8. 8.

    Hsu CH, Feng WC (2005) A power-aware run-time system for high-performance computing. In: Proceedings of the 2005 ACM/IEEE conference on Supercomputing, SC’05. IEEE Computer Society, Washington, p 1. doi:10.1109/SC.2005.3

    Google Scholar 

  9. 9.

    Huang S, Feng W (2009) Energy-efficient cluster computing via accurate workload characterization. In: Proceedings of the 2009 9th IEEE/ACM international symposium on cluster computing and the grid, CCGRID ’09. IEEE Computer Society, Washington, pp 68–75. doi:10.1109/CCGRID.2009.88

    Google Scholar 

  10. 10.

    Kappiah N, Freeh VW, Lowenthal DK (2005) Just in time dynamic voltage scaling: Exploiting inter-node slack to save energy in MPI programs. In: Proceedings of the 2005 ACM/IEEE conference on Supercomputing, SC’05. IEEE Computer Society, Washington, p 33. doi:10.1109/SC.2005.39

    Google Scholar 

  11. 11.

    Knüpfer A, Brunst H, Doleschal J, Jurenz M, Lieber M, Mickler H, Müller MS, Nagel WE (2008) The Vampir performance analysis tool-set. In: Tools for high performance computing, Proceedings of the 2nd international workshop on parallel tools. Springer, Berlin, pp 139–155

    Google Scholar 

  12. 12.

    Lim MY, Freeh VW, Lowenthal DK (2006) Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs. In: Proceedings of the 2006 ACM/IEEE conference on Supercomputing, SC’06. ACM, New York. doi:10.1145/1188455.1188567

    Google Scholar 

  13. 13.

    Lu YH, Benini L, De Micheli G (2000) Operating-system directed power reduction. In: Proceedings of the 2000 international symposium on low power electronics and design, ISLPED ’00. ACM, New York, pp 37–42. doi:10.1145/344166.344189

    Google Scholar 

  14. 14.

    Minartz T, Kunkel J, Ludwig T (2010) Simulation of power consumption of energy efficient cluster hardware. Comput Sci Res Dev 25:165–175

    Article  Google Scholar 

  15. 15.

    Minartz T, Knobloch M, Ludwig T, Mohr B (2011) Managing hardware power saving modes for high performance computing In: Proceedings of the 2nd international green computing Conference (to appear)

  16. 16.

    Minartz T, Molka D, Knobloch M, Krempel S, Ludwig T, Nagel W, Mohr B, Falter H (2011) eeClust—energy-efficient cluster computing. In proceedings of the CiHPC: competence in high performance computing, HPC status konferenz der Gauß-Allianz e.V. (to appear)

  17. 17.

    Nikolopoulos DS (2009) Green building blocks—software stacks for energy-efficient clusters and data centers. ERCIM News 79

  18. 18.

    Pinheiro E, Bianchini R, Carrera EV, Heath T (2001) Load balancing and unbalancing for power and performance in cluster-based systems. In: Workshop on compilers and operating systems for low power.

    Google Scholar 

  19. 19.

    Snowdon DC, Petters SM, Heiser G (2007) Accurate on-line prediction of processor and memory energy usage under voltage scaling. In: Proceedings of the 7th ACM & IEEE international conference on embedded software, EMSOFT’07, pp 84–93

    Google Scholar 

  20. 20.

    Zamani R, Afsahi A (2010) Adaptive estimation and prediction of power and performance in high performance computing. Comput Sci Res Dev 25:177–186

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Michael Knobloch.

Additional information

This project is funded by the BMBF (German federal ministery for education and science) under grant 01—H08008E within the call: “HPC-Software für skalierbare Parallelrechner”.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and Permissions

About this article

Cite this article

Knobloch, M., Mohr, B. & Minartz, T. Determine energy-saving potential in wait-states of large-scale parallel programs. Comput Sci Res Dev 27, 255–263 (2012).

Download citation


  • Power consumption
  • Energy efficiency
  • Energy
  • Performance
  • Analysis
  • Scalasca
  • MPI