Abstract
The behavior of a parallel application can be presented in many ways, but performance visualization tools usually focus on communication graphs and runtime of processes or threads in specific (groups of) functions. A different approach is required when searching for the optimal configuration of tunable parameters, for which it is necessary to run the application several times and compare the resource consumption of these runs. We present RADAR visualizer, a tool that was originally developed to analyze such measurements and to detect the optimal configuration for each instrumented part of the code. In this case, the optimum was defined as the minimum energy consumption of the whole application, but any other metric can be defined.
RADAR visualizer presents the application behavior in several graphical representations and tables including the amount of savings that can be reached. Together with our MERIC library, we provide a complete toolchain for HPC application behavior monitoring, data analysis, and graphical representation. The final part is performing dynamic tuning (applying optimal settings for each region during the application runtime) for the production runs of the analyzed application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
yEd Graph editor can be download from https://www.yworks.com/yed.
- 2.
By default, MERIC does not store the power samples because it creates a much larger output. Both HDEEM and DiG work normally on a 1 kHz sampling frequency for the blade, which means a thousand entries per second of the measurement. HDEEM also has sensors on specific parts of the node (e.g. each CPU or each memory channel) and measures them on 100 Hz sampling frequency, so in cases where these samples are also included, the output size rises accordingly.
References
Calore, E., Gabbana, A., Schifano, S.F., Tripiccione, R.: Evaluation of DVFS techniques on modern HPC processors and accelerators for energy aware applications. Concurr. Comput. Pract. Exp. 29(12), e4143 (2017). https://doi.org/10.1002/cpe.4143
Calore, E., Gabbana, A., Schifano, S.F., Tripiccione, R.: Energy-efficiency tuning of a lattice Boltzmann simulation using MERIC. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K. (eds.) PPAM 2019. LNCS, vol. 12044, pp. 169–180. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43222-5_15
Cesarini, D., Bartolini, A., Bonfà, P., Cavazzoni, C., Benini, L.: COUNTDOWN - three, two, one, low power! A run-time library for energy saving in MPI communication primitives (2018). CoRR abs/1806.07258. http://arxiv.org/abs/1806.07258
Eastep, J., et al.: Global extensible open power manager: a vehicle for HPC community collaboration on co-designed energy management solutions. In: Kunkel, J.M., Yokota, R., Balaji, P., Keyes, D. (eds.) ISC 2017. LNCS, vol. 10266, pp. 394–412. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58667-0_21
Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD1996, pp. 226–231. AAAI Press (1996)
Geimer, M., Wolf, F., Wylie, B.J.N., Ábrahám, E., Becker, D., Mohr, B.: The SCALASCA performance toolset architecture. Concurr. Comput. Pract. Exper. 22(6), 702–719 (2010). https://doi.org/10.1002/cpe.v22:6
Hackenberg, D., Ilsche, T., Schuchart, J., Schöne, R., Nagel, W.E., Simon, M., Georgiou, Y.: HDEEM: High definition energy efficiency monitoring. In: 2014 Energy Efficient Supercomputing Workshop, pp. 1–10 (2014).https://doi.org/10.1109/E2SC.2014.13
Hähnel, M., Döbel, B., Völp, M., Härtig, H.: Measuring energy consumption for short code paths using RAPL. SIGMETRICS Perform. Eval. Rev. 40(3), 13–17 (2012). https://doi.org/10.1145/2425248.2425252
IT4Innovations: MERIC library. https://code.it4i.cz/vys0053/meric. Accessed 21 Apr 2019
IT4Innovations: READEX RADAR library. https://code.it4i.cz/bes0030/readex-radar. Accessed 21 Aug 2019
Knüpfer, A., et al.: Score-p: a joint performance measurement run-time infrastructure for periscope, SCALASCA, TAU, and VAMPIR. In: Brunst, H., Müller, M.S., Nagel, W.E., Resch, M.M. (eds.) Tools for High Performance Computing 2011, pp. 79–91. Springer, Berlin Heidelberg (2012)
Libri, A., Bartolini, A., Benini, L.: Dwarf in a giant: Enabling scalable, high-resolution HPC energy monitoring for real-time profiling and analytics (2018). CoRR abs/1806.02698: http://arxiv.org/abs/1806.02698
Persson, P.O., Strang, G.: Smoothing by Savitzky-Golay and Legendre filters. In: Rosenthal, J., Gilliam, D.S. (eds.) Mathematical Systems Theory in Biology, Communications, Computation, and Finance, pp. 301–315. Springer, New York, New York, NY (2003)
READEX: Horizon 2020 READEX project (2018). https://www.readex.eu
Riha, L., et al.: A massively parallel and memory-efficient fem toolbox with a hybrid total FETI solver with accelerator support. Int. J. High Perform. Comput. Appl. 0(0), 1094342018798452 (0). https://doi.org/10.1177/1094342018798452
Rountree, B., Lowenthal, D.K., de Supinski, B.R., Schulz, M., Freeh, V.W., Bletsch, T.K.: Adagio: making DVS practical for complex HPC applications. In: Proceedings of the 23rd International Conference on Supercomputing, ICS 2009, pp. 460–469. ACM, New York, NY, USA (2009). https://doi.org/10.1145/1542275.1542340
Schuchart, J., et al.: The READEX formalism for automatic tuning for energy efficiency. Computing 99(8), 727–745 (2017). https://doi.org/10.1007/s00607-016-0532-7
Schulz, M., Galarowicz, J., Maghrak, D., Hachfeld, W., Montoya, D., Cranford, S.: Open Speedshop: an open source infrastructure for parallel performance analysis. Sci. Program. 16(2–3), 105–121 (2008). https://doi.org/10.1155/2008/713705
Servat, H., Llort, G., Huck, K., Giménez, J., Labarta, J.: Framework for a productive performance optimization. Parallel Comput. 39(8), 336–353 (2013). https://doi.org/10.1016/j.parco.2013.05.004
Shende, S.S., Malony, A.D.: The TAU parallel performance system. Int. J. High Perform. Comput. Appl. 20(2), 287–311 (2006). https://doi.org/10.1177/1094342006064482
Vysocky, O., et al.: Evaluation of the HPC applications dynamic behavior in terms of energy consumption. In: Proceedings of the Fifth International Conference on Parallel, Distributed, Grid and Cloud Computing for Engineering, pp. 1–19 (2017), paper 3, 2017. https://doi.org/10.4203/ccp.111.3
Vysocky, O., Riha, L., Zapletal, J.: A simple framework for energy efficiency evaluation and hardware parameter tuning with modular support for different HPC platforms. In: Proceedings of the Eighth International Conference on Advanced Communications and Computation, pp. 25–30. IARIA (2018). http://www.thinkmind.org/index.php?view=article&articleid=infocomp_2018_2_10_68005
Acknowledgment
This work was supported by The Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project IT4Innovations National Supercomputing Center LM2015070.
This work was supported by The Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project ,,e-Infrastructure CZ - LM2018140".
This work was supported by the Moravian-Silesian Region from the programme “Support of science and research in the Moravian-Silesian Region 2017” (RRC/10/2017).
This work was partially supported by the SGC grant No. SP2019/59 “Infrastructure research and development of HPC libraries and tools”, VŠB - Technical University of Ostrava, Czech Republic.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vysocky, O., Peterek, I., Beseda, M., Spetko, M., Ulcak, D., Riha, L. (2021). Analysis and Visualization of the Dynamic Behavior of HPC Applications. In: Kozubek, T., Arbenz, P., Jaroš, J., Říha, L., Šístek, J., Tichý, P. (eds) High Performance Computing in Science and Engineering. HPCSE 2019. Lecture Notes in Computer Science(), vol 12456. Springer, Cham. https://doi.org/10.1007/978-3-030-67077-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-67077-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67076-4
Online ISBN: 978-3-030-67077-1
eBook Packages: Computer ScienceComputer Science (R0)