Advertisement

Hybrid Supercomputer Desmos with Torus Angara Interconnect: Efficiency Analysis and Optimization

  • Nikolay Kondratyuk
  • Grigory Smirnov
  • Ekaterina Dlinnova
  • Sergey Biryukov
  • Vladimir StegailovEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 910)

Abstract

The paper describes the first experience of practical deployment of the hybrid supercomputer Desmos at the Joint Institute for High Temperatures of the Russian Academy of Sciences (JIHT RAS). We consider job scheduling statistics, energy efficiency, case studies of GPU acceleration efficiency and benchmarks of the distributed storage with a parallel file system.

Keywords

Job accounting and statistics Energy efficiency GPU acceleration Parallel I/O 

References

  1. 1.
    Stegailov, V., et al.: Early performance evaluation of the hybrid cluster with torus interconnect aimed at molecular-dynamics simulations. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds.) PPAM 2017 Part I. LNCS, vol. 10777, pp. 327–336. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-78024-5_29CrossRefGoogle Scholar
  2. 2.
    Vecher, V.S., Kondratyuk, N.D., Smirnov, G.S., Stegailov, V.V.: Angara-based hybrid supercomputer for efficient acceleration of computational materials science studies. In: Proceeding of International Conference Russian Supercomputing Days 2017, pp. 557–571 (2017)Google Scholar
  3. 3.
    Neuwirth, S., Frey, D., Nuessle, M., Bruening, U.: Scalable communication architecture for network-attached accelerators. In: 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pp. 627–638 (2015).  https://doi.org/10.1109/HPCA.2015.7056068
  4. 4.
    Puente, V., Beivide, R., Gregorio, J.A., Prellezo, J.M., Duato, J., Izu, C.: Adaptive bubble router: a design to improve performance in torus networks. In: Proceedings of the 1999 International Conference on Parallel Processing, pp. 58–67 (1999).  https://doi.org/10.1109/ICPP.1999.797388
  5. 5.
    Scott, S.L., Thorson, G.M.: The Cray T3E network: adaptive routing in a high performance 3D torus. In: HOT Interconnects IV. Stanford University, 15–16 August 1996 (1996)Google Scholar
  6. 6.
    Adiga, N.R., et al.: Blue Gene/L torus interconnection network. IBM J. Res. Dev. 49(2), 265–276 (2005).  https://doi.org/10.1147/rd.492.0265CrossRefGoogle Scholar
  7. 7.
    Gómez-Martín, C., Vega-Rodríguez, M.A., González-Sánchez, J.L.: Fattened backfilling: an improved strategy for job scheduling in parallel systems. J. Parallel Distrib. Comput. 97(Suppl. C), 69–77 (2016).  https://doi.org/10.1016/j.jpdc.2016.06.013CrossRefGoogle Scholar
  8. 8.
    Kraemer, A., Maziero, C., Richard, O., Trystram, D.: Reducing the number of response time SLO violations by a Cloud-HPC convergence scheduler. In: 2016 2nd International Conference on Cloud Computing Technologies and Applications (CloudTech), pp. 293–300 (2016).  https://doi.org/10.1109/CloudTech.2016.7847712
  9. 9.
    Mamaeva, A.A., Voevodin, V.V.: Methods for statistical analysis of large supercomputer job flow. In: Proceeding of International Conference Russian Supercomputing Days 2017, pp. 788–799 (2017)Google Scholar
  10. 10.
    Hoefler, T., Belli, R.: Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015, pp. 73:1–73:12. ACM, New York (2015).  https://doi.org/10.1145/2807591.2807644
  11. 11.
    Scogland, T., Azose, J., Rohr, D., Rivoire, S., Bates, N., Hackenberg, D.: Node variability in large-scale power measurements: perspectives from the Green500, Top500 and EEHPCWG. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015, pp. 74:1–74:11. ACM, New York (2015).  https://doi.org/10.1145/2807591.2807653
  12. 12.
    Höhnerbach, M., Ismail, A.E., Bientinesi, P.: The vectorization of the Tersoff multi-body potential: an exercise in performance portability. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016, pp. 7:1–7:13. IEEE Press, Piscataway (2016).  https://doi.org/10.1109/SC.2016.6
  13. 13.
    Kutzner, C., Pall, S., Fechner, M., Esztermann, A., de Groot, B.L., Grubmuller, H.: Best bang for your buck: GPU nodes for gromacs biomolecular simulations. J. Comput. Chemis. 36(26), 1990–2008 (2015).  https://doi.org/10.1002/jcc.24030CrossRefGoogle Scholar
  14. 14.
    Luehr, N., Ufimtsev, I.S., Martínez, T.J.: Dynamic precision for electron repulsion integral evaluation on graphical processing units (GPUs). J. Chem. Theory Comput. 7(4), 949–954 (2011).  https://doi.org/10.1021/ct100701wCrossRefGoogle Scholar
  15. 15.
    Mills, N., Alex Feltus, F., Ligon III, W.B.: Maximizing the performance of scientific data transfer by optimizing the interface between parallel file systems and advanced research networks. Futur. Gener. Comput. Syst. 79(Part 1), 190–198 (2018).  https://doi.org/10.1016/j.future.2017.04.030CrossRefGoogle Scholar
  16. 16.
    Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003).  https://doi.org/10.1007/10968987_3CrossRefGoogle Scholar
  17. 17.
    Vecher, V., Nikolskii, V., Stegailov, V.: GPU-accelerated molecular dynamics: energy consumption and performance. In: Voevodin, V., Sobolev, S. (eds.) RuSCDays 2016. CCIS, vol. 687, pp. 78–90. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-55669-7_7CrossRefGoogle Scholar
  18. 18.
    Stegailov, V., Vecher, V.: Efficiency analysis of intel and AMD x86\(\_\)64 architectures for Ab initio calculations: a case study of VASP. In: Voevodin, V., Sobolev, S. (eds.) RuSCDays 2017. CCIS, vol. 793, pp. 430–441. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-71255-0_35CrossRefGoogle Scholar
  19. 19.
    Stegailov, V., Vecher, V.: Efficiency analysis of Intel, AMD and Nvidia 64-Bit hardware for memory-bound problems: a case study of Ab Initio calculations with VASP. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds.) PPAM 2017 Part II. LNCS, vol. 10778, pp. 81–90. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-78054-2_8CrossRefGoogle Scholar
  20. 20.
    Smirnov, G.S., Stegailov, V.V.: Anomalous diffusion of guest molecules in hydrogen gas hydrates. High Temp. 53(6), 829–836 (2015).  https://doi.org/10.1134/S0018151X15060188CrossRefGoogle Scholar
  21. 21.
    Orekhov, N.D., Stegailov, V.V.: Simulation of the adhesion properties of the Polyethylene/Carbon nanotube interface. Polym. Sci. Ser. A 58(3), 476–486 (2016).  https://doi.org/10.1134/S0965545X16030135CrossRefGoogle Scholar
  22. 22.
    Pavlov, S.V., Kislenko, S.A.: Effects of carbon surface topography on the electrode/electrolyte interface structure and relevance to li-air batteries. Phys. Chem. Chem. Phys. 18, 30830–30836 (2016).  https://doi.org/10.1039/C6CP05552DCrossRefGoogle Scholar
  23. 23.
    Antropov, A.S., Fidanyan, K.S., Stegailov, V.V.: Phonon density of states for solid uranium: accuracy of the embedded atom model classical interatomic potential. J. Phys.: Conf. Ser. 946(012094), 94 (2018).  https://doi.org/10.1088/1742-6596/946/1/012094CrossRefGoogle Scholar
  24. 24.
    Logunov, M.A., Orekhov, N.D.: Molecular dynamics study of cavitation in carbon nanotube reinforced polyethylene nanocomposite. J. Phys.: Conf. Ser. 946(1), 2044 (2018).  https://doi.org/10.1088/1742-6596/946/1/012044CrossRefGoogle Scholar
  25. 25.
    Stegailov, V.V., Orekhov, N.D., Smirnov, G.S.: HPC hardware efficiency for quantum and classical molecular dynamics. In: Malyshkin, V. (ed.) PaCT 2015. LNCS, vol. 9251, pp. 469–473. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-21909-7_45CrossRefGoogle Scholar
  26. 26.
    Aristova, N.M., Belov, G.V.: Refining the thermodynamic functions of scandium triflouride SCF3 in the condensed state. Russ. J. Phys. Chemis. A 90(3), 700–703 (2016).  https://doi.org/10.1134/S0036024416030031CrossRefGoogle Scholar
  27. 27.
    Kochikov, I.V., Kovtun, D.M., Tarasov, Y.I.: Electron diffraction analysis for the molecules with degenerate large amplitude motions: intramolecular dynamics in arsenic pentafluoride. J. Mol. Struct. 1132, 139–148 (2017).  https://doi.org/10.1016/j.molstruc.2016.09.064CrossRefGoogle Scholar
  28. 28.
    Stegailov, V.V., Zhilyaev, P.A.: Warm dense gold: effective ionioninteraction and ionisation. Mol. Phys. 114(3–4), 509–518 (2016).  https://doi.org/10.1080/00268976.2015.1105390CrossRefGoogle Scholar
  29. 29.
    Minakov, D.V., Levashov, P.R.: Melting curves of metals with excited electrons in the quasiharmonic approximation. Phys. Rev. B 92, 224102 (2015).  https://doi.org/10.1103/PhysRevB.92.224102CrossRefGoogle Scholar
  30. 30.
    Minakov, D., Levashov, P.: Thermodynamic properties of LiD under compression with different pseudopotentials for lithium. Comput Mat. Sci. 114, 128–134 (2016).  https://doi.org/10.1016/j.commatsci.2015.12.008CrossRefGoogle Scholar
  31. 31.
    Eckhardt, W., et al.: 591 TFLOPS multi-trillion particles simulation on SuperMUC. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2013. LNCS, vol. 7905, pp. 1–12. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-38750-0_1CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Nikolay Kondratyuk
    • 1
    • 2
  • Grigory Smirnov
    • 1
  • Ekaterina Dlinnova
    • 3
  • Sergey Biryukov
    • 4
  • Vladimir Stegailov
    • 1
    Email author
  1. 1.Joint Institute for High Temperatures of the RASMoscowRussia
  2. 2.Moscow Institute of Physics and TechnologyDolgoprudnyRussia
  3. 3.National Research University Higher School of EconomicsMoscowRussia
  4. 4.JSC NICEVTMoscowRussia

Personalised recommendations