Skip to main content
SpringerLink
Log in

Hybrid Supercomputer Desmos with Torus Angara Interconnect: Efficiency Analysis and Optimization

  • Conference paper
  • First Online:
Parallel Computational Technologies (PCT 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 910))

Included in the following conference series:

Abstract

The paper describes the first experience of practical deployment of the hybrid supercomputer Desmos at the Joint Institute for High Temperatures of the Russian Academy of Sciences (JIHT RAS). We consider job scheduling statistics, energy efficiency, case studies of GPU acceleration efficiency and benchmarks of the distributed storage with a parallel file system.

The JIHT team was supported by the Russian Science Foundation (grant No. 14-50-00124). The Desmos supercomputer is a part of the Supercomputer Centre of JIHT RAS. The authors acknowledge the Shared Resource Center “Far Eastern Computing Resource” IACP FEB RAS (http://cc.dvo.ru) for granting access to the IRUS17 supercomputer.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Stegailov, V., et al.: Early performance evaluation of the hybrid cluster with torus interconnect aimed at molecular-dynamics simulations. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds.) PPAM 2017 Part I. LNCS, vol. 10777, pp. 327–336. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78024-5_29

    Chapter  Google Scholar 

  2. Vecher, V.S., Kondratyuk, N.D., Smirnov, G.S., Stegailov, V.V.: Angara-based hybrid supercomputer for efficient acceleration of computational materials science studies. In: Proceeding of International Conference Russian Supercomputing Days 2017, pp. 557–571 (2017)

    Google Scholar 

  3. Neuwirth, S., Frey, D., Nuessle, M., Bruening, U.: Scalable communication architecture for network-attached accelerators. In: 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pp. 627–638 (2015). https://doi.org/10.1109/HPCA.2015.7056068

  4. Puente, V., Beivide, R., Gregorio, J.A., Prellezo, J.M., Duato, J., Izu, C.: Adaptive bubble router: a design to improve performance in torus networks. In: Proceedings of the 1999 International Conference on Parallel Processing, pp. 58–67 (1999). https://doi.org/10.1109/ICPP.1999.797388

  5. Scott, S.L., Thorson, G.M.: The Cray T3E network: adaptive routing in a high performance 3D torus. In: HOT Interconnects IV. Stanford University, 15–16 August 1996 (1996)

    Google Scholar 

  6. Adiga, N.R., et al.: Blue Gene/L torus interconnection network. IBM J. Res. Dev. 49(2), 265–276 (2005). https://doi.org/10.1147/rd.492.0265

    Article  Google Scholar 

  7. Gómez-Martín, C., Vega-Rodríguez, M.A., González-Sánchez, J.L.: Fattened backfilling: an improved strategy for job scheduling in parallel systems. J. Parallel Distrib. Comput. 97(Suppl. C), 69–77 (2016). https://doi.org/10.1016/j.jpdc.2016.06.013

    Article  Google Scholar 

  8. Kraemer, A., Maziero, C., Richard, O., Trystram, D.: Reducing the number of response time SLO violations by a Cloud-HPC convergence scheduler. In: 2016 2nd International Conference on Cloud Computing Technologies and Applications (CloudTech), pp. 293–300 (2016). https://doi.org/10.1109/CloudTech.2016.7847712

  9. Mamaeva, A.A., Voevodin, V.V.: Methods for statistical analysis of large supercomputer job flow. In: Proceeding of International Conference Russian Supercomputing Days 2017, pp. 788–799 (2017)

    Google Scholar 

  10. Hoefler, T., Belli, R.: Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015, pp. 73:1–73:12. ACM, New York (2015). https://doi.org/10.1145/2807591.2807644

  11. Scogland, T., Azose, J., Rohr, D., Rivoire, S., Bates, N., Hackenberg, D.: Node variability in large-scale power measurements: perspectives from the Green500, Top500 and EEHPCWG. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015, pp. 74:1–74:11. ACM, New York (2015). https://doi.org/10.1145/2807591.2807653

  12. Höhnerbach, M., Ismail, A.E., Bientinesi, P.: The vectorization of the Tersoff multi-body potential: an exercise in performance portability. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016, pp. 7:1–7:13. IEEE Press, Piscataway (2016). https://doi.org/10.1109/SC.2016.6

  13. Kutzner, C., Pall, S., Fechner, M., Esztermann, A., de Groot, B.L., Grubmuller, H.: Best bang for your buck: GPU nodes for gromacs biomolecular simulations. J. Comput. Chemis. 36(26), 1990–2008 (2015). https://doi.org/10.1002/jcc.24030

    Article  Google Scholar 

  14. Luehr, N., Ufimtsev, I.S., Martínez, T.J.: Dynamic precision for electron repulsion integral evaluation on graphical processing units (GPUs). J. Chem. Theory Comput. 7(4), 949–954 (2011). https://doi.org/10.1021/ct100701w

    Article  Google Scholar 

  15. Mills, N., Alex Feltus, F., Ligon III, W.B.: Maximizing the performance of scientific data transfer by optimizing the interface between parallel file systems and advanced research networks. Futur. Gener. Comput. Syst. 79(Part 1), 190–198 (2018). https://doi.org/10.1016/j.future.2017.04.030

    Article  Google Scholar 

  16. Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3

    Chapter  Google Scholar 

  17. Vecher, V., Nikolskii, V., Stegailov, V.: GPU-accelerated molecular dynamics: energy consumption and performance. In: Voevodin, V., Sobolev, S. (eds.) RuSCDays 2016. CCIS, vol. 687, pp. 78–90. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-55669-7_7

    Chapter  Google Scholar 

  18. Stegailov, V., Vecher, V.: Efficiency analysis of intel and AMD x86\(\_\)64 architectures for Ab initio calculations: a case study of VASP. In: Voevodin, V., Sobolev, S. (eds.) RuSCDays 2017. CCIS, vol. 793, pp. 430–441. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71255-0_35

    Chapter  Google Scholar 

  19. Stegailov, V., Vecher, V.: Efficiency analysis of Intel, AMD and Nvidia 64-Bit hardware for memory-bound problems: a case study of Ab Initio calculations with VASP. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds.) PPAM 2017 Part II. LNCS, vol. 10778, pp. 81–90. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78054-2_8

    Chapter  Google Scholar 

  20. Smirnov, G.S., Stegailov, V.V.: Anomalous diffusion of guest molecules in hydrogen gas hydrates. High Temp. 53(6), 829–836 (2015). https://doi.org/10.1134/S0018151X15060188

    Article  Google Scholar 

  21. Orekhov, N.D., Stegailov, V.V.: Simulation of the adhesion properties of the Polyethylene/Carbon nanotube interface. Polym. Sci. Ser. A 58(3), 476–486 (2016). https://doi.org/10.1134/S0965545X16030135

    Article  Google Scholar 

  22. Pavlov, S.V., Kislenko, S.A.: Effects of carbon surface topography on the electrode/electrolyte interface structure and relevance to li-air batteries. Phys. Chem. Chem. Phys. 18, 30830–30836 (2016). https://doi.org/10.1039/C6CP05552D

    Article  Google Scholar 

  23. Antropov, A.S., Fidanyan, K.S., Stegailov, V.V.: Phonon density of states for solid uranium: accuracy of the embedded atom model classical interatomic potential. J. Phys.: Conf. Ser. 946(012094), 94 (2018). https://doi.org/10.1088/1742-6596/946/1/012094

    Article  Google Scholar 

  24. Logunov, M.A., Orekhov, N.D.: Molecular dynamics study of cavitation in carbon nanotube reinforced polyethylene nanocomposite. J. Phys.: Conf. Ser. 946(1), 2044 (2018). https://doi.org/10.1088/1742-6596/946/1/012044

    Article  Google Scholar 

  25. Stegailov, V.V., Orekhov, N.D., Smirnov, G.S.: HPC hardware efficiency for quantum and classical molecular dynamics. In: Malyshkin, V. (ed.) PaCT 2015. LNCS, vol. 9251, pp. 469–473. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21909-7_45

    Chapter  Google Scholar 

  26. Aristova, N.M., Belov, G.V.: Refining the thermodynamic functions of scandium triflouride SCF3 in the condensed state. Russ. J. Phys. Chemis. A 90(3), 700–703 (2016). https://doi.org/10.1134/S0036024416030031

    Article  Google Scholar 

  27. Kochikov, I.V., Kovtun, D.M., Tarasov, Y.I.: Electron diffraction analysis for the molecules with degenerate large amplitude motions: intramolecular dynamics in arsenic pentafluoride. J. Mol. Struct. 1132, 139–148 (2017). https://doi.org/10.1016/j.molstruc.2016.09.064

    Article  Google Scholar 

  28. Stegailov, V.V., Zhilyaev, P.A.: Warm dense gold: effective ionioninteraction and ionisation. Mol. Phys. 114(3–4), 509–518 (2016). https://doi.org/10.1080/00268976.2015.1105390

    Article  Google Scholar 

  29. Minakov, D.V., Levashov, P.R.: Melting curves of metals with excited electrons in the quasiharmonic approximation. Phys. Rev. B 92, 224102 (2015). https://doi.org/10.1103/PhysRevB.92.224102

    Article  Google Scholar 

  30. Minakov, D., Levashov, P.: Thermodynamic properties of LiD under compression with different pseudopotentials for lithium. Comput Mat. Sci. 114, 128–134 (2016). https://doi.org/10.1016/j.commatsci.2015.12.008

    Article  Google Scholar 

  31. Eckhardt, W., et al.: 591 TFLOPS multi-trillion particles simulation on SuperMUC. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2013. LNCS, vol. 7905, pp. 1–12. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38750-0_1

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Joint Institute for High Temperatures of the RAS, Moscow, Russia

    Nikolay Kondratyuk, Grigory Smirnov & Vladimir Stegailov

  2. Moscow Institute of Physics and Technology, Dolgoprudny, Russia

    Nikolay Kondratyuk

  3. National Research University Higher School of Economics, Moscow, Russia

    Ekaterina Dlinnova

  4. JSC NICEVT, Moscow, Russia

    Sergey Biryukov

Authors
  1. Nikolay Kondratyuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grigory Smirnov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ekaterina Dlinnova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sergey Biryukov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vladimir Stegailov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Stegailov .

Editor information

Editors and Affiliations

  1. South Ural State University, Chelyabinsk, Russia

    Leonid Sokolinsky

  2. South Ural State University, Chelyabinsk, Russia

    Mikhail Zymbler

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kondratyuk, N., Smirnov, G., Dlinnova, E., Biryukov, S., Stegailov, V. (2018). Hybrid Supercomputer Desmos with Torus Angara Interconnect: Efficiency Analysis and Optimization. In: Sokolinsky, L., Zymbler, M. (eds) Parallel Computational Technologies. PCT 2018. Communications in Computer and Information Science, vol 910. Springer, Cham. https://doi.org/10.1007/978-3-319-99673-8_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-99673-8_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-99672-1

  • Online ISBN: 978-3-319-99673-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation