Advertisement

Using Resources of Supercomputing Centers with Everest Platform

  • Sergey Smirnov
  • Oleg Sukhoroslov
  • Vladimir Voloshinov
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 965)

Abstract

High-performance computing plays an increasingly important role in modern science and technology. However, the lack of convenient interfaces and automation tools greatly complicates the widespread use of HPC resources among scientists. The paper presents an approach to solving these problems relying on Everest, a web-based distributed computing platform. The platform enables convenient access to HPC resources by means of domain-specific computational web services, development and execution of many-task applications, and pooling of multiple resources for running distributed computations. The paper describes the improvements that have been made to the platform based on the experience of integration with resources of supercomputing centers. The use of HPC resources via Everest is demonstrated on the example of loosely coupled many-task application for solving global optimization problems.

Keywords

High-performance computing Clusters Many-task applications Distributed computing Web services Global optimization 

Notes

Acknowledgments

This work is supported by the Russian Science Foundation (project No. 16-11-10352). This work has been carried out using computing resources of the federal collective usage center Complex for Simulation and Data Processing for Mega-science Facilities at NRC “Kurchatov Institute”, http://ckp.nrcki.ru/.

References

  1. 1.
  2. 2.
    Talbi, E.G. (ed.): Parallel Combinatorial Optimization, vol. 58. Wiley, Hoboken (2006)Google Scholar
  3. 3.
    Afanasiev, A., Sukhoroslov, O., Voloshinov, V.: MathCloud: publication and reuse of scientific applications as RESTful web services. In: Malyshkin, V. (ed.) PaCT 2013. LNCS, vol. 7979, pp. 394–408. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-39958-9_36CrossRefGoogle Scholar
  4. 4.
    Afgan, E., et al.: Galaxy: a gateway to tools in e-science. In: Yang, X., Wang, L., Jie, W. (eds.) Guide to e-Science. CCN. Springer, London (2011).  https://doi.org/10.1007/978-0-85729-439-5_6CrossRefGoogle Scholar
  5. 5.
    Allan, R.N.: Virtual Research Environments: From Portals to Science Gateways. Elsevier, Amsterdam (2009)CrossRefGoogle Scholar
  6. 6.
    Chen, Z., Maly, K., Mehrotra, P., Vangala, P.K., Zubair, M.: Web-based framework for distributed computing. Concurrency Pract. Exp. 9(11), 1175–1180 (1997)CrossRefGoogle Scholar
  7. 7.
    Formiconi, A., et al.: World wide web interface for advanced spect reconstruction algorithms implemented on a remote massively parallel computer. Int. J. Med. Inform. 47(1–2), 125–138 (1997)CrossRefGoogle Scholar
  8. 8.
    Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure. Elsevier, Amsterdam (2003)Google Scholar
  9. 9.
    Gleixner, A., et al.: The SCIPoptimization suite 5.0. Technical report 17-61, ZIB, Takustr.7, 14195 Berlin (2017)Google Scholar
  10. 10.
    Kacsuk, P.: P-grade portal family for grid infrastructures. Concurrency Computat. Pract. Exp. 23(3), 235–245 (2011)CrossRefGoogle Scholar
  11. 11.
    Kostenetskiy, P., Safonov, A.: SUSU supercomputer resources. In: Proceedings of the 10th Annual International Scientific Conference on Parallel Computing Technologies (PCT 2016), Arkhangelsk, Russia, vol. 1576, pp. 561–573 (2016)Google Scholar
  12. 12.
    McLennan, M., Kennell, R.: HUBzero: a platform for dissemination and collaboration in computational science and engineering. Comput. Sci. Eng. 12(2), 48–53 (2010)CrossRefGoogle Scholar
  13. 13.
    Musin, O.R., Tarasov, A.S.: The Tammes problem for \(N\,{=}\,14\). Exp. Math. 24(4), 460–468 (2015)Google Scholar
  14. 14.
    Pierce, M.E., Youn, C., Fox, G.C.: The gateway computational web portal. Concurrency Comput. Pract. Exp. 14(13–15), 1411–1426 (2002)CrossRefGoogle Scholar
  15. 15.
    Raicu, I., et al.: Toward loosely coupled programming on petascale systems. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, p. 22. IEEE Press (2008)Google Scholar
  16. 16.
    Smirnov, S., Sukhoroslov, O., Volkov, S.: Integration and combined use of distributed computing resources with everest. Procedia Comput. Sci. 101, 359–368 (2016)CrossRefGoogle Scholar
  17. 17.
    Smirnov, S., Voloshinov, V.: Implementation of concurrent parallelization of branch-and-bound algorithm in everest distributed environment. Procedia Comput. Sci. 119, 83–89 (2017)CrossRefGoogle Scholar
  18. 18.
    Sukhoroslov, O., Volkov, S., Afanasiev, A.: A web-based platform for publication and distributed execution of computing applications. In: 2015 14th International Symposium on Parallel and Distributed Computing (ISPDC), pp. 175–184, June 2015Google Scholar
  19. 19.
    Sukhoroslov, O.: Integration of Everest platform with BOINC-based desktop grids (2017)Google Scholar
  20. 20.
    Thomas, M., Mock, S., Boisseau, J.: Development of web toolkits for computational science portals: the NPACI HotPage. In: The Ninth International Symposium on High-Performance Distributed Computing, 2000. Proceedings, pp. 308–309. IEEE (2000)Google Scholar
  21. 21.
    Thomas, M., et al.: Grid portal architectures for scientific applications. In: Journal of Physics: Conference Series, vol. 16, p. 596. IOP Publishing (2005)Google Scholar
  22. 22.
    Vigerske, S., Gleixner, A.: SCIP: global optimization of mixed-integer nonlinear programs in a branch-and-cut framework. Optim. Methods Softw. 33, 1–31 (2017)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Volkov, S., Sukhoroslov, O.: A generic web service for running parameter sweep experiments in distributed computing environment. Procedia Comput. Sci. 66, 477–486 (2015)CrossRefGoogle Scholar
  24. 24.
    Volkov, S., Sukhoroslov, O.: Simplifying the use of clouds for scientific computing with everest. Procedia Comput. Sci. 119, 112–120 (2017)CrossRefGoogle Scholar
  25. 25.
    Voloshinov, V., Smirnov, S., Sukhoroslov, O.: Implementation and use of coarse-grained parallel branch-and-bound in everest distributed environment. Procedia Comput. Sci. 108, 1532–1541 (2017)CrossRefGoogle Scholar
  26. 26.
    Yang, X., Martin, T., Mark, H., Mark, C., Ligang, H., Peter, M.: Survey of major tools and technologies for grid-enabled portal development. In: Proceedings of the UK e-Science All Hands Meeting (NeSC 2006). University of Cambridge Press (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sergey Smirnov
    • 1
  • Oleg Sukhoroslov
    • 1
  • Vladimir Voloshinov
    • 1
  1. 1.Institute for Information Transmission Problems of the Russian Academy of SciencesMoscowRussia

Personalised recommendations