Advertisement

HPC-as-a-Service via HEAppE Platform

  • Vaclav Svaton
  • Jan Martinovic
  • Jan KrenekEmail author
  • Thomas Esch
  • Pavel Tomancak
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 993)

Abstract

The HPC-as-a-Service concept is to provide users with simple and intuitive access to a supercomputing infrastructure without the need to buy and manage their own physical servers or data centers. This article presents the commonly used services and implementations of this concept and introduces our own in-house application framework called High-End Application Execution Middleware (HEAppE Middleware). HEAppE’s universally designed software architecture enables unified access to different HPC systems through simple object-oriented web-based APIs, thus providing HPC capabilities to users without the necessity to manage the running jobs forms the command-line interface of the HPC scheduler directly on the cluster. This article also contains the list of several pilot use-cases from a number of thematic domains where the HEAppE Platform was successfully used. Two of those pilots, focusing on satellite image analysis and bioimage informatics, are presented in more detail.

Notes

Aknowledgments

This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPS II) project ‘IT4Innovations excellence in science - LQ1602’, by the European Regional Development Fund in the IT4Innovations national supercomputing center - path to exascale project, project number CZ.02.1.01/0.0/0.0/16_013/0001791 within the Operational Programme Research, Development and Education, partially supported by The Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project ‘IT4Innovations National Supercomputing Center –LM2015070’ and by the SGC grant No. SP2019/108 ‘Extension of HPC platforms for executing scientific pipelines’, VSB - Technical University of Ostrava, Czech Republic. The project ‘Urban Thematic Exploitation Platform’ was funded by European Space Agency (ESA) of ESA Contract No. 4000113707/15/I-NB.

References

  1. 1.
    Altair Engineering, Inc.: PBS Professional. https://www.pbsworks.com/PBSProduct.aspx?n=Altair-PBS-Professional&c=Overview-and-Capabilities. Accessed 28 Feb 2019
  2. 2.
    Baun, C., Kunze, M., Nimis, J., Tai, S.: Cloud Computing Web-Based Dynamic IT Services. Springer, Heidelberg (2011). ISBN 978-3-642-20917-8zbMATHGoogle Scholar
  3. 3.
    IT4Innovations National Supercomputing Center. High-End Application Execution Middleware. http://www.heappe.eu. Accessed 28 Feb 2019
  4. 4.
    eResearch South Australia. eResearch SA EMU-Cluster. https://www.ersa.edu.au/service/cloud/clusters-in-the-cloud. Accessed 9 Feb 2019
  5. 5.
    Ertaul, L., Kaur, M., Gudise, V.A.K.R.: Implementation and performance analysis of PBKDF2, Bcrypt, Scrypt algorithms. In: Proceedings of the International Conference on Wireless Networks (ICWN), Athens, pp. 66–72 (2016)Google Scholar
  6. 6.
    Fusaro, V.A., Patil, P., Gafni, E., Wall, D.P., Tonellato, P.J.: Biomedical cloud computing with Amazon Web Services. PLOS Computat. Biol. (2011).  https://doi.org/10.1371/journal.pcbi.1002147CrossRefGoogle Scholar
  7. 7.
    Google LLC. Google Cloud HPC Computing. https://cloud.google.com/solutions/hpc. Accessed 28 Feb 2019
  8. 8.
    Halligan, B.D., Geiger, J.F., Vallejos, A.K., Greene, A.S., Twigger, S.N.: Low cost, scalable proteomics data analysis using Amazon’s cloud computing services and open source search algorithms. J. Proteome Res., 3148–3153 (2009).  https://doi.org/10.1021/pr800970z
  9. 9.
    Armbrust, M., Fox, A., Griffith, R., Joseph, D.A., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: A view of cloud computing. Commun. ACM 53(4), 50–58 (2010)CrossRefGoogle Scholar
  10. 10.
    Esch, T., Asamer, H., Bachofer, F., Balhar, J., Böttcher, M., Boissier, E., Hirner, A., Mathot, E., Marconcini, M., Metz-Marconcini, A., Permana, H., Soukup, T., Svaton, V., Üreyen S., Zeidler, J.: New prospects in analysing big data from space - the urban thematic exploitation platform. In: 2018 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2018, pp. 8193–8196 (2018)Google Scholar
  11. 11.
    Krumnikl, M., Moravec, P., Kozusznik, J., Klimova, J., Bainar, P., Svaton, V., Tomancak, P.: SPIM workflow manager for HPC. Bioinformatics (2019).  https://doi.org/10.1093/bioinformatics/btz140
  12. 12.
    Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Job Scheduling Strategies for Parallel Processing, pp. 44–60 (2003).  https://doi.org/10.1007/10968987_3
  13. 13.
    Schadt, E.E., Linderman, M.D., Sorenson, J., Lee, L., Nolan, G.P.: Computational solutions to large-scale data management and analysis. Nat. Rev. Genet. 647–657 (2010).  https://doi.org/10.1038/nrg2857
  14. 14.
    Richardson, L., Ruby, S.: RESTful Web Services, 1st edn. O’Reilly Media, Newton (2007). ISBN 978-0596529260Google Scholar
  15. 15.
    Svaton, V., Podhoranyi, M., Vavrik, R., Veteska, P., Szturcova, D., Vojtek, D., Martinovic, J., Vondrak, V.: Floreon+: a web-based platform for flood prediction, hydrologic modelling and dynamic data analysis. In: GIS OSTRAVA 2017, pp. 409–422 (2018).  https://doi.org/10.1007/978-3-319-61297-3_30
  16. 16.
    Vecchiola, Ch., Pandey, S., Buyya, R.: High-performance cloud computing: a view of scientific applications. In: 10th International Symposium on Pervasive Systems, Algorithms, and Networks, pp. 4–16 (2009).  https://doi.org/10.1109/I-SPAN.2009.150
  17. 17.
    Youseff, L., Butrico, M., Silva, D.D.: Toward a unified ontology of cloud computing. In: Grid Computing Environments Workshop, pp. 1–10, November 2008Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Vaclav Svaton
    • 1
  • Jan Martinovic
    • 1
  • Jan Krenek
    • 1
    Email author
  • Thomas Esch
    • 2
  • Pavel Tomancak
    • 3
  1. 1.IT4InnovationsVSB – Technical University of OstravaOstravaCzech Republic
  2. 2.German Aerospace Center, Earth Observation CenterOberpfaffenhofenGermany
  3. 3.Max Planck Institute of Molecular Cell Biology and GeneticsDresdenGermany

Personalised recommendations