Abstract
A hybrid HPC/Cloud architecture is a potential solution to the ever-increasing demand for high-availability on-demand resources for eScience applications. eScience applications are primarily compute-intensive, and thus require HPC resources. They usually also include pre- and post-processing steps, which can be moved into the Cloud in order to keep costs low. We believe that currently no methodology exists to bridge the gap between HPC and Cloud in a seamless manner. The goal is to lower the gap for non-professionals in order to exploit external facilities through an automated deployment and scaling both vertically (HPC) and horizontally (Cloud). This paper demonstrates how representative eScience applications can easily be transferred from HPC to Cloud using the model-based cross-cloud deployment platform PaaSage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
OmiStack is a private Cloud provided by the University of Ulm based on OpenStack. OpenStack is a leading software to manage Clouds.
- 5.
It should be noted that VM sizes were selected due to cost constraints and availability. m1.medium has 2 vCPUs, 40 GB disk, and 4 GB RAM; m3.medium has 1 vCPU, 4 GB SSD, and 3.75 GB RAM; m3.xlarge has 4 vCPUs, 2\(\,\times \,\)40 SSD, and 15 GB RAM. m3.xlarge was unavailable on Omistack.
References
Balis, B.: HyperFlow: a model of computation, programming approach and enactment engine for complex distributed workflows. Future Gener. Comput. Syst. 55, 147–162 (2016). https://doi.org/10.1016/j.future.2015.08.015
Balis, B., Figiela, K., Malawski, M., Pawlik, M., Bubak, M.: A lightweight approach for deployment of scientific workflows in cloud infrastructures. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 281–290. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32149-3_27
Baur, D., Domaschka, J.: Experiences from building a cross-cloud orchestration tool. In: Proceedings of the 3rd Workshop on CrossCloud Infrastructures and Platforms, pp. 4:1–4:6. ACM (2016). https://doi.org/10.1145/2904111.2904116
Kim, H., el Khamra, Y., Jha, S., Parashar, M.: Exploring application and infrastructure adaptation on hybrid grid-cloud infrastructure. In: Proceedings 19th ACM International Symposium on High Performance Distributed Computing, HPDC 2010, pp. 402–412. ACM (2010). https://doi.org/10.1145/1851476.1851536
Krol, D., Kitowski, J.: Self-scalable services in service oriented software for cost-effective data farming. Future Gener. Comput. Syst. 54(C), 1–15 (2016). https://doi.org/10.1016/j.future.2015.07.003
Marathe, A., Harris, R., Lowenthal, D.K., de Supinski, B.R., Rountree, B., Schulz, M., Yuan, X.: A comparative study of high-performance computing on the cloud. In: Proceedings of the 22nd International Symposium on High-Performance Parallel and distributed Computing, pp. 239–250. ACM (2013). https://doi.org/10.1145/2462902.2462919
Qin, J., Fahringer, T.: Scientific Workflows - Programming, Optimization, and Synthesis with ASKALON and AWDL. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30715-7
Rossini, A.: Cloud application modelling and execution language (CAMEL) and the PaaSage workflow. In: Advances in Service-Oriented and Cloud Computing—Workshops of ESOCC, vol. 567, pp. 437–439, September 2015. https://doi.org/10.1007/978-3-319-33313-7
Acknowledgements
We thankfully acknowledge the support of the EU 7th Framework Programme (FP7/2013-2016) under grant agreement number 317715. Access to Omistack Cloud resources was kindly provided by University of Ulm, Germany. HyperFlow and Scalarm are partially supported by the AGH Statutory Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Hoppe, D. et al. (2018). Bridging the Gap Between HPC and Cloud Using HyperFlow and PaaSage. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2017. Lecture Notes in Computer Science(), vol 10777. Springer, Cham. https://doi.org/10.1007/978-3-319-78024-5_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-78024-5_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78023-8
Online ISBN: 978-3-319-78024-5
eBook Packages: Computer ScienceComputer Science (R0)