Advertisement

A Lightweight Approach for Deployment of Scientific Workflows in Cloud Infrastructures

  • Bartosz BalisEmail author
  • Kamil Figiela
  • Maciej Malawski
  • Maciej Pawlik
  • Marian Bubak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9573)

Abstract

We propose a lightweight solution for deployment of scientific workflows in diverse cloud platforms. In the proposed deployment model, an instance of a workflow runtime environment is created on demand in the cloud as part of the workflow application. Such an approach improves isolation and helps overcome major issues of alternative solutions, leading to an easier integration. The concept has been implemented in the HyperFlow workflow environment. We describe the approach in general and illustrate it with two case studies showing the integration of HyperFlow with the PLGrid infrastructure, and the PaaSage cloud platform. Lessons learned from these two experiences lead to the conclusion that the proposed solution minimizes the development effort required to implement the integration, accelerates the deployment process in a production system, and reduces maintenance issues. Performance evaluation proves that, for certain workflows, the proposed approach can lead to significant improvement of the workflow execution time.

Keywords

Scientific workflows Cloud infrastructures Application deployment 

Notes

Acknowledgments

This work is partially supported by the PLGrid Core project (POIG.02.03.00-00-096/10); and by the EU-FP7 project PaaSage; AGH grant no. 11.11.230.124 is also acknowledged.

References

  1. 1.
    Balasko, A., Farkas, Z., Kacsuk, P.: Building science gateways by utilizing the generic WS-PGRADE/gUSE workflow system. Comput. Sci. J. 14(2), 307–325 (2013)CrossRefGoogle Scholar
  2. 2.
    Balis, B., Figiela, K., Malawski, M., Jopek, K.: Leveraging workflows and clouds for a multi-frontal solver for finite-element meshes. Procedia Comput. Sci. 51, 944–953 (2015)CrossRefGoogle Scholar
  3. 3.
    Balis, B., Kasztelnik, M., Malawski, M., Nowakowski, P., Wilk, B., Pawlik, M., Bubak, M.: Execution management and efficient resource provisioning for flood decision support. Procedia Comput. Sci. 51, 2377–2386 (2015)CrossRefGoogle Scholar
  4. 4.
    Balis, B.: Hyperflow: a model of computation, programming approach and enactment engine for complex distributed workflows. Future Gener. Comput. Syst. 55, 147–162 (2015)CrossRefGoogle Scholar
  5. 5.
    Balis, B.: Increasing scientific workflow programming productivity with HyperFlow. In: Proceedings of 9th Workshop on Workflows in Support of Large-Scale Science. WORKS 2014, pp. 59–69. IEEE Press (2014)Google Scholar
  6. 6.
    Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P.J., Mayani, R., Chen, W., da Silva, R.F., Livny, M., et al.: Pegasus, a workflow management system for science automation. Future Gener. Comput. Syst. 46, 17–35 (2014)CrossRefGoogle Scholar
  7. 7.
    Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., Good, J.: On the use of cloud computing for scientific workflows. In: eScience, 2008. eScience 2008, pp. 640–645. IEEE (2008)Google Scholar
  8. 8.
    Jeffery, K., Horn, G., Schubert, L.: A vision for better cloud applications. In: Proceedings of the 2013 International Workshop on Multi-Cloud Applications and Federated Clouds, pp. 7–12. ACM (2013)Google Scholar
  9. 9.
    Juve, G., Deelman, E.: Scientific workflows and clouds. Crossroads 16(3), 14–18 (2010)CrossRefGoogle Scholar
  10. 10.
    Kahn, G.: The semantics of a simple language for parallel programming. In: Information Processing, pp. 471–475. North-Holland(1974)Google Scholar
  11. 11.
    Kitowski, J., Turała, M., Wiatr, K., Dutka, Ł.: PL-Grid: foundations and perspectives of national computing infrastructure. In: Bubak, M., Szepieniec, T., Wiatr, K. (eds.) PL-Grid 2011. LNCS, vol. 7136, pp. 1–14. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Knyazkov, K.V., Kovalchuk, S.V., Tchurov, T.N., Maryin, S.V., Boukhanovsky, A.V.: CLAVIRE: e-Science infrastructure for data-driven computing. J. Comput. Sci. 3(6), 504–510 (2012)CrossRefGoogle Scholar
  13. 13.
    Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In: Proceedings of International Conference on High Performance Computing, Networking, Storage and Analysis, p. 22. IEEE Computer Society Press (2012)Google Scholar
  14. 14.
    Missier, P., Soiland-Reyes, S., Owen, S., Tan, W., Nenadic, A., Dunlop, I., Williams, A., Oinn, T., Goble, C.: Taverna, reloaded. In: Gertz, M., Ludäscher, B. (eds.) SSDBM 2010. LNCS, vol. 6187, pp. 471–481. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Nowakowski, P., Bartyński, T., Gubała, T., Harȩżlak, D., Kasztelnik, M., Meizner, J., Bubak, M.: Managing cloud resources for medical applications. In: Proceedings of the Cracow Grid Workshop, vol. 2012 (2012)Google Scholar
  16. 16.
    Radecki, M., Szymocha, T., Piontek, T., Bosak, B., Mamoński, M., Wolniewicz, P., Benedyczak, K., Kluszczyński, R.: Reservations for compute resources in federated e-infrastructure. In: Bubak, M., Kitowski, J., Wiatr, K. (eds.) eScience on Distributed Computing Infrastructure. LNCS, vol. 8500, pp. 80–93. Springer, Heidelberg (2014)Google Scholar
  17. 17.
    Vöckler, J.S., Juve, G., Deelman, E., Rynge, M., Berriman, B.: Experiences using cloud computing for a scientific workflow application. In: Proceedings of 2nd International Workshop on Scientific Cloud Computing, pp. 15–24. ACM (2011)Google Scholar
  18. 18.
    Wang, J., Altintas, I.: Early cloud experiences with the Kepler scientific workflow system. Procedia Comput. Sci. 9, 1630–1634 (2012)CrossRefGoogle Scholar
  19. 19.
    Wolstencroft, K., et al.: The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic Acids Research (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Bartosz Balis
    • 1
    • 2
    Email author
  • Kamil Figiela
    • 1
    • 2
  • Maciej Malawski
    • 1
    • 2
  • Maciej Pawlik
    • 1
    • 2
  • Marian Bubak
    • 1
    • 2
  1. 1.Department of Computer ScienceAGH University of Science and TechnologyKrakowPoland
  2. 2.ACC Cyfronet AGHAGH University of Science and TechnologyKrakowPoland

Personalised recommendations