Advertisement

Journal of Grid Computing

, Volume 14, Issue 4, pp 641–654 | Cite as

Infrastructure Aware Scientific Workflows and Infrastructure Aware Workflow Managers in Science Gateways

  • Peter Kacsuk
  • Gabor Kecskemeti
  • Attila KerteszEmail author
  • Zsolt Nemeth
  • József Kovács
  • Zoltán Farkas
Article

Abstract

The workflow interoperability problem was successfully solved by the SHIWA project if the workflows to be integrated were running in the same grid infrastructure. However, in the more generic case when the workflows were running in different infrastructures the problem has not been solved yet. In the current paper we show a solution for this problem by introducing a new type of workflow called infrastructure-aware workflow. These are scientific workflows extended with new node types that enable the on-the-fly creation and destruction of the required infrastructures in the clouds. The paper shows the semantics of these new types of nodes and workflows and also how they can solve the workflow interoperability problem. The paper also describes how these new type of workflows can be implemented by a new service called Occopus, and how this service can be integrated with the existing SHIWA Simulation Platform services like the WS-PGRADE/gUSE portal to provide the required functionalities of solving the workflow interoperability problem.

Keywords

Workflow Cloud Virtual infrastructure Dynamic deployment 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Chen, Y., Wo, T., Li, J.: An efficient resource management system for on-line virtual clusterprovision. In: IEEE CLOUD, pp 72–79 (2009)Google Scholar
  3. 3.
    Kacsuk P. (ed.): Science Gateways for Distributed Computing Infrastructures: Development Framework and Exploitation by Scientific User Communities, pp. 301. Springer (2014)Google Scholar
  4. 4.
  5. 5.
    Docker. https://www.docker.io (2014)
  6. 6.
    Dörnemann, T., Juhnke, E., Freisleben, B.: On-demand resource provisioning for bpel workflows using amazon’s elastic compute cloud. In: IEEE/ACM International Symposium on Cluster Computing and the Grid(CCGrid), pp 140–147 (2009)Google Scholar
  7. 7.
    Dörnemann, T., Juhnke, E., Noll, T., Seiler, D., Freisleben, B.: Data flow driven scheduling of bpel workflows using cloud resources. In: IEEE CLOUD, pp 196–203 (2010)Google Scholar
  8. 8.
    Elmroth, E., Hernández, F., Tordsson, J.: Three fundamental dimensions of scientific workflow interoperability: Model of computation, language, and execution environment. Fut. Gen. Comp. Syst 26 (2), 245–256 (2010)CrossRefGoogle Scholar
  9. 9.
    Building a European Research Community through Interoperable Workflows and Data (ER-flow) Eu FP7 project. Online: http://www.erflow.eu/ (2013)
  10. 10.
    Huang, C.-J., Guan, C.-T., Chen, H.-M., Wang, Y.-W., Chang, S.-C., Li, C.-Y., Weng, C.-H.: An adaptive resource management scheme in cloud computing. Eng. Appl. AI 26(1), 382–389 (2013)CrossRefGoogle Scholar
  11. 11.
    P Kacsuk, Z., Farkas, M., Kozlovszky, G., Hermann, A., Balasko, K., Karoczkai: WS-PGRADE/gUSE generic DCI gateway framework for a large variety of user communities. J. Grid Comput. 10(4), 601–630 (2012)CrossRefGoogle Scholar
  12. 12.
    Kangetal, S.: Orchestrator: An active resource orchestration framework for mobile context monitoring in sensor-rich mobile environments. In: IEEE International Conference on Pervasive Computing and Communications (2010)Google Scholar
  13. 13.
    Kecskemeti, G., Gergely, M., Visegradi, A., Nemeth, Z., Kovacs, J., Kacsuk, P.: One click cloud orchestrator: Bringing complex applications effortlessly to the clouds. In: The 2nd InternationalDIHCWorkshop held in conjunction with Euro-Par, Porto, Portugal (2014)Google Scholar
  14. 14.
    Lorincz, K., Chen, B.R., Waterman, J., Werner-Allen, G., Welsh, M.: Resource aware programming in the pixie os. In: SenSystem, pp 211–224 (2008)Google Scholar
  15. 15.
    Maurer, M., Brandic, I., Sakellariou, R.: Adaptive resource configuration for cloud infrastructure management, vol. 29 (2013)Google Scholar
  16. 16.
    Van der Merwe, J., Ramakrishnan, K., Fairchild, M., Flavel, A., Houle, J., Lagar-Cavilla, H.A., Mulligan, J.: Towards a ubiquitous cloud computing infrastructure. In: Proceedings of the IEEE LANMAN Workshop (2010)Google Scholar
  17. 17.
    Plale, B., Withana, E., Herath, C., Chandrasekar, K., Luo, Y.: Effectiveness of Hybrid Workflow Systems for Computational Science. ICCS, 508–517 (2012)Google Scholar
  18. 18.
    Plankensteiner, K., et al.: Fine-Grain Interoperability of Scientific Workflows in Distributed Computing Infrastructures. Journal of grid computing 11.3, 429–455 (2013)Google Scholar
  19. 19.
    Puppet. http://puppetlabs.com (2014)
  20. 20.
    SaltStack. http://www.saltstack.com (2014)
  21. 21.
    SHaring Interoperable Workflows for large-scale scientific simulations on Available DCIs (SHIWA) Eu FP7 project. Online: http://www.shiwa-workflow.eu/ (2013)
  22. 22.
    Terstyanszky, G., Kukla, T., Kiss, T., Kacsuk, P., Balasko, A., Farkas, Z.: Enabling scientific workflow sharing through coarse-grained interoperability. Fut. Gener. Comput. Syst. 37, 46–59 (2014)CrossRefGoogle Scholar
  23. 23.
    Ubuntu Juju: http://juju.ubuntu.com (2014)
  24. 24.
    Vukojevic-Haupt, K., Karastoyanova, D., Leymann, F.: On-demand Provisioning of Infrastructure, Middleware and Services for Simulation Workflows. SOCA, 91–98 (2013)Google Scholar
  25. 25.
  26. 26.
  27. 27.
  28. 28.
    Kozlovszky, M., Karóczkai, K., Márton, I., Kacsuk, P.: Tibor gottdank: DCI bridge: Executing WS-PGRADE workflows in distributed computing infrastructures. In: Science Gateways for Distributed Computing Infrastructures, Book Chapter. Springer Verlag (2014)Google Scholar
  29. 29.
    Farkas, Z., Hajnal, Á., Kacsuk, P., Kacsuk, P.: WS-PGRADE/gUSE and clouds. In: Kacsuk, P. (ed.) Science Gateways for Distributed Computing Infrastructures: Development Framework and Exploitation by Scientific User Communities, pp 97–109. Springer (2014)Google Scholar
  30. 30.
    Gugnani, S., Kiss, T.: Extending Scientific Workflow Systems to Support MapReduce Based Applications in the Cloud, 7th International Workshop on Science Gateways, IWSG 2015, pp 16–21, Budapest, Hungary (2015). doi: 10.1109/IWSG.2015.15
  31. 31.
    CloudBroker Platform: http://cloudbroker.com/ (2015)
  32. 32.
    Kacsuk, P., Kecskemeti, G., Kertesz, A., Nemeth, Z., Visegradi, A., Gergely, M.: Infrastructure Aware Scientific Workflows and Their Support by a Science Gateway. 7th International Workshop on Science Gateways - IWSG 2015:, pp 22–27. Proceedings (2015)Google Scholar
  33. 33.
    Krüger, J., et al.: The MoSGrid science gateway – A complete solution for molecular simulations. J. Chem. Theory Comput. 10(6), 2232–2245 (2014). doi: 10.1021/ct500159h CrossRefGoogle Scholar
  34. 34.
    Grunzke, R., Krüger, J., Jäkel, R., et al.: Metadata management in the MoSGrid science gateway - evaluation and the expansion of quantum chemistry support. Journal Grid Computing (2016). doi: 10.1007/s10723-016-9362-2
  35. 35.
    Korkhov, V., Krefting, D., Kukla, T., et al.: Exploring workflow interoperability for neuroimage analysis on the SHIWA platform. J. Grid Comput. 11, 505 (2013). doi: 10.1007/s10723-013-9262-7 CrossRefGoogle Scholar
  36. 36.
    Kiss, T.: Science gateways for the broader take-up of distributed computing infrastructures. J Grid Comput. 10, 599 (2012). doi: 10.1007/s10723-012-9245-0 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Peter Kacsuk
    • 1
  • Gabor Kecskemeti
    • 1
  • Attila Kertesz
    • 1
    Email author
  • Zsolt Nemeth
    • 1
  • József Kovács
    • 1
  • Zoltán Farkas
    • 1
  1. 1.Laboratory of Parallel and Distributed SystemsMTA SZTAKI the Institute for Computer Science and Control of the Hungarian Academy of SciencesBudapestHungary

Personalised recommendations