A Formal Approach to Support Interoperability in Scientific Meta-workflows

Abstract

Scientific workflows orchestrate the execution of complex experiments frequently using distributed computing platforms. Meta-workflows represent an emerging type of such workflows which aim to reuse existing workflows from potentially different workflow systems to achieve more complex and experimentation minimizing workflow design and testing efforts. Workflow interoperability plays a profound role in achieving this objective. This paper is focused at fostering interoperability across meta-workflows that combine workflows of different workflow systems from diverse scientific domains. This is achieved by formalizing definitions of meta-workflow and its different types to standardize their data structures used to describe workflows to be published and shared via public repositories. The paper also includes thorough formalization of two workflow interoperability approaches based on this formal description: the coarse-grained and fine-grained workflow interoperability approach. The paper presents a case study from Astrophysics which successfully demonstrates the use of the concepts of meta-workflows and workflow interoperability within a scientific simulation platform.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Plankensteiner, K., Prodan, R., Janetschek, M., Fahringer, T., Montagnat, J., Rogers, D., Harvey, I., Taylor, I., Balasko, A., Kacsuk, P.: Fine-Grained Interoperability of scientific workflows in distributed computing infrastructures. In: The Journal Of Grid Computing, vol. 11, pp. 429–455 (2013)

  2. 2.

    Mosgrid Science Gateway. https://www.mosgrid.de/ [Last Accessed: 29 th June 2016]

  3. 3.

    Pierantoni, G., And Carley, E.: Metaworkflows and workflow interoperability for heliophysics. In: The Proceedings Of The 6 th,International Workshop On Science Gateways, pp. 79–84 (2014)

  4. 4.

    Castelli, G., Taffoni, G., Sciacca, E., Becciani, U., Costa, A., Krokos, M., Pasian, F., Vuerli, C.: VO-Compliant Workflows And Science Gateways. Astron. Comput. 11, 102–108 (2015)

    Article  Google Scholar 

  5. 5.

    Arshad, J., Terstyanszky, G., Kiss, T., Weingarten, N.: A Definition And Analysis Of The Role Of Meta-Workflows In Workflow Interoperability. In: the Proceedings of the 7 th International Workshop On Science Gateways (IWSG) 8-15 Budapest, Hungary (2015)

  6. 6.

    SHIWA: Sharing Interoperable Workflows For Large-Scale Scientific Simulation On Available DCIs. http://www.shiwa-workflow.eu, 2011. [Last Acccessed: 29 th June 2016]

  7. 7.

    Terstyanszky, G., Kukla, T., Kiss, T., Kacuuk, P., Balasko, A., Farkas, Z.: Enabling Scientific Workflow Sharing Through Coarse Grained Interoperability. Futur. Gener. Comput. Syst. 37, 46–59 (2014)

    Article  Google Scholar 

  8. 8.

    Guse Grid And Cloud Science Gateway Available online At: https://sourceforge.net/projects/guse/ [Last Accessed: 21 st June 2016]

  9. 9.

    Kertész, A., Sipos, G., And Kacsuk, P.: Brokering Multi-Grid Workflows in the p-GRADE portal. In: Euro-Par 2006: Parallel processing, vol. 4375, Springer, Berlin, pp 138–149 (2007)

  10. 10.

    Giardine, B., Riemer, C., Hardison, R.C., Burhans, R., Elnitski, L., Shah, P., Zhang, Y., Blankenberg, D., Albert, I., Taylor, J., Miller, W., Kent, W.J., Nekrutenko, A.: Galaxy: a platform for interactive Large-Scale genome analysis. Genome Res. 15(10), 1451–5 (2005)

    Article  Google Scholar 

  11. 11.

    Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M.R., Wipat, A., Li, P.: Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics 20(17), 3045–54 (2004)

    Article  Google Scholar 

  12. 12.

    ISO/IEC 13568: Information Technology- Z Formal Specification Notation- Syntax, Type System And Semantics, International Standard (2002)

  13. 13.

    Woodcock, J., Stepney, S., Cooper, D., Clark, J., And Jacob, I.: The certificationof the mondex electronic purse to ITSEC level e6, Formal Aspects of Computing 20(1) (2008)

  14. 14.

    Mcdermott, And Freitas, L.: A Formal Security Policy For Xenon, ACM Conference on Computer And Communications Security, Virginia, USA, 43–52 (2008)

  15. 15.

    Toda, Y.: The forMalization Of Simple Graphs. Formalized Math. 5(1), 137–144 (1996)

    Google Scholar 

  16. 16.

    Virtual Observatory Use-Case Report. ER Flow Project. Available At: http://www.erflow.eu/virtual-observatory-use-case [Last Accessed: 29 th June 2016]

  17. 17.

    Bertin, E., Arnouts, S.: Sextractor: Software For Source Extraction. Astron. Astrophys., Suppl. 317, 393 (1996)

    Article  Google Scholar 

  18. 18.

    Starnet Gateway Federation At: www.oact.inaf.it/starnet/[Last Accessed: 29 th June 2016]

  19. 19.

    Mink, J., Mann, R.G., Hanisch, R., Rots, A., Seaman, R., Jenness, T., Thomas, B., O’Mullane, W.: The Past, Present, And Future Of Astronomical Data Formats, 2015ASPC, 495, 11M

  20. 20.

    Williams, R., Hanisch, R., Szalay, A., Plante, R.: IVOA Recommendation: Simple Cone Search Version 1.03, astro-ph.IM. arXiv:1110.0498

  21. 21.

    Dowler, P., Tody, D., Bonnarel, F.: IVOA Simple Image Access, 2016, Astro-ph.IM. arXiv:1601.00519

  22. 22.

    Herres-Pawlis, S., Hoffman, A., Balasko, A., Kacsuk, P., Birkenheuer, G., Brinkman, A., Garza, L., Kruger, J., Gesing, S., Grunzke, R., Terstyansky, G., Weingarten, N.: Quantum Chemical Meta-Workflows In Mosgrid. Concurrency and Comput.: Pract. Experience 27(2), 344–357 (2015)

    Article  Google Scholar 

  23. 23.

    Herres-Pawlis, S., Hoffman, A., Garza, L., Kruger, J., Gesing, S., Grunzke, R., Nagel, W.E., Terstyansky, G., Weingarten, N.: Meta-Metaworkflows For Combining Quantum Chemistry And Molecular Dynamics. In: The Mosgrid Science Gateway in the 6 th, International Workshop On Science Gateways, pp. 73–78 (2014)

  24. 24.

    Herres-Pawlis, S., Hoffman, A., Gesing, S., Kruger, J., Balasko, A., Kacsuk, P., Grunzke, R., Birkenheuer, G., Packschies, L.: User-Friendly Metaworkflows In Quantum Chemistry. In: The IEEE International Conference On Cluster Computing, pp. 1–3 (2013)

  25. 25.

    Becciani, U., Sciacca, E., Costa, A., Massimino, P., Pistagna, C., Riggi, S., Vitello, F., Petta, C., Bandieramonte, M., Krokos, M.: Science gateway technologies for the astrophysics community in the concurrency and computation: Practice and experience (2014)

  26. 26.

    Abouelhoda, M., Alaa, S., Ghanem, M.: Meta-Workflows: Pattern-Based Interoperability Between Galaxy And Taverna In The International Workshop On Workflow Approaches For New Data-Centric Science (2010)

  27. 27.

    Glatard, T., Montagnat, J., Lingrand, D., Pennec, X.: Flexible And Efficient Workflow Deployment Of Data-Intensive Applications On Grids With MOTEUR. In: The International Journal Of High Performance Computing Applications (2008)

  28. 28.

    Korkhov, V., Krefting, D., Montagnat, J., Truong Huu, T., Kukla, T., Terstyanszky, G., Manset, D., Caan, M., Olabarriaga, S.: SHIWA Workflow interoperability solutions for neuroimaging data analysis. In: The Studies In Health Technology And Informaitcs, vol. 175, pp. 109–10 (2012)

  29. 29.

    Korkhov, V., Krefting, D., Kukla, T., Terstyanszky, G., Caan, M., Olabarriaga, S.D.: Exploring workflow interoperability tools For neuroimaging data analysis. In: the Proceedings of the 6 th workshop on the Workflows in support of Large-Scale Science, Seattle, U.S., pp. 87–96 (2011)

  30. 30.

    Kranjc, J., Podpecan, V., Lavrac, N.: Clowdflows: a cloud based scientific platform, in machine learning and knowledge discovery in databases. LNCS 7524, 816–819 (2012)

    Google Scholar 

  31. 31.

    Gil, Y., Ratnakar, V., Dellman, E.: Wings For Pegasus: A Semantic Approach To Creating Very Large Scientific Workflows. In: OWLED (2006)

  32. 32.

    Van Der Aalst, W.M.: The application of petri nets to workflow management. J. Circuits Syst. Comput. 8(1), 21–66 (1998)

    Article  Google Scholar 

  33. 33.

    Sroka, J., Hidders, J., Missier, P., Globe, C.: A formal semantics for the taverna2 workflow model. J. Comput. Syst. Sci. 76, 490–508 (2010)

    Article  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Junaid Arshad.

Additional information

This research has been funded by the EU-FP7 funding programme for research, technological development and demonstration under ER-FLOW project with grant agreement no. 312579.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Arshad, J., Terstyanszky, G., Kiss, T. et al. A Formal Approach to Support Interoperability in Scientific Meta-workflows. J Grid Computing 14, 655–671 (2016). https://doi.org/10.1007/s10723-016-9383-x

Download citation

Keywords

  • Meta-workflows
  • Workflow interoperability
  • Science gateways
  • Workflow repository