Yin & Yang: Demonstrating Complementary Provenance from noWorkflow & YesWorkflow

  • João Felipe Pimentel
  • Saumen Dey
  • Timothy McPhillips
  • Khalid Belhajjame
  • David Koop
  • Leonardo Murta
  • Vanessa Braganholo
  • Bertram Ludäscher
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9672)

Abstract

The noWorkflow and YesWorkflow toolkits both enable researchers to capture, store, query, and visualize the provenance of results produced by scripts that process scientific data. noWorkflow captures prospective provenance representing the program structure of Python scripts, and retrospective provenance representing key events observed during script execution. YesWorkflow captures prospective provenance declared through annotations in the comments of scripts, and supports key retrospective provenance queries by observing what files were used or produced by the script. We demonstrate how combining complementary information gathered by noWorkflow and YesWorkflow enables provenance queries and data lineage visualizations neither tool can provide on its own.

References

  1. 1.
    Dey, S., Belhajjame, K., Koop, D., Raul, M., Ludäscher, B.: Linking prospective and retrospective provenance in scripts. In: Theory and Practice of Provenance (TaPP) (2015)Google Scholar
  2. 2.
    Lerner, B., Boose, E.: RDataTracker: collecting provenance in an interactive scripting environment. In: Theory and Practice of Provenance (TaPP). Cologne, Germany (2014)Google Scholar
  3. 3.
    McPhillips, T., Bowers, S., Belhajjame, K., Ludäscher, B.: Retrospective provenance without a runtime provenance recorder. In: Theory and Practice of Provenance (TaPP) (2015)Google Scholar
  4. 4.
    Murta, L., Braganholo, V., Chirigati, F., Koop, D., Freire, J.: noWorkflow: capturing and analyzing provenance of scripts. In: Ludaescher, B., Plale, B. (eds.) IPAW 2014. LNCS, vol. 8628, pp. 71–83. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  5. 5.
    Pimentel, J., Freire, J., Murta, L., Braganholo, V.: Fine-grained provenance collection over scripts through program slicing. In: International Provenance and Annotation Workshop (IPAW), Washington D.C. (2016)Google Scholar
  6. 6.
    Pimentel, J.F., Dey, S., McPhillips, T., Belhajjame, K., Koop, D., Murta, L., Braganholo, V., Ludäscher, B.: Yin & Yang: demonstrating complementary provenance from noWorkflow & YesWorkflow. In: Mattoso, M., Glavic, B. (eds.): IPAW 2016. LNCS, vol. 9672, pp. 161–165 (2016). Technical Report & Demo. https://github.com/gems-uff/yin-yang-demo
  7. 7.
    Tariq, D., Ali, M., Gehani, A.: Towards automated collection of application-level data provenance. In: Theory and Practice of Provenance (TaPP) (2012)Google Scholar
  8. 8.
    Tsai, Y., McPhillips, S.E., González, A., McPhillips, T.M., Zinn, D., Cohen, A.E., Feese, M.D., Bushnell, D., Tiefenbrunn, T., Stout, C., Ludäscher, B., Hedman, B., Hodgson, K.O., Soltis, S.M.: AutoDrug: fully automated macromolecular crystallography workflows for fragment-based drug discovery. Acta Crystallogr. Sect. D: Biol. Crystallogr. 69(5), 796–803 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • João Felipe Pimentel
    • 1
  • Saumen Dey
    • 2
  • Timothy McPhillips
    • 3
  • Khalid Belhajjame
    • 4
  • David Koop
    • 5
  • Leonardo Murta
    • 1
  • Vanessa Braganholo
    • 1
  • Bertram Ludäscher
    • 3
  1. 1.Universidade Federal FluminenseNiteróiBrazil
  2. 2.University of California, DavisDavisUSA
  3. 3.University of Illinois, Urbana-ChampaignChampaignUSA
  4. 4.Université Paris-DauphineParisFrance
  5. 5.University of MassachusettsDartmouthUSA

Personalised recommendations