, Volume 27, Issue 2, pp 139-167
Date: 16 Jan 2010

Understanding provenance black boxes

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


Current provenance stores associated with workflow management systems (WfMSs) capture enough coarse-grained information to describe which datasets were used and which processes were run. While this information is enough to rebuild a workflow run, it is not enough to facilitate user understanding. Because the data is manipulated via a series of black boxes, it is often impossible for a human to understand what happened to the data. In this work, we highlight the missing information that can assist user understanding. Unfortunately, provenance information is already very complex and difficult for a user to comprehend, which can be exacerbated by adding the extra information needed for deeper blackbox understanding. In order to alleviate this, we develop a model of provenance answers that follow a “roll up”, “drill down” strategy. We evaluate these techniques to determine if users have better understanding of provenance information. We show how this information can be captured by workflow management systems, and that the structures and information needed for this model are a negligible addition to standard provenance stores. Finally, we implement these techniques in a real provenance system, and evaluate implementation feasibility.

Communicated by Walid G. Aref and Ouzzani Mourad.
This work was supported in part by NSF grant number IIS 0741620 and by NIH grant number U54 DA021519.