ES3: A Demonstration of Transparent Provenance for Scientific Computation

  • James Frew
  • Peter Slaughter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5272)


The Earth System Science Server (ES3) is a software environment for data-intensive Earth science, with unique capabilities for automatically and transparently capturing and managing the provenance of arbitrary computations. Transparent acquisition avoids the scientist having to express their computations in specific languages or schemas for provenance to be available. ES3 models provenance as relationships between processes and their input and output files. These relationships are captured by monitoring read and write accesses at various levels in the science software and asynchronously converting them to time-ordered streams of provenance events which are stored in an XML database. An ES3 provenance query returns an XML serialization of a provenance graph, forward or backwards from a specified process or file. We demonstrate ES3 provenance by generating complex data products from Earth satellite imagery.


ES3 provenance instrumentation passive transparency 


  1. 1.
    Frew, J., Metzger, D., Slaughter, P.: Automatic capture and reconstruction of computational provenance. Concurrency and Computation: Practice and Experience 20(5), 485–496 (2008)CrossRefGoogle Scholar
  2. 2.
    The IDL Computing Environment for Data Visualization & Analysis from ITT,
  3. 3.
    Brandes, U., Eiglsperger, M., Herman, I., Himsolt, M., Marshall, M.S.: GraphML Progress Report–Structural Layer Proposal. In: Mutzel, P., Jünger, M., Leipert, S. (eds.) GD 2001. LNCS, vol. 2265, p. 501. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Gansner, E.R., North, S.C.: An open graph visualization system and its applications to software engineering. Software—Practice and Experience 30(11), 1203–1233 (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    yEd - Java Graph Editor,
  6. 6.
    Moreau, L., Ludäscher, B., Altintas, I., Barga, R.S., Bowers, S., Callahan, S., Chin, G., Clifford, B., Cohen, S., Cohen-Boulakia, S., Davidson, S., Deelman, E., Digiampietri, L., Foster, I., Freire, J., Frew, J., Futrelle, J., Gibson, T., Gil, Y., Goble, C., Golbeck, J., Groth, P., Holland, D.A., Jiang, S., Kim, J., Koop, D., Krenek, A., McPhillips, T., Mehta, G., Miles, S., Metzger, D., Munroe, S., Myers, J., Plale, B., Podhorszki, N., Ratnakar, V., Santos, E., Scheidegger, C., Schuchardt, K., Seltzer, M., Simmhan, Y.L., Silva, C., Slaughter, P., Stephan, E., Stevens, R., Turi, D., Vo, H., Wilde, M., Zhao, J., Zhao, Y.: Special Issue: The First Provenance Challenge. Concurrency and Computation: Practice and Experience 20(5), 409–418 (2008)CrossRefGoogle Scholar
  7. 7.
    ImageMagick: Convert, Edit, and Compose Images,

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • James Frew
    • 1
  • Peter Slaughter
    • 1
  1. 1.Donald Bren School of Environmental Science and ManagementUniversity of CaliforniaSanta BarbaraUSA

Personalised recommendations