gLite Job Provenance
- 4 Citations
- 932 Downloads
Abstract
The Job Provenance (JP) service is designed to automate keeping track of computations on large scale Grids, giving thus users a tool to correctly archive information about their jobs and to re-submit any job in a reconstructed environment. JP provides a permanent minimal record of job (and its environment) related information, to which free-form user annotations can be added. JP also offers the capability of configuring any number of indexed logical views on the large collections of raw data, allowing efficient processing of even complex user queries selecting on both system data and the annotations. The scalable architecture, capable to handle millions of jobs in a single JP installation, and integrated into the EGEE gLite middleware environment is presented.
Keywords
Service Orient Architecture User Query Index Server Logical View Primary StorageReferences
- 1.AliEn (ALIce ENvironment), http://aliceinfo.cern.ch/AliEn
- 2.EU FP6 Programme Enabling and Supporting Provenance in Grids for Complex Problems, http://twiki.gridprovenance.org/bin/view/Provenance/ProjectInformation
- 3.gLite—Ligthweight Middleware for Grid Computing, http://glite.web.cern.ch/glite/default.asp
- 4.Mygrid Provenance Outline, http://phoebus.cs.man.ac.uk/twiki/bin/view/Mygrid/ProvenanceOutline
- 5.PASOA: Provenance Aware Service Oriented Architecture, http://twiki.pasoa.ecs.soton.ac.uk/bin/view/PASOA/AboutPasoa
- 6.SAM (Sequential data Access via Meta-data), http://d0db.fnal.gov/sam/
- 7.The Atlas Metadata Interface, https://atlastagcollector.in2p3.fr:8443/AMI/
- 8.Chen, L., Tan, V., Xu, F., Biller, A., Groth, P., Miles, S., Ibbotson, J., Luck, M., Moreau, L.: A proof of concept: Provenance in a Service Oriented Architecture. In: Proceedings of the fourth UK e-Science All Hands Meeting, Nottingham, UK (2005)Google Scholar
- 9.EGEE JRA1. EGEE Middleware Design—Release 1, https://edms.cern.ch/document/487871/
- 10.Groth, P., Luck, M., Moreau, L.: A protocol for recording provenance in service-oriented grids. In: Higashino, T. (ed.) OPODIS 2004. LNCS, vol. 3544, pp. 124–139. Springer, Heidelberg (2005)CrossRefGoogle Scholar
- 11.EGEE JRA2. Job Metrics, http://egee-jra2.web.cern.ch/EGEE-JRA2/QoS/JobsMetrics/JobMetrics.htm
- 12.Rajbhandari, S., Walker, D.W.: Support for Provenance in a Service-based Computing Grid. In: Proceedings of the third UK e-Science All Hands Meeting, Nottingham, UK (2004)Google Scholar
- 13.Townend, P., Groth, P., Xu, J.: A provenance-aware weighted fault tolerance scheme for service-based applications. In: Proceedings of the 8th IEEE International Symposium on Object-oriented Real-time distributed Computing (ISORC 2005) (2005)Google Scholar