Towards Low Overhead Provenance Tracking in Near Real-Time Stream Filtering
- Cite this paper as:
- Vijayakumar N.N., Plale B. (2006) Towards Low Overhead Provenance Tracking in Near Real-Time Stream Filtering. In: Moreau L., Foster I. (eds) Provenance and Annotation of Data. IPAW 2006. Lecture Notes in Computer Science, vol 4145. Springer, Berlin, Heidelberg
Data streams flowing from the physical environment are as unpredictable as the environment itself. Radars go down, long haul networks drop packets, and readings are corrupted on the wire. Yet the data driven scientific models and data mining algorithms do not necessarily account for the inaccuracies when assimilating the data. Low overhead provenance collection partially solves this problem. We propose a data model and collection model for near real time provenance collection. We define a system architecture for stream provenance tracking and motivate with a real-world application in meteorology forecasting.