A Data Stream Publish/Subscribe Architecture with Self-adapting Queries
In data stream applications, streams typically arise from a geographically distributed collection of producers and may be queried by consumers, which may be distributed as well. In such a setting, a query can be seen as a subscription asking to be informed of all tuples that satisfy a specific condition. We propose to support the publishing and querying of distributed data streams by a publish/subscribe architecture.
To enable such a system to scale to a large number of producers and consumers requires the introduction of republishers which collect together data streams and make the merged stream available. If republishers consume from other republishers, a hierarchy of republishers results.
We present a formalism that allows distributed data streams, published by independent stream producers, to be integrated as views on a mediated schema. We use the formalism to develop methods to adapt query plans to changes in the set of available data streams and allow consumers to dynamically change which streams they subscribe to.
KeywordsData Stream Relevant Publisher Query Plan Continuous Query Local Query
Unable to display preview. Download preview PDF.
- 1.Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: Data-Stream Management: Processing High-Speed Data Streams. In: STREAM: The Stanford Data Stream Management System, Springer, Heidelberg (2005)Google Scholar
- 2.Carzaniga, A., Wolf, A.L.: Forwarding in a content-based network. In: Proceedings of ACM SIGCOMM, Karlsruhe (Germany), August 2003, pp. 163–174 (2003)Google Scholar
- 4.Cooke, A.W., Gray, A.J.G., Nutt, W., Magowan, J., Oevers, M., Taylor, P., Cordenonsi, R., Byrom, R., Cornwall, L., Djaoui, A., Field, L., Fisher, S.M., Hicks, S., Leake, J., Middleton, R., Wilson, A., Zhu, X., Podhorszki, N., Coghlan, B., Kenny, S., O’Callaghan, D., Ryan, J.: The relational grid monitoring architecture: Mediating information about the grid. Journal of Grid Computing 2(4), 323–339 (2004)CrossRefGoogle Scholar
- 12.Stegmaier, B., Kuntschke, R., Kemper, A.: StreamGlobe: Adaptive query processing and optimization in streaming P2P environments. In: Proc. of the 1st International Workshop on Data Management for Sensor Networks, Toronto (Canada), August 2004. VLDB, pp. 88–97 (2004)Google Scholar
- 13.Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A Grid monitoring architecture. In: Global Grid Forum Performance Working Group (January 2002) (revised)Google Scholar
- 14.Wiederhold, G.: Mediators in the architecture of future information systems. IEEE Computer 25(3), 38–49 (1992)Google Scholar