A Data Stream Publish/Subscribe Architecture with Self-adapting Queries

  • Alasdair J. G. Gray
  • Werner Nutt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3760)


In data stream applications, streams typically arise from a geographically distributed collection of producers and may be queried by consumers, which may be distributed as well. In such a setting, a query can be seen as a subscription asking to be informed of all tuples that satisfy a specific condition. We propose to support the publishing and querying of distributed data streams by a publish/subscribe architecture.

To enable such a system to scale to a large number of producers and consumers requires the introduction of republishers which collect together data streams and make the merged stream available. If republishers consume from other republishers, a hierarchy of republishers results.

We present a formalism that allows distributed data streams, published by independent stream producers, to be integrated as views on a mediated schema. We use the formalism to develop methods to adapt query plans to changes in the set of available data streams and allow consumers to dynamically change which streams they subscribe to.


Data Stream Relevant Publisher Query Plan Continuous Query Local Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: Data-Stream Management: Processing High-Speed Data Streams. In: STREAM: The Stanford Data Stream Management System, Springer, Heidelberg (2005)Google Scholar
  2. 2.
    Carzaniga, A., Wolf, A.L.: Forwarding in a content-based network. In: Proceedings of ACM SIGCOMM, Karlsruhe (Germany), August 2003, pp. 163–174 (2003)Google Scholar
  3. 3.
    Cooke, A., Gray, A.J.G., Nutt, W.: Stream integration techniques for grid monitoring. Journal on Data Semantics 2, 136–175 (2005)CrossRefGoogle Scholar
  4. 4.
    Cooke, A.W., Gray, A.J.G., Nutt, W., Magowan, J., Oevers, M., Taylor, P., Cordenonsi, R., Byrom, R., Cornwall, L., Djaoui, A., Field, L., Fisher, S.M., Hicks, S., Leake, J., Middleton, R., Wilson, A., Zhu, X., Podhorszki, N., Coghlan, B., Kenny, S., O’Callaghan, D., Ryan, J.: The relational grid monitoring architecture: Mediating information about the grid. Journal of Grid Computing 2(4), 323–339 (2004)CrossRefGoogle Scholar
  5. 5.
    Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid information services for distributed resource sharing. In: 10th International Symposium on High Performance Distributed Computing, June 2001, pp. 181–194. IEEE Computer Society, Los Alamitos (2001)CrossRefGoogle Scholar
  6. 6.
    Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.: The many faces of publish/subscribe. ACM Computing Surveys 35(2), 114–131 (2003)CrossRefGoogle Scholar
  7. 7.
    Fiege, L., Mühl, G., Gärtner, F.C.: A modular approach to build structured event-based systems. In: Proc. 2002 ACM Symposium on Applied Computing, Madrid (Spain), March 2002, pp. 385–392. ACM Press, New York (2002)CrossRefGoogle Scholar
  8. 8.
    Gray, A.J.G., Nutt, W.: Republishers in a publish/subscribe architecture for data streams. In: Jackson, M., Nelson, D., Stirk, S. (eds.) BNCOD 2005. LNCS, vol. 3567, pp. 179–184. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Halevy, A.: Answering queries using views: A survey. The VLDB Journal 10(4), 270–294 (2001)zbMATHCrossRefGoogle Scholar
  10. 10.
    Madden, S.R., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data. In: Proc. 18th International Conference on Data Engineering, San Jose, CA, USA, pp. 555–566. IEEE Computer Society, Los Alamitos (2002)CrossRefGoogle Scholar
  11. 11.
    Plale, B., Schwan, K.: Dynamic querying of streaming data with the dQUOB system. IEEE Transactions on Parallel and Distributed Systems 14(3), 422–432 (2003)CrossRefGoogle Scholar
  12. 12.
    Stegmaier, B., Kuntschke, R., Kemper, A.: StreamGlobe: Adaptive query processing and optimization in streaming P2P environments. In: Proc. of the 1st International Workshop on Data Management for Sensor Networks, Toronto (Canada), August 2004. VLDB, pp. 88–97 (2004)Google Scholar
  13. 13.
    Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A Grid monitoring architecture. In: Global Grid Forum Performance Working Group (January 2002) (revised)Google Scholar
  14. 14.
    Wiederhold, G.: Mediators in the architecture of future information systems. IEEE Computer 25(3), 38–49 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Alasdair J. G. Gray
    • 1
  • Werner Nutt
    • 1
  1. 1.School of Mathematical and Computer SciencesHeriot-Watt UniversityEdinburghUK

Personalised recommendations