A Multiple Continuous Query Optimization Method Based on Query Execution Pattern Analysis

  • Yousuke Watanabe
  • Hiroyuki Kitagawa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2973)


Many data streams are provided through the network today, and continuous queries are often used to extract useful information from data streams. When a system must process many queries continuously, query optimization is quite important for their efficient execution. In this paper, we propose a novel multiple query optimization method for continuous queries based on query execution pattern analysis. In the advanced stream processing environment assumed in the paper, we use window operators to specify time intervals to select information of interest and the execution time specification to designate when the query should be evaluated. Queries having the same operators may share many intermediate results when they are executed at close instants, but may involve only disjoint data when executed at completely different instants. Thus, query execution timing as well as common subexpressions is a key to deciding an efficient query execution plan. The basic idea of the proposed method is to identify query execution patterns from data arrival logs of data streams and to make the most of the information in deciding an efficient query execution plan. The proposed query optimization scheme first analyzes data arrival logs and extracts query execution patterns. It then forms clusters of continuous queries such that queries in the same cluster are likely to be executed at close instants. Finally, it extracts common subexpressions from among queries in each cluster and decides the query execution plan. We also show experiment results using the prototype implementation, and discuss effectiveness of the proposed approach.


Data Stream Data Arrival Average Response Time Total Processing Time Query Optimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Avnur, R., Hellerstein, J.M.: Eddies: Continuously Adaptive Query Processing. In: Proc. ACM SIGMOD, pp. 261–272 (2000)Google Scholar
  2. 2.
    Carney, D., et al.: Monitoring Streams – A New Class of Data Management Applictions. In: Proc. VLDB, pp. 215–226 (2002)Google Scholar
  3. 3.
    Chen, J., DeWitt, D.J., Naughton, J.F.: Design and Evaluation of Alternative Selection Placement Strategies in Optimizing Continuous Queries. In: Proc. ICDE, pp. 345–356 (2002)Google Scholar
  4. 4.
    Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A Scalable Continuous Query System for Internet Databases. In: Proc. ACM SIGMOD, pp. 379–390 (2000)Google Scholar
  5. 5.
    Chandrasekaran, S., Franklin, M.J.: Streaming Queries over Streaming Data. In: Proc. VLDB, pp. 203–214 (2002)Google Scholar
  6. 6.
    Gedik, B., Liu, L.: PeerCQ: A Decentralized and Self-Configuring Peer-to-Peer Informaiton Monitoring System. In: Proc. Intl. Conf. on Distributed Computing Systems, pp. 490–499 (2003)Google Scholar
  7. 7.
    Kang, J., Naughton, J.F., Viglas, S.D.: EvaluatingWindowJoins over Unbounded Streams. In: Proc. ICDE, pp. 341–352 (2003)Google Scholar
  8. 8.
    Liu, L., Pu, C., Tang, W.: Continual Queries for Internet Scale Event-Driven Information Delivery. IEEE TKDE 11(4), 610–628 (1999)Google Scholar
  9. 9.
    Madden, S., Franklin, M.J.: Fjording the Stream:AnArchitecture for Queries over Streaming Sensor Data. In: Proc. ICDE, pp. 555–566 (2002)Google Scholar
  10. 10.
    Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: The Design of an Acquisitional Query Processor For Sensor Networks. In: Proc. ACM SIGMOD, pp. 491–502 (2003)Google Scholar
  11. 11.
    Mistry, H., Roy, P., Sudarshan, S., Ramamritham, K.: Materialized View Selection and Maintenance Using Multi-Query Optimization. In: Proc. ACM SIGMOD, pp. 307–318 (2001)Google Scholar
  12. 12.
    Madden, S., Shah, M., Hellerstein, J.M., Raman, V.: Continuously Adaptive Continuous Queries over Streams. In: Proc. ACM SIGMOD, pp. 49–60 (2002)Google Scholar
  13. 13.
    Roy, P., Seshadri, S., Sudarshan, S., Bhobe, S.: Efficient and Extensible Algorithms for Multi Query Optimization. In: Proc. ACM SIGMOD, pp. 249–260 (2000)Google Scholar
  14. 14.
    Salton, G.: Automatic Information Organization and Retrieval. McGraw-Hill Book Company, New York (1968)Google Scholar
  15. 15.
    Sellis, T.K.: Multiple-Query Optimization. ACM TODS 13(1), 23–52 (1988)CrossRefGoogle Scholar
  16. 16.
    Tatbul, N., Cetintemel, U., Zdonik, S., Cherniack, M., Stonebraker, M.: Load Shedding in a Data Stream Manager. In: Proc. VLDB, pp. 309–320 (2003)Google Scholar
  17. 17.
    Terry, D., Goldberg, D., Nichols, D.: Continuous Queries over Append-Only Databases. In: Proc. ACM SIGMOD, pp. 321–330 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Yousuke Watanabe
    • 1
  • Hiroyuki Kitagawa
    • 2
  1. 1.Graduate School of Systems and Information EngineeringUniversity of Tsukuba 
  2. 2.Institute of Information Sciences and ElectronicsUniversity of Tsukuba 

Personalised recommendations