Advertisement

Dynamic Plan Migration for Snapshot-Equivalent Continuous Queries in Data Stream Systems

  • Jürgen Krämer
  • Yin Yang
  • Michael Cammert
  • Bernhard Seeger
  • Dimitris Papadias
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4254)

Abstract

A data stream management system executes a large number of continuous queries in parallel. As stream characteristics and query workload change over time, the plan initially installed for a continuous query may become inefficient. As a consequence, the query optimizer will re-optimize this plan based on the current statistics. The replacement of the running plan with a more efficient but semantically equivalent plan at runtime is called dynamic plan migration. In order to have a sound semantic foundation for query optimization, we investigate dynamic plan migration for snapshot-equivalent plans. We develop a general method for dynamic plan migration that treats the old and new plan as snapshot-equivalent black boxes. This enables the query optimizer to apply the conventional transformation rules during re-optimization. As a consequence, our approach supports the dynamic optimization of arbitrary continuous queries expressible in CQL, whereas existing solutions are limited in their scope.

Keywords

Window Operator Input Stream Query Plan Continuous Query Split Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zhu, Y., Rundensteiner, E.A., Heineman, G.T.: Dynamic Plan Migration for Continuous Queries Over Data Streams. In: Proc. of the ACM SIGMOD, pp. 431–442 (2004)Google Scholar
  2. 2.
    Krämer, J., Seeger, B.: A Temporal Foundation for Continuous Queries over Data Streams. In: Proc. of the Int. Conf. on Management of Data (COMAD), pp. 70–82 (2005)Google Scholar
  3. 3.
    Slivinskas, G., Jensen, C.S., Snodgrass, R.T.: Query Plans for Conventional and Temporal Queries Involving Duplicates and Ordering. In: Proc. of the IEEE Conference on Data Engineering (ICDE), pp. 547–558 (2000)Google Scholar
  4. 4.
    Böhlen, M.H., Busatto, R., Jensen, C.S.: Point-Versus Interval-Based Temporal Data Models. In: Proc. of the IEEE Conference on Data Engineering (ICDE), pp. 192–200 (1998)Google Scholar
  5. 5.
    Arasu, A., Babu, S., Widom, J.: An Abstract Semantics and Concrete Language for Continuous Queries over Streams and Relations. In: Proc. of the Int. Conf. on Data Base Programming Languages (DBPL), pp. 1–19 (2003)Google Scholar
  6. 6.
    Dayal, U., Goodman, N., Katz, R.H.: An Extended Relational Algebra with Control Over Duplicate Elimination. In: Proc. of the ACM SIGMOD, pp. 117–123 (1982)Google Scholar
  7. 7.
    Albert, J.: Algebraic Properties of Bag Data Types. In: Proc. of the Int. Conf. on Very Large Databases (VLDB), pp. 211–219 (1991)Google Scholar
  8. 8.
    Krämer, J., Seeger, B.: A Temporal Foundation for Continuous Queries over Data Streams. Technical report, University of Marburg, No. 45 (2004)Google Scholar
  9. 9.
    Hammad, M., Aref, W., Franklin, M., Mokbel, M., Elmagarmid, A.: Efficient Execution of Sliding Window Queries over Data Streams. Technical report, Purdue University, No. 35 (2003)Google Scholar
  10. 10.
    Golab, L., Özsu, M.T.: Issues in Data Stream Management. SIGMOD Record 32(2), 5–14 (2003)CrossRefGoogle Scholar
  11. 11.
    Srivastava, U., Widom, J.: Flexible Time Management in Data Stream Systems. In: Symp. on Principles of Database Systems (PODS), pp. 263–274 (2004)Google Scholar
  12. 12.
    Arasu, A., Babu, S., Widom, J.: The CQL Continuous Query Language: Semantic Foundations and Query Execution. Technical report, Stanford University, No. 57 (2003)Google Scholar
  13. 13.
    Galindo-Legaria, C., Joshi, M.: Orthogonal Optimization of Subqueries and Aggregation. In: Proc. of the ACM SIGMOD, pp. 571–581 (2001)Google Scholar
  14. 14.
    Babcock, B., Babu, S., Datar, M., Motwani, R.: Chain: Operator Scheduling for Memory Minimization in Data Stream Systems. In: Proc. of the ACM SIGMOD, pp. 253–264 (2003)Google Scholar
  15. 15.
    Carney, D., Cetintemel, U., Zdonik, S., Rasin, A., Cerniak, M., Stonebraker, M.: Operator Scheduling in a Data Stream Manager. In: Proc. of the Int. Conf. on Very Large Databases (VLDB), pp. 838–849 (2003)Google Scholar
  16. 16.
    Jiang, Q., Chakravarthy, S.: Scheduling Strategies for Processing Continuous Queries over Streams. In: Williams, H., MacKinnon, L.M. (eds.) BNCOD 2004. LNCS, vol. 3112, pp. 16–30. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Cammert, M., Krämer, J., Seeger, B., Vaupel, S.: An Approach to Adaptive Memory Management in Data Stream Systems. In: Proc. of the IEEE Conference on Data Engineering (ICDE), pp. 137–139 (2006)Google Scholar
  18. 18.
    Seeger, B.: Performance Comparison of Segment Access Methods Implemented on Top of the Buddy-Tree. In: Günther, O., Schek, H.-J. (eds.) SSD 1991. LNCS, vol. 525, pp. 277–296. Springer, Heidelberg (1991)Google Scholar
  19. 19.
    van den Bercken, J., Seeger, B.: Query Processing Techniques for Multiversion Access Methods. In: Proc. of the Int. Conf. on Very Large Databases (VLDB), pp. 168–179 (1996)Google Scholar
  20. 20.
    Krämer, J., Seeger, B.: PIPES - A Public Infrastructure for Processing and Exploring Streams. In: Proc. of the ACM SIGMOD, pp. 925–926 (2004)Google Scholar
  21. 21.
    Cammert, M., Heinz, C., Krämer, J., Riemenschneider, T., Schwarzkopf, M., Seeger, B., Zeiss, A.: Stream Processing in Production-to-Business Software. In: Proc. of the IEEE Conference on Data Engineering (ICDE), pp. 168–169 (2006)Google Scholar
  22. 22.
    Golab, L., Öszu, M.T.: Processing Sliding Window Multi-Joins in Continuous Queries over Data Streams. In: Proc. of the Int. Conf. on Very Large Databases (VLDB), pp. 500–511 (2003)Google Scholar
  23. 23.
    Babu, S., Bizarro, P.: Adaptive Query Processing in the Looking Glass. In: Proc. of the Conf. on Innovative Data Systems Research (CIDR), pp. 238–249 (2005)Google Scholar
  24. 24.
    Babu, S., Munagala, K., Widom, J., Motwani, R.: Adaptive Caching for Continuous Queries. In: Proc. of the IEEE Conference on Data Engineering (ICDE), pp. 118–129 (2005)Google Scholar
  25. 25.
    Babu, S., Bizarro, P., DeWitt, D.: Proactive Re-Optimization. In: Proc. of the ACM SIGMOD, pp. 107–118 (2005)Google Scholar
  26. 26.
    Deshpande, A., Hellerstein, J.M.: Lifting the Burden of History from Adaptive Query Processing. In: In: Proc. of the Int. Conf. on Very Large Databases (VLDB), pp. 948–959 (2004)Google Scholar
  27. 27.
    Abadi, D.J., Ahmad, Y., Balazinska, M., et al.: The Design of the Borealis Stream Processing Engine. In: Proc. of the Conf. on Innovative Data Systems Research (CIDR), pp. 277–289 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jürgen Krämer
    • 1
  • Yin Yang
    • 2
  • Michael Cammert
    • 1
  • Bernhard Seeger
    • 1
  • Dimitris Papadias
    • 2
  1. 1.University of MarburgGermany
  2. 2.Hong Kong University of Science and TechnologyChina

Personalised recommendations