Advertisement

Load Shedding for Window Joins over Streams

  • Donghong Han
  • Chuan Xiao
  • Rui Zhou
  • Guoren Wang
  • Huan Huo
  • Xiaoyun Hui
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4016)

Abstract

We present a novel load shedding technique over sliding window joins. We first construct a dual window architectural model including join-windows and aux-windows. With the statistics built on aux-windows, an effective load shedding strategy is developed to produce maximum subset join outputs. For the streams with high arrival rates, we propose an approach incorporating front-shedding and rear-shedding, and then address the problem of how to cooperate these two shedding processes through a series of calculations. Based on extensive experimentation with synthetic data and real life data, we show that our load shedding strategy delivers superb join output performance, and dominates the existing strategies.

Keywords

Window Size Queue Length Heuristic Information Stream Speed Zipfian Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kang, J., Naughton, J.F., Viglas, S.D.: Evaluating Window Joins over Unbounded Streams. In: Proc. 2003 Intl. Conf. on Data Engineering (March 2003)Google Scholar
  2. 2.
    The STREAM Group: STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin 26(1), 19–26 (2003)Google Scholar
  3. 3.
    Ayad, A.M., Naughton, J.F.: Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams. In: Proc. ACM SIGMOD Conf. (June 2004)Google Scholar
  4. 4.
    Das, A., Gehrke, J., Riedewald, M.: Approximate Join Processing Over Data Streams. In: Proc. 2003 ACM SIGMOD Conf. (June 2003)Google Scholar
  5. 5.
    Srivastava, U., Widom, J.: Memory-Limited Execution of Windowed Stream Joins. In: Proc. 30th Int. Conf. on Very Large Data Bases (2004)Google Scholar
  6. 6.
    Abadi, D., Carney, D., et al.: Aurora: a new model and architecture for data stream management. VLDB Journal 12(2), 120–139 (2003)CrossRefGoogle Scholar
  7. 7.
    Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A scalable continous query system for internet databasses. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 379–390 (2000)Google Scholar
  8. 8.
    Fenwich, P.M.: A New Data Structure for Cumulative Frequency Tables. Software - Practice and Experience 24(3), 327–336 (1994)CrossRefGoogle Scholar
  9. 9.
    Hellerstein, J.M., Franklin, M.J., Chandrasekaran, S., et al.: Adaptive query processing: Technology in evolution. IEEE data Engineering Bulletin 23(2), 7–18 (2000)Google Scholar
  10. 10.
    Baldocchi, D., Wilson, K., et al.: Half-Hourly Measurements of CO2, Water Vapor, and Energy Exchange Using the Eddy Covariance Technique from Walker Branch Watershed, Tennessee (1995-1998), http://cdiac.esd.ornl.gov/ftp/ameriflux/data/us-sites/walker-branch/
  11. 11.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proc. Principles of Database Systems (PODS) (June 2002)Google Scholar
  12. 12.
    Golab, L., Ozmu, M.T.: Processing Sliding Window Multi-joins in Continuous Queries over Data Streams. In: Proc. Conf. on Very Large Databases (September 2003)Google Scholar
  13. 13.
    Viglas, S.D., Naughton, J.F., Burger, J.: Maximizing the Output Rate of Multi- Way Join Queries over Streaming Information Sources. In: Proc. Int. Conf. on Very Large Databases (VLDB) (September 2003)Google Scholar
  14. 14.
    Han, D., Zhou, R., Xiao, C.: Load shedding for Window Joins over Data Streams, Technical report, Northeastern University (June 2004), http://mitt.neu.edu.cn/publications/HZX05-Joins.pdf

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Donghong Han
    • 1
  • Chuan Xiao
    • 1
  • Rui Zhou
    • 1
  • Guoren Wang
    • 1
  • Huan Huo
    • 1
  • Xiaoyun Hui
    • 1
  1. 1.Northeastern UniversityShenyangChina

Personalised recommendations