Abstract
We present a novel load shedding technique over sliding window joins. We first construct a dual window architectural model including join-windows and aux-windows. With the statistics built on aux-windows, an effective load shedding strategy is developed to produce maximum subset join outputs. For the streams with high arrival rates, we propose an approach incorporating front-shedding and rear-shedding, and then address the problem of how to cooperate these two shedding processes through a series of calculations. Based on extensive experimentation with synthetic data and real life data, we show that our load shedding strategy delivers superb join output performance, and dominates the existing strategies.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Kang, J., Naughton, J.F., Viglas, S.D.: Evaluating Window Joins over Unbounded Streams. In: Proc. 2003 Intl. Conf. on Data Engineering (March 2003)
The STREAM Group: STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin 26(1), 19–26 (2003)
Ayad, A.M., Naughton, J.F.: Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams. In: Proc. ACM SIGMOD Conf. (June 2004)
Das, A., Gehrke, J., Riedewald, M.: Approximate Join Processing Over Data Streams. In: Proc. 2003 ACM SIGMOD Conf. (June 2003)
Srivastava, U., Widom, J.: Memory-Limited Execution of Windowed Stream Joins. In: Proc. 30th Int. Conf. on Very Large Data Bases (2004)
Abadi, D., Carney, D., et al.: Aurora: a new model and architecture for data stream management. VLDB Journal 12(2), 120–139 (2003)
Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A scalable continous query system for internet databasses. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 379–390 (2000)
Fenwich, P.M.: A New Data Structure for Cumulative Frequency Tables. Software - Practice and Experience 24(3), 327–336 (1994)
Hellerstein, J.M., Franklin, M.J., Chandrasekaran, S., et al.: Adaptive query processing: Technology in evolution. IEEE data Engineering Bulletin 23(2), 7–18 (2000)
Baldocchi, D., Wilson, K., et al.: Half-Hourly Measurements of CO2, Water Vapor, and Energy Exchange Using the Eddy Covariance Technique from Walker Branch Watershed, Tennessee (1995-1998), http://cdiac.esd.ornl.gov/ftp/ameriflux/data/us-sites/walker-branch/
Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proc. Principles of Database Systems (PODS) (June 2002)
Golab, L., Ozmu, M.T.: Processing Sliding Window Multi-joins in Continuous Queries over Data Streams. In: Proc. Conf. on Very Large Databases (September 2003)
Viglas, S.D., Naughton, J.F., Burger, J.: Maximizing the Output Rate of Multi- Way Join Queries over Streaming Information Sources. In: Proc. Int. Conf. on Very Large Databases (VLDB) (September 2003)
Han, D., Zhou, R., Xiao, C.: Load shedding for Window Joins over Data Streams, Technical report, Northeastern University (June 2004), http://mitt.neu.edu.cn/publications/HZX05-Joins.pdf
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Han, D., Xiao, C., Zhou, R., Wang, G., Huo, H., Hui, X. (2006). Load Shedding for Window Joins over Streams. In: Yu, J.X., Kitsuregawa, M., Leong, H.V. (eds) Advances in Web-Age Information Management. WAIM 2006. Lecture Notes in Computer Science, vol 4016. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11775300_40
Download citation
DOI: https://doi.org/10.1007/11775300_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35225-9
Online ISBN: 978-3-540-35226-6
eBook Packages: Computer ScienceComputer Science (R0)