Fast Approximate Wavelet Tracking on Streams

  • Graham Cormode
  • Minos Garofalakis
  • Dimitris Sacharidis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3896)

Abstract

Recent years have seen growing interest in effective algorithms for summarizing and querying massive, high-speed data streams. Randomized sketch synopses provide accurate approximations for general-purpose summaries of the streaming data distribution (e.g., wavelets). The focus of existing work has typically been on minimizing space requirements of the maintained synopsis — however, to effectively support high-speed data-stream analysis, a crucial practical requirement is to also optimize: (1) the update time for incorporating a streaming data element in the sketch, and (2) the query time for producing an approximate summary (e.g., the top wavelet coefficients) from the sketch. Such time costs must be small enough to cope with rapid stream-arrival rates and the real-time querying requirements of typical streaming applications (e.g., ISP network monitoring). With cheap and plentiful memory, space is often only a secondary concern after query/update time costs.

In this paper, we propose the first fast solution to the problem of tracking wavelet representations of one-dimensional and multi-dimensional data streams, based on a novel stream synopsis, the Group-Count Sketch (GCS). By imposing a hierarchical structure of groups over the data and applying the GCS, our algorithms can quickly recover the most important wavelet coefficients with guaranteed accuracy. A tradeoff between query time and update time is established, by varying the hierarchical structure of groups, allowing the right balance to be found for specific data stream. Experimental analysis confirms this tradeoff, and shows that all our methods significantly outperform previously known methods in terms of both update time and query time, while maintaining a high level of accuracy.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alon, N., Gibbons, P.B., Matias, Y., Szegedy, M.: Tracking join and self-join sizes in limited storage. In: ACM PODS (1999)Google Scholar
  2. 2.
    Alon, N., Matias, Y., Szegedy, M.: The space complexity of approximating the frequency moments. In: ACM STOC (1996)Google Scholar
  3. 3.
    Babcock, B., Babu, S., Datar, M., Motwani, R.: Jennifer Widom. “Models and issues in data stream systems”. In: ACM PODS (2002)Google Scholar
  4. 4.
    Chakrabarti, K., Garofalakis, M.N., Rastogi, R., Shim, K.: Approximate query processing using wavelets. In: VLDB (2000)Google Scholar
  5. 5.
    Charikar, M., Chen, K., Farach-Colton, M.: Finding frequent items in data streams. In: Widmayer, P., Triguero, F., Morales, R., Hennessy, M., Eidenbenz, S., Conejo, R. (eds.) ICALP 2002. LNCS, vol. 2380, p. 693. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Cormode, G., Muthukrishnan, S.: What’s hot and what’s not: Tracking most frequent items dynamically. In: ACM PODS (2003)Google Scholar
  7. 7.
    Cormode, G., Muthukrishnan, S.: What’s new: Finding significant differences in networkdata streams. In: IEEE Infocom (2004)Google Scholar
  8. 8.
    Deligiannakis, A., Roussopoulos, N.: Extended wavelets for multiple measures. In: ACM SIGMOD (2003)Google Scholar
  9. 9.
    Dobra, A., Garofalakis, M.N., Gehrke, J., Rastogi, R.: Processing complex aggregate queries over data streams. In: ACM SIGMOD (2002)Google Scholar
  10. 10.
    Garofalakis, M., Kumar, A.: Deterministic Wavelet Thresholding for Maximum-Error Metrics. In: ACM PODS (2004)Google Scholar
  11. 11.
    Gilbert, A., Kotidis, Y., Muthukrishnan, S., Strauss, M.: One-pass wavelet decomposition of data streams. IEEE TKDE 15(3) (2003)Google Scholar
  12. 12.
    Gilbert, A., Guha, S., Indyk, P., Kotidis, Y., Muthukrishnan, S., Strauss, M.: Fast, smallspace algorithms for approximate histogram maintenance. In: ACM STOC (2002)Google Scholar
  13. 13.
    Gilbert, A., Kotidis, Y., Muthukrishnan, S., Strauss, M.: How to summarize the universe: Dynamic maintenance of quantiles. In: VLDB (2002)Google Scholar
  14. 14.
    Guha, S., Harb, B.: Wavelet Synopsis for Data Streams: Minimizing non-Euclidean Error. In: KDD (2005)Google Scholar
  15. 15.
    Jahangiri, M., Sacharidis, D., Shahabi, C.: Shift-Split: I/O efficient maintenance of wavelet-transformed multidimensional data. In: ACM SIGMOD (2005)Google Scholar
  16. 16.
    Jawerth, B., Sweldens, W.: An Overview of Wavelet Based Multiresolution Analyses. SIAM Review 36(3) (1994)Google Scholar
  17. 17.
    Karras, P., Mamoulis, N.: One-pass wavelet synopses for maximum-error metrics. In: VLDB (2005)Google Scholar
  18. 18.
    Manku, G.S., Motwani, R.: Approximate frequency counts over data streams. In: VLDB (2002)Google Scholar
  19. 19.
    Matias, Y., Vitter, J.S., Wang, M.: Wavelet-based histograms for selectivity estimation. In: ACM SIGMOD (1998)Google Scholar
  20. 20.
    Muthukrishnan, S.: Data streams: algorithms and applications. In: SODA (2003)Google Scholar
  21. 21.
    Schmidt, R.R., Shahabi, C.: Propolyne: A fast wavelet-based technique for progressive evaluation of polynomial range-sum queries. In: Jensen, C.S., Jeffery, K., Pokorný, J., Šaltenis, S., Bertino, E., Böhm, K., Jarke, M. (eds.) EDBT 2002. LNCS, vol. 2287, p. 664. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  22. 22.
    Stollnitz, E.J., Derose, T.D., Salesin, D.H.: Wavelets for computer graphics: theory and applications. Morgan Kaufmann Publishers, San Francisco (1996)Google Scholar
  23. 23.
    Thaper, N., Guha, S., Indyk, P., Koudas, N.: Dynamic multidimensional histograms. In: ACM SIGMOD (2002)Google Scholar
  24. 24.
    Vitter, J.S., Wang, M.: Approximate computation of multidimensional aggregates of sparse data using wavelets. In: ACM SIGMOD (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Graham Cormode
    • 1
  • Minos Garofalakis
    • 2
  • Dimitris Sacharidis
    • 3
  1. 1.Bell LabsLucent Technologies 
  2. 2.Intel Research Berkeley 
  3. 3.National Technical University of Athens 

Personalised recommendations