Advertisement

STREAM: The Stanford Data Stream Management System

  • Arvind Arasu
  • Brian Babcock
  • Shivnath Babu
  • John Cieslewicz
  • Mayur Datar
  • Keith Ito
  • Rajeev Motwani
  • Utkarsh Srivastava
  • Jennifer Widom
Chapter
Part of the Data-Centric Systems and Applications book series (DCSA)

Abstract

Traditional database management systems are best equipped to run one-time queries over finite stored data sets. However, many modern applications such as network monitoring, financial analysis, manufacturing, and sensor networks require long-running, or continuous, queries over continuous unbounded streams of data. In the STREAM project at Stanford, we are investigating data management and query processing for this class of applications. As part of the project we are building a general-purpose prototype Data Stream Management System (DSMS), also called STREAM, that supports a large class of declarative continuous queries over continuous streams and traditional stored data sets. The STREAM prototype targets environments where streams may be rapid, stream characteristics and query loads may vary over time, and system resources may be limited.

Keywords

Data Stream Queue Size Input Stream Query Plan Continuous Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    M.K. Aguilera, R.E. Strom, D.C. Sturman, M. Astley, T.D. Chandra, Matching events in a content-based subscription system, in Proc. of the 18th Annual ACM Symp. on Principles of Distributed Computing (1999), pp. 53–61 Google Scholar
  2. 2.
    A. Arasu, B. Babcock, S. Babu, J. McAlister, J. Widom, Characterizing memory requirements for queries over continuous data streams. ACM Trans. Database Syst. 29(1), 1–33 (2004) CrossRefGoogle Scholar
  3. 3.
    A. Arasu, S. Babu, J. Widom, The CQL continuous query language: semantic foundations and query execution. VLDB J. 15(2), 121–142 (2006) CrossRefGoogle Scholar
  4. 4.
    B. Babcock, S. Babu, M. Datar, R. Motwani, Chain: operator scheduling for memory minimization in data stream systems, in Proc. of the 2003 ACM SIGMOD Intl. Conf. on Management of Data (2003), pp. 253–264 CrossRefGoogle Scholar
  5. 5.
    B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in Proc. of the 21st ACM SIGACT–SIGMOD–SIGART Symp. on Principles of Database Systems (2002), pp. 1–16 Google Scholar
  6. 6.
    B. Babcock, M. Datar, R. Motwani, Load shedding for aggregation queries over data streams, in Proc. of the 20th Intl. Conf. on Data Engineering (2004) Google Scholar
  7. 7.
    S. Babu, R. Motwani, K. Munagala, I. Nishizawa, J. Widom, Adaptive ordering of pipelined stream filters, in Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data (2004) Google Scholar
  8. 8.
    S. Babu, K. Munagala, J. Widom, R. Motwani, Adaptive caching for continuous queries, in Proc. of the 21st Intl. Conf. on Data Engineering (2005), pp. 118–129 Google Scholar
  9. 9.
    S. Babu, U. Srivastava, J. Widom, Exploiting \(k\)-constraints to reduce memory overhead in continuous queries over data streams. ACM Trans. Database Syst. 29(3), 545–580 (2004) CrossRefGoogle Scholar
  10. 10.
    S. Babu, J. Widom, StreaMon: an adaptive engine for stream query processing, in Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data (2004). Demonstration description Google Scholar
  11. 11.
    B.H. Bloom, Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970) CrossRefzbMATHGoogle Scholar
  12. 12.
    K. Chakrabarti, M.N. Garofalakis, R. Rastogi, K. Shim, Approximate query processing using wavelets, in Proc. of the 26th Intl. Conf. on Very Large Data Bases (2000), pp. 111–122 Google Scholar
  13. 13.
    J. Gehrke (ed.), Data stream processing. IEEE Comput. Soc. Bull. Technical Comm. Database Eng. 26(1) (2003) Google Scholar
  14. 14.
    F. Fabret, H.-.A. Jacobsen, F. Llirbat, J. Pereira, K.A. Ross, D. Shasha, Filtering algorithms and implementation for very fast publish/subscribe, in Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data (2001), pp. 115–126 Google Scholar
  15. 15.
    R.E. Gruber, B. Krishnamurthy, E. Panagos, READY: a high performance event notification system, in Proc. of the 16th Intl. Conf. on Data Engineering (2000), pp. 668–669 Google Scholar
  16. 16.
    W. Hoeffding, Probability inequalities for sums of bounded random variables. J. Am. Stat. Soc. 58(301), 13–30 (1963) MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    U. Srivastava, J. Widom, Flexible time management in data stream systems, in Proc. of the 23rd ACM SIGACT-SIGMOD-SIGART Symp. on Principles of Database Systems (2004) Google Scholar
  18. 18.
    N. Tatbul, U. Cetintemel, S.B. Zdonik, M. Cherniak, M. Stonebraker, Load shedding in a data stream manager, in Proc. of the 29th Intl. Conf. on Very Large Data Bases (2003), pp. 309–320 Google Scholar
  19. 19.
    N. Thaper, S. Guha, P. Indyk, N. Koudas, Dynamic multidimensional histograms, in Proc. of the 2002 ACM SIGMOD Intl. Conf. on Management of Data (2002), pp. 428–439 CrossRefGoogle Scholar
  20. 20.
    D. Thomas, R. Motwani, Caching queues in memory buffers, in Proc. of the 15th Annual ACM–SIAM Symp. on Discrete Algorithms (2004) Google Scholar
  21. 21.
    P.A. Tucker, D. Maier, T. Sheard, L. Fegaras, Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng. 15(3), 555–568 (2003) CrossRefGoogle Scholar
  22. 22.
    S. Viglas, J.F. Naughton, J. Burger, Maximizing the output rate of multi-way join queries over streaming information sources, in Proc. of the 29th Intl. Conf. on Very Large Data Bases (2003), pp. 285–296 Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Arvind Arasu
    • 1
  • Brian Babcock
    • 1
  • Shivnath Babu
    • 1
  • John Cieslewicz
    • 1
  • Mayur Datar
    • 1
  • Keith Ito
    • 1
  • Rajeev Motwani
    • 1
  • Utkarsh Srivastava
    • 1
  • Jennifer Widom
    • 1
  1. 1.Department of Computer ScienceStanford UniversityStanfordUSA

Personalised recommendations