Advertisement

The Anatomy of a Stream Processing System

  • Altaf Gilani
  • Satyajeet Sonune
  • Balakumar Kendai
  • Sharma Chakravarthy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4042)

Abstract

Data intensive applications such as network monitoring, financial applications; sensor-based applications etc. need to be supported by general-purpose systems rather than customized implementations. They have a continuous, unpredictable and unbounded flow of data as input, referred as streams. The fact that data comes as a stream with varying input rates (instead of accessing data stored on a disk in a predictable way) and that quality of service (QoS) requirements are stringent for these applications warrants a re-examination of the fundamental architecture of a DBMS. This paper describes the basic processing model and architecture of MavStream – a new Data Stream Management System (DSMS) being developed at UT Arlington. The architecture of MavStream is the primary focus of this paper. The user can give a continuous query from a graphical user interface (GUI), which is instantiated, scheduled, and executed by the MavStream server. We first provide an overview of the basic model and architecture and then describe some of the components of the system. We provide some experimental results to demonstrate the utility of the system and the effect of different scheduling strategies and buffer sizes on the performance and output.

Keywords

Query Processing Schedule Strategy Buffer Size Input Buffer Buffer Management 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring Streams- A new class of data management applications. In: Proc. of the 28th Intl. Conf. on VLDB (2002)Google Scholar
  2. 2.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and Issues in Data Stream Systems. In: Proc. of PODS (2002) (Invited paper)Google Scholar
  3. 3.
    Chandrasekharan, S., Franklin, M.J.: Streaming Queries over Streaming Data. In: Proc. of the 28th Intl. Conf. VLDB, pp. 203–214 (2002)Google Scholar
  4. 4.
    Chandrasekharan, S., Cooper, O., Deshpande, A., Franklin, M.J., Hellerstein, J.M., Hong, W., Krishnamurthy, S., Madden, S., Raman, V., Reiss, F., Shah, M.A.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: Proc. of CIDR (2003)Google Scholar
  5. 5.
    Avnur, R., Hellerstein, J.M.: Eddies: Continuously adaptive query processing. In: Proceedings of ACM SIGMOD, pp. 261–272 (2000)Google Scholar
  6. 6.
    Madden, S., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data. In: Proc. of the Intl. Conf. on Data Engineering, pp. 555–566 (2002)Google Scholar
  7. 7.
    Bonnet, P., Gehrke, J., Seshadri, P.: Towards sensor database systems. In: Tan, K.-L., Franklin, M.J., Lui, J.C.-S. (eds.) MDM 2001. LNCS, vol. 1987, pp. 3–14. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A scalable continuous query system for internet databases. In: Proc. of ACM SIGMOD (2000)Google Scholar
  9. 9.
    Abadi, D., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: A New Model and Architecture for Data Stream Management. VLDB Journal 2(12), 120–139 (2003)Google Scholar
  10. 10.
    Carney, D., Çetintemel, U., Rasin, A., Zdonik, S., Cherniack, M., Stonebraker, M.: Operator Scheduling in a Data Stream Manager. In: Proc. of VLDB (2003)Google Scholar
  11. 11.
    The STREAM Group. STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin (March 2003)Google Scholar
  12. 12.
    Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G., Olston, C., Rosenstein, J., Varma, R.: Query Processing Resource Management and Approximation in a Data Stream Management System. In: Proc. of CIDR (2003)Google Scholar
  13. 13.
    Gilani, A.: Design and implementation of stream operators, query instantiator and stream buffer manager. MS Thesis, CSE Department, UT Arlington (2003) [online], http://www.cse.uta.edu/research/publications/Downloads/CSE-2003-37.pdf
  14. 14.
    Sonune, S.: Design and implementation of windowed operators and scheduler for stream data. MS Thesis, CSE Department, University of Texas at Arlington (2003) [online], http://www.cse.uta.edu/research/publications/Downloads/CSE-2003-38.pdf
  15. 15.
    Cook, D.J., et al.: MavHome: An Agent-Based Smart Home. In: Proc. of the Conference on Pervasive Computing (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Altaf Gilani
    • 1
  • Satyajeet Sonune
    • 1
  • Balakumar Kendai
    • 1
  • Sharma Chakravarthy
    • 1
  1. 1.Information Technology Laboratory, and Department of Computer Science and EngineeringThe University of Texas at Arlington 

Personalised recommendations