Encyclopedia of Database Systems

Living Edition
| Editors: Ling Liu, M. Tamer Özsu

Stream Processing on Modern Hardware

  • Buğra Gedik
Living reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7993-3_80758-1

Definition

Stream processing is a computational paradigm for on-the-fly analysis of live data at scale. Given the ever-increasing number of online data sources, rate of streaming data, and growing demand for timely analysis, stream processing has gained an important place in today’s data-driven solution architectures. A key feature of the stream processing paradigm is its amenability to parallel execution, which in turn makes stream processing an attractive domain for taking advantage of modern hardware.

To cope with the increased power consumption associated with frequency scaling, hardware manufacturers have moved toward processors and coprocessors that contain multiple cores. Such designs can provide increased computational capacity without having to work at high frequencies and thus do not suffer from high power consumption. Yet, taking advantage of such hardware requires additional effort on the software side.

In the context of stream processing systems, three major kinds of...

Keywords

Stream Processing Streaming Application Data Parallelism Task Parallelism Synchronous Data Flow 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Carney D, Çetintemel U, Cherniack M, Convey C, Lee S, Seidman G, Stonebraker M, Tatbul N, Zdonik BS. Monitoring streams – a new class of data management applications. In: Proceedings of the very large data bases conference (PVLDB); 2002. p. 215–26.Google Scholar
  2. 2.
    Cugola G, Margara A. Low latency complex event processing on parallel hardware. J Parallel Distrib Comput(JPDC). 2012;72(2):205–18.Google Scholar
  3. 3.
    Gedik B, Schneider S, Hirzel M, Wu K-L. Elastic scaling for data stream processing. IEEE Trans Parallel Distrib Syst (TPDS). 2014;25(6):1447–63.CrossRefGoogle Scholar
  4. 4.
    Gordon MI, Thies W, Amarasinghe S. Exploiting coarse-grained task, data, and pipeline parallelism in stream programs. In: Proceedings of the architectural support for programming languages and operating systems (ASPLOS); 2006. p. 151–62.Google Scholar
  5. 5.
    Hormati A, Kudlur M, Mahlke SA, Bacon DF, Rabbah MR. Optimus: efficient realization of streaming applications on FPGAs. In: Proceedings of the international conference on compilers, architecture, and synthesis for embedded systems (CASES); 2008. p. 41–50.Google Scholar
  6. 6.
    Karger D, Sherman A, Berkheimer A, Bogstad B, Dhanidina R. Web caching with consistent hashing. Comput Netw. 1999;31(11):1203–13.CrossRefGoogle Scholar
  7. 7.
    Karnagel T, Habich D, Schlegel B, Lehner W, The HELLS-join: a heterogeneous stream join for extremely large windows. In: Proceedings of the international workshop on data management on new hardware (DAMON); 2013.Google Scholar
  8. 8.
    Lee EA, Messerschmitt GD. Synchronous data flow. Proc IEEE. 1987;75(9):1235–45.CrossRefGoogle Scholar
  9. 9.
    Müller R, Teubner J, Alonso G. Data processing on FPGAs. Proc VLDB Endow (PVLDB). 2009;2(1):910–21.CrossRefGoogle Scholar
  10. 10.
    Müller R, Teubner J, Alonso G, Streams on wires – a query compiler for FPGAs. Proc VLDB Endow (PVLDB). 2009;21(1):229–40.CrossRefGoogle Scholar
  11. 11.
    Sadoghi M, Javed R, Tarafdar N, Singh H, Palaniappan R, Jacobsen H-A. Multi-query stream processing on FPGAs. In: Proceedings of the international conference on data engineering (ICDE); 2012. p. 1229–32.Google Scholar
  12. 12.
    Schneider S, Andrade H, Gedik B, Wu K-L, Nikolopoulos D. Evaluation of streaming aggregation on parallel hardware architectures. In: Proceedings of the fourth ACM international conference on distributed event-based systems, DEBS 2010, Cambridge; 12–15 July 2010. p. 248–57.Google Scholar
  13. 13.
    Tang Y, Gedik B. Auto-pipelining for data stream processing. IEEE Trans Parallel Distrib Syst (TPDS). 2013; 24(12):2344–54.CrossRefGoogle Scholar
  14. 14.
    Udupa A, Govindarajan R, Thazhuthaveetil JM. Software pipelined execution of stream programs on GPUs. In: Proceedings of the IEEE/ACM international symposium on code generation and optimization (CGO); 2009. p. 200–9.Google Scholar
  15. 15.
    Zhang Y, Mueller F. GStream: a general-purpose data streaming framework on GPU clusters. In: Proceedings of the international conference on parallel processing (ICPP); 2011. p. 245–54.Google Scholar

Copyright information

© Springer Science+Business Media LLC 2017

Authors and Affiliations

  1. 1.Department of Computer EngineeringBilkent UniversityAnkaraTurkey

Section editors and affiliations

  • Ugur Cetintemel
    • 1
  1. 1.Department of Computer ScienceBrown UniversityProvidenceUSA