Synonyms
Adaptive query processing
Definition
When querying long-lived data streams, the characteristics of the data may change over time or data may arrive in bursts – hence, the traditional model of optimizing a query prior to executing it is insufficient. As a result, most data stream management systems employ feedback-driven adaptive stream processing, which continuously re-optimizes the query execution plan based on data and stream properties, in order to meet certain performance or resource consumption goals. Adaptive stream processing is a special case of the more general problem of adaptive query processing, with the special property that intermediate results are bounded in size (by stream windows), but where query processing may have quality-of-service constraints.
Historical Background
The field of adaptive stream processing emerged in the early 2000s, as two separate developments converged. Adaptivetechniques for database query processing had become an area of increasing...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Recommended Reading
Abadi DJ, Carney D, Cetintemel U, Cherniack M, Convey C, Lee S, Stonebraker M, Tatbul N, Zdonik S. Aurora: a new model and architecture for data stream management. VLDB J. 2003;12(2):120–39.
Avnur R, Hellerstein JM. Eddies: continuously adaptive query processing. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2000. p. 261–72.
Babcock B, Babu S, Datar M, Motwani R. Chain: operator scheduling for memory minimization in data stream systems. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 253–64.
Babcock B, Datar M, Motwani R. Load shedding for aggregation queries over data streams. In: Proceedings of the 20th International Conference on Data Engineering; 2004. p. 350.
Babu S, Motwani R, Munagala K, Nishizawa I, Widom J. Adaptive ordering of pipelined stream filters. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2004. p. 407–18.
Balazinska M, BalaKrishnan H, Stonebraker M. Demonstration: load management and high availability in the Medusa distributed stream processing system. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2004. p. 929–30.
Bizarro P, Babu S, De Witt DJ, Widom J. Content-based routing: different plans for different data In: Proceedings of the 31st International Conference on Very Large Data; 2005. p. 757–68.
Chandrasekaran S, Cooper O, Deshpande A, Franklin MJ, Hellerstein JM, Hong W, Krishnamurthy S, Madden S, Raman V, Reiss F, Shah MA. TelegraphCQ: continuous dataflow processing for an uncertain world. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research; 2003.
Deshpande A. An initial study of overheads of eddies. ACM SIGMOD Rec. 2004;33(1):44–9.
Deshpande A, Ives Z, Raman V. Adaptive query processing. Found. Trends Databases. 2007;1(1):1–140.
Madden S, Shah MA, Hellerstein JM, Raman V. Continuously adaptive continuous queries over streams. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2002. p. 49–60.
Motwani R, Widom J, Arasu A, Babcock B, Babu S, Datar M, Manku G, Olston C, Rosenstein J, Varma R. Query processing, resource management, and approximation in a data stream management system. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research; 2003.
Olston C, Jiang J, Widom J. Adaptive filters for continuous queries over distributed data streams. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 563–74.
Raman V, Deshpande A, Hellerstein JM. Using state modules for adaptive query processing. In: Proceedings of the 19th International Conference on Data Engineering; 2003. p. 353–66.
Tatbul N, Cetintemel U, Zdonik SB, Cherniack M, Stonebraker M. Load shedding in a data stream manager. In: Proceedings of the 29th International Conference on Very Large Data Bases; 2003. p. 309–20.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer Science+Business Media, LLC, part of Springer Nature
About this entry
Cite this entry
Ives, Z.G. (2018). Adaptive Stream Processing. In: Liu, L., Özsu, M.T. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8265-9_11
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8265-9_11
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8266-6
Online ISBN: 978-1-4614-8265-9
eBook Packages: Computer ScienceReference Module Computer Science and Engineering