Transactional Stream Processing
We can broadly define transactional stream processing as processing streaming data with correctness guarantees. These guarantees include not only properties that are intrinsic to stream processing (e.g., order, exactly-once semantics), but also ACID properties of traditional OLTP-oriented databases, which arise in streaming applications in case of shared mutable state or failures.
Stream processing emerged as a research area in the database community circa early 2000s. The initial focus of the community was on enabling relational-style query processing over ordered and unbounded data from push-based data sources such as sensors. New models, algorithms, and systems were developed to achieve low-latency continuous processing over streams arriving at high or unpredictable rates. Storing streaming data for longer term use beyond answering real-time continuous queries was not a primary concern. Thus, storage management was limited to...
- 1.Hwang JH, Balazinska M, Rasin A, Cetintemel U, Stonebraker M, Zdonik S. High-availability algorithms for distributed stream processing. In: ICDE; 2005. p. 779–90.Google Scholar
- 2.Toshniwal A, Taneja S, Shukla A, Ramasamy K, Patel JM, Kulkarni S, et al. Storm @Twitter. In: SIGMOD; 2014. p. 147–56.Google Scholar
- 3.Zaharia M, Das T, Li H, Hunter T, Shenker S, Stoica I. Discretized streams: fault-tolerant streaming computation at scale. In: SOSP; 2013. p. 423–38.Google Scholar
- 4.Chandramouli B, Goldstein J, Barnett M, DeLine R, Fisher D, Platt JC, et al. Trill: a high-performance incremental query processor for diverse analytics. PVLDB. 2014;8(4):401–12.Google Scholar
- 5.Akidau T, Bradshaw R, Chambers C, Chernyak S, Fernandez-Moctezuma RJ, Lax R, et al. The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. PVLDB. 2015;8(12): 1792–1803.Google Scholar
- 6.Meehan J, Tatbul N, Zdonik S, Aslantas C, Cetintemel U, Du J, et al. S-store: streaming meets transaction processing. PVLDB. 2015;8(13): 2134–45.Google Scholar
- 7.Ramnarayan J, Mozafari B, Wale S, Menon S, Kumar N, Bhanawat H, et al. SnappyData: streaming, transactions, and interactive analytics in a unified engine. In: SIGMOD; (2016, to appear).Google Scholar
- 9.Golab L, Bijay KG, Ozsu MT. On concurrency control in sliding window queries over data streams. In: EDBT; 2006. p. 608–26.Google Scholar
- 10.Wang D, Rundensteiner EA, Ellison RT. Active complex event processing over event streams. PVLDB. 2011;4(10):634–45.Google Scholar
- 11.Balazinska M, Balakrishnan H, Madden SR, Stonebraker M. Fault-tolerance in the Borealis distributed stream processing system. ACM TODS. 2008;33(1):3:1–3:44.Google Scholar
- 12.Akidau T, Balikov A, Bekiroglu K, Chernyak S, Haberman J, Lax R, et al. MillWheel: fault-tolerant stream processing at Internet scale. PVLDB. 2013;6(11):734–46.Google Scholar
- 13.Botan I, Fischer PM, Kossmann D, Tatbul N. Transactional stream processing. In: EDBT; 2012. p. 204–15.Google Scholar
- 14.Kreps J, Narkhede N, Rao J. Kafka: a distributed messaging system for log processing. In: NetDB workshop; 2011.Google Scholar