High Efficient Complex Event Processing Based on Storm
In recent years there is a huge increase in real-time data, which cannot be processed efficiently. Complex event processing has become a very important method to get meaningful information. However, supporting complex event detection in multiple sources environments is a challenging problem. To allow for inferring high level information from vast amounts of continuous arriving data. In this paper, we present a complex event processing system based on a novel distributed computing platform Storm, which goes further than distributing queries and achieves better scalability by parallelizing event detection, and also higher efficiency through the use of some optimizations. The experimental shows that the event processing system is effective and better scalability.
KeywordsComplex event processing Event detection Distributed processing
This project is sponsored by Hunan Provincial Natural Science Foundation of China “Context-aware and proactive complex event processing for large scale internet of things (13JJ3046)” and supported by the “complex event processing in large scale internet of things (K120326-11)” project of Changsha technological plan.
- 1.Luckham DC (2001) The power of events: an introduction to complex event processing in distributed enterprise systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USAGoogle Scholar
- 2.Zimmer D, Unland R (1999) On the semantics of complex events in active database management systems. In: ICDE, p 392–399Google Scholar
- 3.Wu E, Diao Y, Rizvi S (2006) High-performance complex event processing over streams. In: SIGMOD conference, p 407–418Google Scholar
- 4.Barga RS, Goldstein J, Ali MH, Hong M (2007) Consistent streaming through time: a vision for event stream processing. In: CIDR, p 363–374Google Scholar
- 5.Li G, Jacobsen HA (2005) Composite subscriptions in content-based publish/subscribe systems. In: Middleware’05. Springer, New YorkGoogle Scholar
- 6.Schultz-Moeller NP, Migliavacca M, Pietzuch P (2009) Distributed complex event processing with query optimization. In: DEBS’09. ACM, NashvilleGoogle Scholar
- 7.Demers AJ, Gehrke J, Panda B, Riedewald M, Sharma V, White WM (2007) Cayuga: a general purpose event monitoring system. In: CIDR, p 412–422Google Scholar
- 8.Chen J, DeWitt DJ, Tian F, Wang Y (2000) Niagara CQ: a scalable continuous query system for internet databases. ACM SIGMOD Record 29(2):390Google Scholar
- 9.Mert Akdere, Ugur Çetintemel, Nesime Tatbul, Plan-based complex event detection across distributed sources, Proceedings of the VLDB Endowment, vol 1(1), Aug 2008Google Scholar
- 10.Arasu A, Babcock B, Babu S, Cieslewicz J, Datar M, Ito K, Motwani R, Srivastava U, Widom J (2004) STREAM: the Stanford data stream management system. In: Garofalakis, Gehrke, and Rastogi (eds) A book on data stream managementGoogle Scholar
- 11.Arasu A, Babu S, Widom J. The CQL continuous query language: semantic foundations and query execution. Technical report, Stanford UniversityGoogle Scholar
- 12.Marz N. Storm wiki. URL https://github.com/nathanmarz/storm/wiki