Advertisement

D\(^2\)IA: Stream Analytics on User-Defined Event Intervals

Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11483)

Abstract

Nowadays, modern Big Stream Processing Solutions (e.g. Spark, Flink) are working towards ultimate frameworks for streaming analytics. In order to achieve this goal, they started to offer extensions of SQL that incorporate stream-oriented primitives such as windowing and Complex Event Processing (CEP). The former enables stateful computation on infinite sequences of data items while the latter focuses on the detection of events pattern. In most of the cases, data items and events are considered instantaneous, i.e., they are single time points in a discrete temporal domain. Nevertheless, a point-based time semantics does not satisfy the requirements of a number of use-cases. For instance, it is not possible to detect the interval during which the temperature increases until the temperature begins to decrease, nor all the relations this interval subsumes. To tackle this challenge, we present \(\texttt {D}^2{\texttt {IA}}\); a set of novel abstract operators to define analytics on user-defined event intervals based on raw events and to efficiently reason about temporal relationships between intervals and/or point events. We realize the implementation of the concepts of \(\texttt {D}^2{\texttt {IA}}\) on top of Esper, a centralized stream processing system, and Flink, a distributed stream processing engine for big data.

Keywords

Big Stream Processing Complex event processing User-defined event intervals 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Akidau, T., et al.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. PVLDB 8(12), 1792–1803 (2015)Google Scholar
  2. 2.
    Alharbi, A., Bulpitt, A., Johnson, O.: Improving pattern detection in healthcare process mining using an interval-based event selection method. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNBIP, vol. 297, pp. 88–105. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65015-9_6CrossRefGoogle Scholar
  3. 3.
    Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)CrossRefGoogle Scholar
  4. 4.
    Anicic, D., Rudolph, S., Fodor, P., Stojanovic, N.: Stream reasoning and complex event processing in ETALIS. Semant. Web 3(4), 397–407 (2012)Google Scholar
  5. 5.
    Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: semantic foundations and query execution. VLDB J. 15(2), 121–142 (2006)CrossRefGoogle Scholar
  6. 6.
    Barga, R.S., Goldstein, J., Ali, M.H., Hong, M.: Consistent streaming through time: a vision for event stream processing. In: CIDR, pp. 363–374 (2007)Google Scholar
  7. 7.
    Codd, E.F.: A database sublanguage founded on the relational calculus. In: Proceedings of the ACM-SIGFIDET Workshops (1971)Google Scholar
  8. 8.
    Cugola, G., Margara, A.: Low latency complex event processing on parallel hardware. J. Parallel Distrib. Comput. 72(2), 205–218 (2012)CrossRefGoogle Scholar
  9. 9.
    Dindar, N., et al.: Modeling the execution semantics of stream processing engines with secret. VLDB J. 22(4), 421–446 (2013)CrossRefGoogle Scholar
  10. 10.
    Etzion, O., Niblett, P.: Event Processing in Action. Manning, Shelter Island (2010)Google Scholar
  11. 11.
    Georgala, K., Sherif, M.A., Ngomo, A.N.: An efficient approach for the generation of Allen relations. In: ECAI, pp. 948–956 (2016)Google Scholar
  12. 12.
    Grossniklaus, M., Maier, D., Miller, J., Moorthy, S., Tufte, K.: Frames: data-driven windows. In: DEBS, pp. 13–24. ACM (2016)Google Scholar
  13. 13.
    Hirzel, M., Baudart, G., Bonifati, A., Valle, E.D., Sakr, S., Vlachou, A.: Stream processing languages in the big data era. SIGMOD Rec. 47(2), 29 (2018)CrossRefGoogle Scholar
  14. 14.
    Körber, M., Glombiewski, N., Seeger, B.: TPStream: low-latency temporal pattern matching on event streams. In: EDBT, pp. 313–324 (2018)Google Scholar
  15. 15.
    Li, M., Mani, M., Rundensteiner, E.A., Lin, T.: Complex event pattern detection over streams with interval-based temporal semantics. In: DEBS (2011)Google Scholar
  16. 16.
    Ostovar, A., Maaradji, A., La Rosa, M., ter Hofstede, A.H.M.: Characterizing drift from event streams of business processes. In: Dubois, E., Pohl, K. (eds.) CAiSE 2017. LNCS, vol. 10253, pp. 210–228. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59536-8_14CrossRefGoogle Scholar
  17. 17.
    Paschke, A.: ECA-RULEML: an approach combining ECA rules with temporal interval-based KR event/action logics and transactional update logics. CoRR (2006)Google Scholar
  18. 18.
    Richter, F., Seidl, T.: TESSERACT: time-drifts in event streams using series of evolving rolling averages of completion times. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNCS, vol. 10445, pp. 289–305. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65000-5_17CrossRefGoogle Scholar
  19. 19.
    van Zelst, S.J., Fani Sani, M., Ostovar, A., Conforti, R., La Rosa, M.: Filtering spurious events from event streams of business processes. In: Krogstie, J., Reijers, H.A. (eds.) CAiSE 2018. LNCS, vol. 10816, pp. 35–52. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-91563-0_3CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of TartuTartuEstonia
  2. 2.Politecnico di MilanoMilanItaly

Personalised recommendations

Cite paper

Buy options