Skip to main content

Online fleet monitoring with scalable event recognition and forecasting

Abstract

Moving object monitoring is becoming essential for companies and organizations that need to manage thousands or even millions of commercial vehicles or vessels, detect dangerous situations (e.g., collisions or malfunctions) and optimize their behavior. It is a task that must be executed in real-time, reporting any such situations or opportunities as soon as they appear. Given the growing sizes of fleets worldwide, a monitoring system must be highly efficient and scalable. It is becoming an increasingly common requirement that such monitoring systems should be able to automatically detect complex situations, possibly involving multiple moving objects and requiring extensive background knowledge. Building a monitoring system that is both expressive and scalable is a significant challenge. Typically, the more expressive a system is, the less flexible it becomes in terms of its parallelization potential. We present a system that strikes a balance between expressiveness and scalability. Going beyond event detection, we also present an approach towards event forecasting. We show how event patterns may be given a probabilistic description so that our system can forecast when a complex event is expected to occur. Our proposed system employs a formalism that allows analysts to define complex patterns in a user-friendly manner while maintaining unambiguous semantics and avoiding ad hoc constructs. At the same time, depending on the problem at hand, it can employ different parallelization strategies in order to address the issue of scalability. It can also employ different training strategies in order to fine-tune the probabilistic models constructed for event forecasting. Our experimental results show that our system can detect complex patterns over moving entities with minimal latency, even when the load on our system surpasses what is to be realistically expected in real-world scenarios.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. http://www.acea.be/statistics/article/vehicles-in-use-europe-2017

  2. http://www.imo.org/OurWork/Safety/Navigation/Pages/AIS.aspx

  3. https://www.vesselfinder.com

  4. [1]: R2C3: motivating the forecasting part of the paper.

  5. [2]: R2C1: clarifying the relationship of this paper to its workshop version.

  6. [3]: R1C2,R2C7: commenting on the generic nature of our engine.

  7. [4] R2C6, R2C11: providing a more concrete example in the context of moving object monitoring.

  8. [5] R2C6, R2C11 continued.

  9. [6] R2C6, R2C11 continued.

  10. [7] R1C1: explaining why we focus on CLASSIFICATION-NETXW.

  11. https://www.vodafoneinnovus.com/

  12. [8] R2C10: explaining that Wayeb does not necessarily require preprocessing of the input stream.

  13. R2C9: explaining that noise patterns are treated as complex events.

  14. [10] R3C2: showing results in a table.

  15. [11] R1C3: explaining why we currently use only classification metrics.

  16. [12] R1C4: adding a note about order optimization.

References

  1. Apache flink - stateful computations over data streams. https://flink.apache.org/

  2. Apache kafka. https://kafka.apache.org/

  3. Esper. http://www.espertech.com/esper

  4. Esperonstorm. https://github.com/tomdz/storm-esper

  5. Flinkcep - complex event processing for flink. https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/cep.html

  6. Monitoring Back Pressure. https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/back∖_pressure.html

  7. Siddhi cep. https://github.com/wso2/siddhi

  8. Task chaining and resource groups. https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/∖#task-chaining-and-resource-groups

  9. Wso2. creating a storm based distributed execu-tionplan. https://docs.wso2.com/display/CEP410/Creating+a+Storm+Based+Distributed+Execution+Plan

  10. Abe N, Warmuth MK (1992) On the computational complexity of approximating distributions by probabilistic automata. Mach Learn 9:205–260

    MATH  Google Scholar 

  11. Alevizos E, Artikis A, Paliouras G (2017) Event forecasting with pattern markov chains. In: DEBS

  12. Alevizos E, Artikis A, Paliouras G (2018) Wayeb: a tool for complex event forecasting. In: LPAR

  13. Alevizos E, Artikis A, Paliouras G (2021) Complex event forecasting with prediction suffix trees. VLDB J

  14. Alevizos E, Skarlatidis A, Artikis A, Paliouras G (2017) Probabilistic complex event recognition: A survey. ACM Comput Surv 50(5):71:1–71:31

    Google Scholar 

  15. Artikis A, Sergot M, Paliouras G (2015) An event calculus for event recognition. IEEE Trans Knowl Data Eng

  16. Balkesen C, Dindar N, Wetter M, Tatbul N (2013) RIP: run-based intra-query parallelism for scalable complex event processing. In: DEBS

  17. Begleiter R, El-Yaniv R, Yona G (2004) On prediction using variable order markov models. J Artif Intell Res 22:385–421

    MathSciNet  Article  Google Scholar 

  18. Carbone P, Katsifodimos A, Ewen S, Markl V, Haridi S, Tzoumas K (2015) Apache flink™: Stream and batch processing in a single engine. IEEE Data Eng Bull

  19. Christ M, Krumeich J, Kempa-Liehr AW (2016) Integrating predictive analytics into complex event processing by using conditional density estimations. In: EDOC Workshops. IEEE Computer Society, pp 1–8

  20. Cleary JG, Witten IH (1984) Data compression using adaptive coding and partial string matching. IEEE Trans Commun 32(4):396–402

    Article  Google Scholar 

  21. Cugola G, Margara A (2012) Complex event processing with T-REX. J Syst Softw

  22. Cugola G, Margara A (2012) Processing flows of information: From data stream to complex event processing. ACM Comput Surv

  23. D’Antoni L, Veanes M (2017) The power of symbolic automata and transducers. In: CAV (1)

  24. Demers A, Gehrke J, Panda B, Riedewald M, Sharma V, White W (2007) Cayuga: A general purpose event monitoring system. In: CIDR

  25. Engel Y, Etzion O (2011) Towards proactive event-driven computing. In: DEBS. ACM, pp 125–136

  26. Fülöp LJ, Beszėdes Á, Toth G, Demeter H, Vidȧcs L, Farkas L (2012) Predictive complex event processing: a conceptual framework for combining complex event processing and predictive analytics. In: BCI. ACM, pp 26–31

  27. Giatrakos N, Alevizos E, Artikis A, Deligiannakis A, Garofalakis M (2020) Complex event recognition in the big data era: a survey. VLDB J

  28. Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102(477):359–378

    MathSciNet  Article  Google Scholar 

  29. Hirzel M (2012) Partition and compose: parallel complex event processing. In: DEBS

  30. Koutroumanis N, Santipantakis G, Glenis A, Doulkeridis C, Vouros G (2019) Integration of mobility data with weather information. In: EDBT/ICDT Workshops

  31. Li Y, Ge T, Chen CX (2020) Data stream event prediction based on timing knowledge and state transitions. Proc VLDB Endow 13(10):1779–1792

    Article  Google Scholar 

  32. Liu M, Rundensteiner E, Greenfield K, Gupta C, Wang S, Ari I, Mehta A (2011) E-cube: multi-dimensional event sequence analysis using hierarchical pattern query sharing. In: SIGMOD

  33. Mei Y, Madden S (2009) Zstream: a cost-based query processor for adaptively detecting composite events. In: SIGMOD

  34. Muthusamy V, Liu H, Jacobsen H (2010) Predictive publish/subscribe matching. In: DEBS. ACM, pp 14–25

  35. Ntoulias E, Alevizos E, Artikis A, Koumparos A (2021) Online trajectory analysis with scalable event recognition. In: EDBT/ICDT Workshops, CEUR Workshop Proceedings. CEUR-WS.org, vol 2841

  36. Pandey S, Nepal S, Chen S (2011) A test-bed for the evaluation of business process prediction techniques. In: CollaborateCom. ICST / IEEE, pp 382–391

  37. Patroumpas K, Alevizos E, Artikis A, Vodas M, Pelekis N, Theodoridis Y (2017) Online event recognition from moving vessel trajectories. GeoInformatica

  38. Patroumpas K, Spirelis D, Chondrodima E, Georgiou H, P P P (2018) Final dataset of Trajectory Synopses over AIS kinematic messages in Brest area (ver 0.8) [Data set]. https://doi.org/10.5281/zenodo.2563256

  39. Pitsikalis M, Artikis A, Dreo R, Ray C, Camossi E, Jousselme A (2019) Composite event recognition for maritime monitoring. In: DEBS

  40. Ray C, Dreo R, Camossi E, Jousselme A (2018) Heterogeneous integrated dataset for maritime intelligence, surveillance and reconnaissance. https://doi.org/10.5281/zenodo.1167595

  41. Ron D, Singer Y, Tishby N (1993) The power of amnesia. In: NIPS. Morgan Kaufmann, pp 176–183

  42. Ron D, Singer Y, Tishby N (1996) The power of amnesia: Learning probabilistic automata with variable memory length. Mach Learn 25(2-3):117–149

    Article  Google Scholar 

  43. Sakr MA, Gu̇ting RH (2011) Spatiotemporal pattern queries. GeoInformatica 15(3):497–540

    Article  Google Scholar 

  44. Sakr MA, Gu̇ting RH (2014) Group spatiotemporal pattern queries. GeoInformatica 18(4):699–746

    Article  Google Scholar 

  45. Schultz-Møller N, Migliavacca M, Pietzuch P (2009) Distributed complex event processing with query rewriting. In: DEBS

  46. Snidaro L, Visentini I, Bryan K (2015) Fusing uncertain knowledge and evidence for maritime situational awareness via markov logic networks. Inf Fusion

  47. Terroso-Saenz F, Valdės-Vela M, Skarmeta-Gȯmez A (2016) A complex event processing approach to detect abnormal behaviours in the marine environment. Inf. Syst Frontiers

  48. Tsilionis E, Koutroumanis N, Nikitopoulos P, Doulkeridis C, Artikis A (2019) Online event recognition from moving vehicles: Application paper TPLP

  49. Willems FMJ, Shtarkov YM, Tjalkens TJ (1995) The context-tree weighting method: basic properties. IEEE Trans Inf Theory 41(3):653–664

    Article  Google Scholar 

  50. Zhang H, Diao Y, Immerman N (2014) On complexity and optimization of expensive queries in complex event processing. In: SIGMOD

Download references

Acknowledgements

This work was funded by European Union’s Horizon 2020 research and innovation programme Track & Know “Big Data for Mobility Tracking Knowledge Extraction in Urban Areas”, under grant agreement No 780754. It is also supported by the European Commission under the INFORE project (H2020-ICT- 825070).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Emmanouil Ntoulias.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ntoulias, E., Alevizos, E., Artikis, A. et al. Online fleet monitoring with scalable event recognition and forecasting. Geoinformatica (2022). https://doi.org/10.1007/s10707-022-00465-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10707-022-00465-2

Keywords

  • Complex event processing
  • Big data
  • Distributed computing