Abstract
The growing number of vessel data being constantly reported by a variety of remote sensors, such as the Automatic Identification System (AIS), requires new data analytics that can operate at high data rates and are highly scalable. Based on a real-world dataset from maritime transport, we propose a large scale vessel trajectory tracking application implemented in the distributed stream processing system Apache Flink. By implementing a state-space model (SSM) - the Extended Kalman Filter (EKF) - we firstly demonstrate that an implementation of SSMs is feasible in modern distributed data flow systems and secondly we show that we can reach a high performance by leveraging the inherent parallelization of the distributed system. In our experiments we show that the distributed tracking system is able to handle a throughput of several hundred vessels per ms. Moreover, we show that the latency to predict the position of a vessel is well below 500 ms on average, allowing for real-time applications.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
DEBS 2018 Grand Challenge. http://www.cs.otago.ac.nz/debs2018/calls/gc.html. Accessed 27 Nov 2018
Hadoop. http://hadoop.apache.org/. Accessed 27 Nov 2018
Alessandrini, A., et al.: Mining vessel tracking data for maritime domain applications. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 361–367. IEEE (2016)
Apache Flink: Application Development, Working with State. https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html. Accessed 10 June 2019
Apache Flink: Configuration. https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/config.html#configuring-taskmanager-processing-slots. Accessed 10 June 2019
Apache Flink: Fast and reliable large-scale data processing engine. http://flink.apache.org. Accessed 27 Nov 2018
Apache Spark. https://spark.apache.org/. Accessed 27 Nov 2018
Balzer, P.: Multidimensional Kalman-Filter (2017). https://github.com/balzer82/Kalman/. Accessed 27 Nov 2018
Brandt, T., Grawunder, M.: Moving object stream processing with short-time prediction. In: Proceedings of the 8th ACM SIGSPATIAL Workshop on GeoStreaming, pp. 49–56. ACM (2017)
Dalsnes, B.R., Hexeberg, S., Flåten, A.L., Eriksen, B.H., Brekke, E.F.: The neighbor course distribution method with Gaussian mixture models for AIS-based vessel trajectory prediction. In: 2018 21st International Conference on Information Fusion (FUSION), pp. 580–587, July 2018. https://doi.org/10.23919/ICIF.2018.8455607
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
He, B., et al.: Comet: batched stream processing for data intensive distributed computing. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 63–74. ACM (2010)
Jwo, D.J., Wang, S.H.: Adaptive fuzzy strong tracking extended Kalman filtering for GPS navigation. IEEE Sens. J. 7(5), 778–789 (2007)
Karimov, J., Rabl, T., Katsifodimos, A., Samarev, R., Heiskanen, H., Markl, V.: Benchmarking distributed stream data processing systems. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 1507–1518. IEEE (2018)
Kelly, A.: A 3D state space formulation of a navigation Kalman filter for autonomous vehicles. Technical report, Carnegie-Mellon University Pittsburgh PA Robotics Institute (1994)
Korn, U.: A simple method for modelling changes over time. Casualty Actuarial Society E-Forum (2018)
Lee, J.W., Kim, M.S., Kweon, I.S.: A Kalman filter based visual tracking algorithm for an object moving in 3D. In: Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots, vol. 1, pp. 342–347. IEEE (1995)
Lipka, M., Sippel, E., Vossiek, M.: An extended Kalman filter for direct, real-time, phase-based high precision indoor localization. IEEE Access 7, 25288–25297 (2019). https://doi.org/10.1109/ACCESS.2019.2900799
Moussa, R.: Scalable maritime traffic map inference and real-time prediction of vessels’ future locations on apache spark. In: Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems, pp. 213–216. ACM (2018)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)
Perera, L.P., Oliveira, P., Soares, C.G.: Maritime traffic monitoring based on vessel detection, tracking, state estimation, and trajectory prediction. IEEE Trans. Intell. Transp. Syst. 13(3), 1188–1200 (2012)
Perera, S., Suhothayan, S.: Solution patterns for realtime streaming analytics. In: Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems, DEBS 2015, pp. 247–255. ACM, New York (2015). https://doi.org/10.1145/2675743.2774214. http://doi.acm.org/10.1145/2675743.2774214
Sheng, C., Zhao, J., Leung, H., Wang, W.: Extended Kalman filter based echo state network for time series prediction using mapreduce framework. In: 2013 IEEE 9th International Conference on Mobile Ad-hoc and Sensor Networks, pp. 175–180. IEEE (2013)
Tu, E., Zhang, G., Rachmawati, L., Rajabally, E., Huang, G.B.: Exploiting AIS data for intelligent maritime navigation: a comprehensive survey from data to methodology. IEEE Trans. Intell. Transp. Syst. 19(5), 1559–1582 (2017)
Welch, P., Bishop, G.: An Introduction to the Kalman Filter (coursePack) (2001). http://www.cs.unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_08.pdf. Accessed 27 Nov 2018
Acknowledgements
This work was partly supported by the German Federal Ministry for Education and Research (BMBF) as Berlin Big Data Center (BBDC2) (grant no. 01IS18025A), the German Federal Ministry of Transport and Digital Infrastructure (BMVI) through the Daystream Project (grant no. 19F2031A).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Juraszek, K., Saini, N., Charfuelan, M., Hemsen, H., Markl, V. (2020). Extended Kalman Filter for Large Scale Vessels Trajectory Tracking in Distributed Stream Processing Systems. In: Lemaire, V., Malinowski, S., Bagnall, A., Bondu, A., Guyet, T., Tavenard, R. (eds) Advanced Analytics and Learning on Temporal Data. AALTD 2019. Lecture Notes in Computer Science(), vol 11986. Springer, Cham. https://doi.org/10.1007/978-3-030-39098-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-39098-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39097-6
Online ISBN: 978-3-030-39098-3
eBook Packages: Computer ScienceComputer Science (R0)