Evaluation of Maritime Event Detection Against Missing Data

  • Maximilian ZochollEmail author
  • Clément Iphar
  • Manolis PitsikalisEmail author
  • Anne-Laure JousselmeEmail author
  • Alexander ArtikisEmail author
  • Cyril RayEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1010)


Detecting and preventing maritime events like collisions or unusual behaviour of vessels are of high importance for maritime safety and security. As the trust of human operators in automated maritime event detection and prediction depends on the quality of the corresponding algorithms, the evaluation methodology becomes a driving force for the future development of maritime event detection and forecasting methods. The main contribution of this article consists in the development of an evaluation methodology and its application to a selected set of maritime event detectors. The approach links a reference dataset, controlled data variations, maritime event detection algorithms with internal parameters, and performance criteria. Among pre-established possible input data variations applied to a reference Automatic Identification System (AIS) dataset, the article focuses on the evaluation of detection accuracy of maritime event detectors implemented with the Event Calculus logical language against variable amounts of missing data, as a frequently observable type of AIS data degradation. Twelve maritime event pattern detectors are evaluated and most of them are found to vary very little in performance while only one detector shows an unexpected strong performance drop giving insights into how to improve the detection method. Results are provided on a real AIS data enriched with specific simulated events.


Evaluation methodology Maritime event detection Event Calculus Datasets creation Missing data Data veracity 



This work was supported by project datAcron, which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 687591. The authors wish to thank the NATO Allied Command Transformation (NATO-ACT) for supporting the CMRE project on Data Knowledge and Operational Effectiveness (DKOE).


  1. 1.
    Andler, S., Fredin, M., Gustafsson, F., van Laere, J., Nilsson, M., Svenson, P.: SMARTracIn - a concept for spoof resistant tracking of vessel and detection of adverse intentions. In: SPIE Defense, Security, and Sensing, Orlando, FL (2009)Google Scholar
  2. 2.
    Artikis, A., Sergot, M.J.: Executable specification of open multi-agent systems. Logic J. IGPL 18(1), 31–65 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Artikis, A., Sergot, M.J., Paliouras, G.: An event calculus for event recognition. IEEE Trans. Knowl. Data Eng. 27(4), 895–908 (2015)CrossRefGoogle Scholar
  4. 4.
    Auslander, B., Gupta, K.M., Aha, D.W.: A comparative evaluation of anomaly detection algorithms for maritime video surveillance. In: Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense X, vol. 8019, p. 801907. SPIE (2011)Google Scholar
  5. 5.
    Iphar, C., Jousselme, A.L., Ray, C.: Pseudo-synthetic datasets in support to maritime surveillance algorithms assessment. In: Proceedings of the VERITA Workshop, 19ieme Journées Francophones Extraction et Gestion des Connaissances (EGC 2019), January 2019Google Scholar
  6. 6.
    ITU: Technical characteristics for an automatic identification system using time-division multiple access in the VHF maritime mobile band (2010)Google Scholar
  7. 7.
    Jousselme, A.L., Maupin, P.: Comparison of uncertainty representations for missing data in information retrieval. In: Proceedings of the 16th International Conference on Information Fusion, pp. 1902–1909. IEEE (2013)Google Scholar
  8. 8.
    Kowalski, R.A., Sergot, M.J.: A logic-based calculus of events. New Gener. Comput. 4(1), 67–95 (1986)CrossRefGoogle Scholar
  9. 9.
    Lavesson, N., Davidsson, P.: Evaluating learning algorithms and classifiers. Int. J. Intell. Inf. Database Syst. 1(1), 37–52 (2007)Google Scholar
  10. 10.
    Margineantu, D.D., Dietterich, T.G., et al.: Bootstrap methods for the cost-sensitive evaluation of classifiers (2000)Google Scholar
  11. 11.
    Provost, F., Fawcett, T.: Robust classification for imprecise environments. Mach. Learn. 42(3), 203–231 (2001)CrossRefGoogle Scholar
  12. 12.
    Przymusinski, T.: On the declarative semantics of stratified deductive databases and logic programs. In: Foundations of Deductive Databases and Logic Programming. Morgan (1987)Google Scholar
  13. 13.
    Ray, C., Dréo, R., Camossi, E., Jousselme, A.L., Iphar, C.: Heterogeneous integrated dataset for maritime intelligence, surveillance, and reconnaissance. Data Brief (2019, in Press). Scholar
  14. 14.
    Riveiro, M., Falkman, G.: Supporting the analytical reasoning process in maritime anomaly detection: evaluation and experimental design. In: 2010 14th International Conference Information Visualisation, pp. 170–178. IEEE (2010)Google Scholar
  15. 15.
    Roy, J., Davenport, M.: Exploitation of maritime domain ontologies for anomaly detection and threat analysis. In: Proceedings of the IEEE international Waterside Security Conference (WSS) (2010)Google Scholar
  16. 16.
    Rubin, D.B.: Inference and missing data. Biometrika 63(3), 581–592 (1976)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.NATO STO Centre for Maritime Research and ExperimentationLa SpeziaItaly
  2. 2.NCSR DemokritosAthensGreece
  3. 3.University of PiraeusPiraeusGreece
  4. 4.Naval Academy Research InstituteBrestFrance

Personalised recommendations