Clustering Environmental Conditions of Historical Accident Data to Efficiently Generate Testing Sceneries for Maritime Systems
Vessels are getting more and more equipped with highly-automated assistant systems that benefit from the use of machine learning. Such trained safety-critical systems demand for new means of Verification and Validation (V+V). Their complex decision making process is hidden and traditional system analysis and functional testing is no longer possible as the testing space becomes too large to test. Scenario-based V+V performed in a simulation environment is a promising approach to tackle these challenges, triggering potential system malfunctions and covering as much as possible of the problem space.
The authors propose a data-driven method to identify relevant sceneries, which describe states of a system in a scenario by a set of parameters. These states are derived from accident reports, summarizing the most critical situations a vessel and its automated assistant systems might be confronted with. By a chain of several methods, such as Principal Component Analysis and K-Mean Clustering the authors show that the value space of scenery parameters to be tested can be reduced and clusters can be identified that define equivalence classes of accidents. These clusters can then be partitioned depending on their probability distributions and open up a (reduced) space for random sampling of testing sceneries.
The authors tested the method focusing on a weather-related parameter set of 1700 accidents in 2016 and 2017 that were retrieved from three different sources. Results show, that the first three principal components of the environmental parameters explain over 90% of the original variance and can be divided into 13 clusters. The authors then manually identified those accidents of a different data pool from 2013–2015 for that weather conditions were reported as the main cause of the accident and found the majority of them (61%) within the clusters and further 23% already in close distance. The more accidents are considered as input for the method the better would be the cluster fitting.
KeywordsScenario-based testing Principal component analysis K-Mean Clustering Latin Hypercube Ship accidents
This research is supported by the state of Lower Saxony as part of the project Architecture and Technology – Development – Platform for Realtime Safe and Secure Systems (ACTRESS).
- 2.Brinkmann, M., Böde, E., Lamm, A., Maelen, S.V., Hahn, A.: Learning from automotive: testing maritime assistance systems up to autonomous vessels. In: OCEANS 2017 – Aberdeen, pp. 1–8 (2017). https://doi.org/10.1109/oceanse.2017.8084951
- 3.Lamm, A., Hahn, A.: Detecting maneuvers in maritime observation data with CUSUM. In: 2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), pp. 122–127. IEEE, Bilbao (2017). https://doi.org/10.1109/isspit.2017.8388628
- 4.Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F., Maurer, M.: Defining and substantiating the terms scene, situation, and scenario for automated driving. In: 2015 IEEE 18th International Conference on Intelligent Transportation Systems, pp. 982–988. IEEE, Gran Canaria (2015). https://doi.org/10.1109/itsc.2015.164
- 5.Shahir, H.Y., Glässer, U., Farahbod, R., Jackson, P., Wehn, H.: Generating test cases for marine safety and security scenarios: a composition framework. Secur. Inform. 1 (2012). https://doi.org/10.1186/2190-8532-1-4
- 6.Schuldt, F., Reschka, A., Maurer, M.: A method for an efficient, systematic test case generation for advanced driver assistance systems in virtual environments. In: Winner, H., Prokop, G., Maurer, M. (eds.) Automotive Systems Engineering II, pp. 147–175. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-61607-0_7CrossRefGoogle Scholar
- 7.Lamm, A., Hahn, A.: Towards critical-scenario based testing with maritime observation data. In: 2018 OCEANS - MTS/IEEE Kobe Techno-Oceans (OTO), pp. 1–10. IEEE, Kobe (2018). https://doi.org/10.1109/oceanskobe.2018.8559045
- 8.Pütz, A., Zlocki, A., Bock, J., Eckstein, L.: System validation of highly automated vehicles with a database of relevant traffic scenarios. In: 12th ITS European Congress, p. 8 (2017)Google Scholar
- 10.Esnaf, S., Koldemir, B., Küçükdeniz, T., Akten, N.: Fuzzy cluster analysis of shipping accidents in the bosporus. Eur. J. Navig. 6 (2008)Google Scholar
- 11.Lema, E., Papaioannou, D., Vlachos, G.P.: Investigation of coinciding shipping accident factors with the use of partitional clustering methods. In: Proceedings of the 7th International Conference on Pervasive Technologies Related to Assistive Environments - PETRA 2014, pp. 1–4. ACM Press, Rhodes (2014). https://doi.org/10.1145/2674396.2674461
- 13.Guedes, S.C., Bitner-Gregersen, E.M., Antão, P.: Analysis of the frequency of ship accidents under severe North Atlantic weather conditions. In: Conference: Design and Operation for Abnormal Conditions, vol. 2, pp. 221–230 (2001)Google Scholar
- 16.Ueno, M., Kitamura, F., Sogihnara, N., Fujiwara, T.: A simple method to estimate wind loads on ships. In: Advances in Civil, Environmental, and Materials Research, pp. 26–30 (2012)Google Scholar
- 18.Gluver, H.: Ship Collision Analysis: Proceedings of the International Symposium on Advances in Ship Collision Analysis, Copenhagen, Denmark, 10–13 May 1998. Routledge, London (2017)Google Scholar
- 21.Caliendo, C., Parisi, A.: Principal component analysis applied to crash data on multilane roads. In: Proceedings of Third International SIIV Congress, vol. 1, pp. 1–7 (2005)Google Scholar
- 22.Golob, T.F., Recker, W.W.: Relationships among urban freeway accidents, traffic flow, weather, and lighting conditions. J. Transp. Eng. 129, 342–353 (2003). https://doi.org/10.1061/(asce)0733-947x(2003)129:4(342)CrossRefGoogle Scholar
- 28.Copernicus Climate Change Service: ERA5: Fifth generation of ECMWF atmospheric reanalyses of the global climate. Copernicus Climate Change Service Climate Data Store (2017)Google Scholar
- 29.Amante, C., Eakins, B.W.: ETOPO1 1 arc-minute global relief model: procedures, data sources and analysis. NOAA Technical Memorandum NESDIS NGDC-24 (2009). https://doi.org/10.7289/v5c8276m
- 32.Reynolds, J.: Ship-turning characteristics in different water depths. Safety at Sea International, no. 90 (1976)Google Scholar
- 33.Hair, J.F. (ed.): Multivariate Data Analysis. Pearson, Harlow (2014)Google Scholar