Abstract
Making an informed and right decision poses huge challenges for drivers in day-to-day traffic situations. This task vastly depends on many subjective and objective factors, including the current driver state, her destination, personal preferences and abilities as well as surrounding environment. In this paper, we present CoSI (Context and Situation Intelligence), a Knowledge Graph (KG)-based approach for fusing and organizing heterogeneous types and sources of information. The KG serves as a coherence layer representing information in the form of entities and their inter-relationships augmented with additional semantic axioms. Harnessing the power of axiomatic rules and reasoning capabilities enables inferring additional knowledge from what is already encoded. Thus, dedicated components exploit and consume the semantically enriched information to perform tasks such as situation classification, difficulty assessment, and trajectory prediction. Further, we generated a synthetic dataset to simulate real driving scenarios with a large range of driving styles and vehicle configurations. We use KG embedding techniques based on a Graph Neural Network (GNN) architecture for a classification task of driving situations and achieve over 95% accuracy whereas vector-based approaches achieve only 75% accuracy for the same task. The results suggest that the KG-based information representation combined with GNN are well suited for situation understanding tasks as required in driver assistance and automated driving systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
References
Bazilinskyy, P., Petermeijer, S.M., Petrovych, V., Dodou, D., Winter, J.D.: Take-over requests in highly automated driving: a crowdsourcing survey on auditory, vibrotactile, and visual displays. Transp. Res. Part F-Traffic Psychol. Behav. 56, 82–98 (2018)
Becker, J., Helmle, M., Pink, O.: System architecture and safety requirements for automated driving (2017)
Becker, J., Kammel, S., Pink, O., Fausten, M.: Bosch’s approach toward automated driving. at - Automatisierungstechnik 63, 180–190 (2015)
Bengler, K., Dietmayer, K., Färber, B., Maurer, M., Stiller, C., Winner, H.: Three decades of driver assistance systems: review and future perspectives. IEEE Intell. Transp. Syst. Mag. 6, 6–22 (2014)
Bolle, M., Knoop, S., Niewels, F., Schamm, T.: Early level 4/5 automation by restriction of the use-case. In: Internationales Stuttgarter Symposium (2017)
Buechel, M., Hinz, G., Ruehl, F., Schroth, H., Gyoeri, C., Knoll, A.: Ontology-based traffic scene modeling, traffic regulations dependent situational awareness and decision-making for automated vehicles. In: IEEE IV, pp. 1471–1476 (2017)
Campbell, K.: The shrp 2 naturalistic driving study: Addressing driver performance and behavior in traffic safety (2012)
Chen, C., et al.: Gated residual recurrent graph neural networks for traffic prediction. In: AAAI (2019)
Datta, S.K., Costa, R.P.F.D., Härri, J., Bonnet, C.: Integrating connected vehicles in internet of things ecosystems: challenges and solutions. In: 17th International Symposium on WoWMoM, Portugal, pp. 1–6. IEEE Computer Society (2016)
Di, S., Zhang, H., Li, C.G., Mei, X., Prokhorov, D., Ling, H.: Cross-domain traffic scene understanding: a dense correspondence-based transfer learning approach. IEEE Trans. Intell. Transp. Syst. 19, 745–757 (2018)
Diehl, F., Brunner, T., Truong-Le, M., Knoll, A.: Graph neural networks for modelling traffic participant interaction. In: IEEE IV, pp. 695–701 (2019)
Ding, C., Mao, Y., Wang, W., Baumann, M.: Driving situation awareness in transport operations. In: Computational Intelligence for Traffic and Mobilit (2013). https://doi.org/10.2991/978-94-91216-80-0_3
Ding, W., Chen, J., Shen, S.: Predicting vehicle behaviors over an extended horizon using behavior interaction network. In: ICRA, pp. 8634–8640 (2019)
Erdmann, J.: Lane-changing model in sumo. In: SUMO2014. Reports of the DLR-Institute of Transportation Systems Proceedings, vol. 24, pp. 77–88. Deutsches Zentrum für Luft- und Raumfahrt e.V., May 2014. https://elib.dlr.de/89233/
Gao, J., et al.: VectorNet: encoding HD maps and agent dynamics from vectorized representation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11522–11530 (2020)
Guo, J., Kurup, U., Shah, M.: Is it safe to drive? An overview of factors, metrics, and datasets for driveability assessment in autonomous driving. IEEE Trans. Intell. Transp. Syst. 21(8), 3135–3151 (2020)
Haller, A., et al.: The modular SSN ontology: a joint W3C and OGC standard specifying the semantics of sensors, observations, sampling, and actuation. Semant. Web 10(1), 9–32 (2019)
Hamid, S.A., Hassanein, H.S., Takahara, G.: Vehicle as a resource (VAAR). IEEE Netw. 29(1), 12–17 (2015)
Huang, L., Liang, H., Yu, B., Li, B., Zhu, H.: Ontology-based driving scene modeling, situation assessment and decision making for autonomous vehicles. In: 2019 4th Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), pp. 57–62 (2019)
Hügle, M., Kalweit, G., Mirchevska, B., Werling, M., Boedecker, J.: Dynamic input for deep reinforcement learning in autonomous driving. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7566–7573 (2019)
Hülsen, M., Zöllner, J.M., Weiss, C.: Traffic intersection situation description ontology for advanced driver assistance. In: IEEE IV Symposium, pp. 993–999 (2011)
Kalra, N., Paddock, S.: Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp. Res. Part A-Policy Pract. 94, 182–193 (2016)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th ICLR (2017). https://openreview.net/forum?id=SJU4ayYgl
Krauss, S., Wagner, P., Gawron, C.: Metastable states in a microscopic model of traffic flow. Phys. Rev. E 55, 5597–5602 (1997)
Kusano, K., Gabler, H.C.: Safety benefits of forward collision warning, brake assist, and autonomous braking systems in rear-end collisions. IEEE Trans. Intell. Transp. Syst. 13, 1546–1555 (2012)
Lefèvre, S., Vasquez, D., Laugier, C.: A survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH J. 1, 1–14 (2014)
Lenz, D., Diehl, F., Truong-Le, M., Knoll, A.C.: Deep neural networks for Markovian interactive scene prediction in highway scenarios. In: IEEE Intelligent Vehicles Symposium, IV 2017, pp. 685–692. IEEE (2017)
Li, M., Cao, H., Song, X., Huang, Y., Wang, J., Huang, Z.: Shared control driver assistance system based on driving intention and situation assessment. IEEE Trans. Industr. Inf. 14, 4982–4994 (2018)
Mirchevska, B., Pek, C., Werling, M., Althoff, M., Boedecker, J.: High-level decision making for safe and reasonable autonomous lane changing using reinforcement learning. In: 21st ITSC, pp. 2156–2162 (2018)
Mukadam, M., Cosgun, A., Nakhaei, A., Fujimura, K.: Tactical decision making for lane changing with deep reinforcement learning (2017)
Petersen, N., Halilaj, L., Grangel-González, I., Lohmann, S., Lange, C., Auer, S.: Realizing an RDF-based information model for a manufacturing company – a case study. In: d’Amato, C., et al. (eds.) ISWC 2017. LNCS, vol. 10588, pp. 350–366. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68204-4_31
Poggi, A., Lembo, D., Calvanese, D., Giacomo, G.D., Lenzerini, M., Rosati, R.: Linking data to ontologies. J. Data Semant. 10, 133–173 (2008)
Röckl, M., Robertson, P., Frank, K., Strang, T.: An architecture for situation-aware driver assistance systems. In: IEEE 65th Vehicular Technology Conference - VTC-Spring, pp. 2555–2559 (2007)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Schukraft, M., Rothermel, S., Luettin, J., Halilaj, L.: Towards a rule-based approach for estimating the situation difficulty in driving scenarios. In: 7th International Conference on Vehicle Technology and Intelligent Transport System (VEHITS) (2021, to appear)
Takahashi, R., Naoya Inoue, Y.K., Kobayashi, S., Inui, K.: Explaining potential risks in traffic scenes by combining logical inference and physical simulation, vol. 6, pp. 248–255 (2016)
Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F., Maurer, M.: Defining and substantiating the terms scene, situation, and scenario for automated driving. In: 18th ITSC, Gran Canaria, Spain, pp. 982–988. IEEE (2015)
Vinyals, O., Bengio, S., Kudlur, M.: Order matters: sequence to sequence for sets. In: Bengio, Y., LeCun, Y. (eds.) 4th International Conference on Learning Representations, ICLR 2016 (2016)
Wang, C., Winner, H.: Overcoming challenges of validation automated driving and identification of critical scenarios. In: IEEE ITSC, pp. 2639–2644 (2019)
Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29, 2724–2743 (2017)
Wilcke, W.X., Bloem, P., de Boer, V., t Veer, R., van Harmelen, F.: End-to-end entity classification on multimodal knowledge graphs abs/2003.12383 (2020)
Wishart, J., et al.: Driving safety performance assessment metrics for ads-equipped vehicles (2020)
Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R.R., Smola, A.J.: Deep sets. In: Advances in Neural Information Processing System, vol. 30 (2017)
Zofka, M.R., Klemm, S., Kuhnt, F., Schamm, T., Zöllner, J.M.: Testing and validating high level components for automated driving: simulation framework for traffic scenarios. In: 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 144–150 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Halilaj, L., Dindorkar, I., Lüttin, J., Rothermel, S. (2021). A Knowledge Graph-Based Approach for Situation Comprehension in Driving Scenarios. In: Verborgh, R., et al. The Semantic Web. ESWC 2021. Lecture Notes in Computer Science(), vol 12731. Springer, Cham. https://doi.org/10.1007/978-3-030-77385-4_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-77385-4_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77384-7
Online ISBN: 978-3-030-77385-4
eBook Packages: Computer ScienceComputer Science (R0)