A Diagrammatic Approach for Visual Question Answering over Knowledge Graphs

  • Dmitry Mouromtsev
  • Gerhard WohlgenanntEmail author
  • Peter Haase
  • Dmitry Pavlov
  • Yury Emelyanov
  • Alexey Morozov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11155)


In this demo we present a tool for visual question answering (QA) over the Wikidata knowledge graph based on diagrammatic representation and reasoning. The demo is built on top of the metaphacts platform with the Ontodia library embedded. In a user study, we demonstrate and evaluate the approach of diagrammatic question answering using questions from the QALD7 (Question Answering over Linked Data) benchmark for Wikidata. The effectiveness and limitations of the proposed approach are discussed in the evaluation and conclusions sections. In the demo session at ESWC, we plan to present our tool for visual QA and show its QA capabilities using incremental creation of diagrams.


Visual data exploration Knowledge graphs Diagrammatic question answering Wikidata 



This work was supported by the Government of the Russian Federation (Grant 074-U01) through the ITMO Fellowship and Professorship Program.


  1. 1.
    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606 (2016)
  2. 2.
    López, V., Unger, C., Cimiano, P., Motta, E.: Evaluating question answering over linked data. J. Web Sem. 21, 3–13 (2013)CrossRefGoogle Scholar
  3. 3.
    Moldovan, D., et al.: The structure and performance of an open-domain question answering system. In: Proceedings of the 38th Annual Meeting of ACL, ACL 2000, pp. 563–570 (2000)Google Scholar
  4. 4.
    Mouromtsev, D., Pavlov, D., Emelyanov, Y., Morozov, A., Razdyakonov, D., Parkhimovich, O.: Workflow supporting toolset for diagram-based collaborative ontology development implemented in the open budget domain. In: Sack, H., Rizzo, G., Steinmetz, N., Mladenić, D., Auer, S., Lange, C. (eds.) ESWC 2016. LNCS, vol. 9989, pp. 178–182. Springer, Cham (2016). Scholar
  5. 5.
    Narayanan, N.H., Hübscher, R.: Visual language theory: towards a human-computer interaction perspective. In: Marriott, K., Meyer, B. (eds.) Visual Language Theory, pp. 87–128. Springer, New York (1998)CrossRefGoogle Scholar
  6. 6.
    Wohlgenannt, G., Klimov, N., Mouromtsev, D., Razdyakonov, D., Pavlov, D., Emelyanov, Y.: Using word embeddings for visual data exploration with Ontodia and Wikidata. In: BLINK/NLIWoD3@ISWC, International Semantic Web Conference (ISWC), vol. 1932 (2017).

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Dmitry Mouromtsev
    • 1
  • Gerhard Wohlgenannt
    • 1
    Email author
  • Peter Haase
    • 2
  • Dmitry Pavlov
    • 3
  • Yury Emelyanov
    • 3
  • Alexey Morozov
    • 1
  1. 1.Intern. Laboratory of Information Science and Semantic TechnologiesITMO UniversitySt. PetersburgRussia
  2. 2.metaphacts GmbHWalldorfGermany
  3. 3.Vismart Ltd.St. PetersburgRussia

Personalised recommendations