EARL: Joint Entity and Relation Linking for Question Answering over Knowledge Graphs

  • Mohnish DubeyEmail author
  • Debayan Banerjee
  • Debanjan Chaudhuri
  • Jens Lehmann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11136)


Many question answering systems over knowledge graphs rely on entity and relation linking components in order to connect the natural language input to the underlying knowledge graph. Traditionally, entity linking and relation linking have been performed either as dependent sequential tasks or as independent parallel tasks. In this paper, we propose a framework called EARL, which performs entity linking and relation linking as a joint task. EARL implements two different solution strategies for which we provide a comparative analysis in this paper: The first strategy is a formalisation of the joint entity and relation linking tasks as an instance of the Generalised Travelling Salesman Problem (GTSP). In order to be computationally feasible, we employ approximate GTSP solvers. The second strategy uses machine learning in order to exploit the connection density between nodes in the knowledge graph. It relies on three base features and re-ranking steps in order to predict entities and relations. We compare the strategies and evaluate them on a dataset with 5000 questions. Both strategies significantly outperform the current state-of-the-art approaches for entity and relation linking.


Entity linking Relation linking GTSP Question answering 



This work is supported by the funding received from the EU H2020 projects WDAqua (ITN, GA. 642795) and HOBBIT (GA. 688227).


  1. 1.
    Berant, J., Chou, A., Frostig, R., Liang, P.: Semantic parsing on freebase from question-answer pairs. In: EMNLP, vol. 2, p. 6 (2013)Google Scholar
  2. 2.
    Both, A., Diefenbach, D., Singh, K., Shekarpour, S., Cherix, D., Lange, C.: Qanary – a methodology for vocabulary-driven open question answering systems. In: Sack, H., Blomqvist, E., d’Aquin, M., Ghidini, C., Ponzetto, S.P., Lange, C. (eds.) ESWC 2016. LNCS, vol. 9678, pp. 625–641. Springer, Cham (2016). Scholar
  3. 3.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)Google Scholar
  4. 4.
    Dubey, M., Dasgupta, S., Sharma, A., Höffner, K., Lehmann, J.: AskNow: a framework for natural language query formalization in SPARQL. In: Sack, H., Blomqvist, E., d’Aquin, M., Ghidini, C., Ponzetto, S.P., Lange, C. (eds.) ESWC 2016. LNCS, vol. 9678, pp. 300–316. Springer, Cham (2016). Scholar
  5. 5.
    Gerber, D., Ngomo, A.-C.N.: Bootstrapping the linked data web. In: 1st Workshop on Web Scale Knowledge Extraction@ ISWC, vol. 2011 (2011)Google Scholar
  6. 6.
    Gubichev, A., Then, M.: Graph pattern matching: do we have to reinvent the wheel? In: Proceedings of Workshop on GRAph Data. ACM (2014)Google Scholar
  7. 7.
    Helsgaun, K.: Solving the equality generalized traveling salesman problem using the Lin-Kernighan-Helsgaun algorithm. Math. Program. Comput. 7, 269–287 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Höffner, K., Walter, S., Marx, E., Usbeck, R., Lehmann, J., Ngonga Ngomo, A.-C.: Survey on challenges of question answering in the semantic web. Semant. Web 8(6), 895–920 (2017)CrossRefGoogle Scholar
  9. 9.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  10. 10.
    Laporte, G., Mercure, H., Nobert, Y.: Generalized travelling salesman problem through n sets of nodes: the asymmetrical case. Discrete Appl. Math. 18(2), 185–197 (1987)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Lukovnikov, D., Fischer, A., Lehmann, J., Auer, S.: Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of the 26th International Conference on World Wide Web, pp. 1211–1220 (2017)Google Scholar
  12. 12.
    Mendes, P.N., Jakob, M., García-Silva, A., Bizer, C.: DBpedia spotlight: shedding light on the web of documents. In: Proceedings of the 7th International Conference on Semantic Systems, pp. 1–8. ACM (2011)Google Scholar
  13. 13.
    Moro, A., Raganato, A., Navigli, R.: Entity linking meets word sense disambiguation: a unified approach. Trans. Assoc. Comput. Linguist. (2014)Google Scholar
  14. 14.
    Mulang, I.O., Singh, K., Orlandi, F.: Matching natural language relations to knowledge graph properties for question answering. In: Proceedings of the 13th International Conference on Semantic Systems, pp. 89–96. ACM (2017)Google Scholar
  15. 15.
    Nakashole, N., Weikum, G., Suchanek, F.: Patty: a taxonomy of relational patterns with semantic types. In: Proceedings of the EMNLP 2012, pp. 1135–1145. Association for Computational Linguistics (2012)Google Scholar
  16. 16.
    Park, S., Kwon, S., Kim, B., Lee, G.G.: ISOFT at QALD-5: hybrid question answering system over linked data and text data. In: CLEF (Working Notes) (2015)Google Scholar
  17. 17.
    Pinter, Y., Guthrie, R., Eisenstein, J.: Mimicking word embeddings using subword RNNs. In: EMNLP, pp. 102–112 (2017)Google Scholar
  18. 18.
    Ristoski, P., Paulheim, H.: RDF2Vec: RDF graph embeddings for data mining. In: Groth, P., et al. (eds.) ISWC 2016. LNCS, vol. 9981, pp. 498–514. Springer, Cham (2016). Scholar
  19. 19.
    Serban, I.V., et al.: Generating factoid questions with recurrent neural networks: the 30m factoid question-answer corpus. arXiv preprint arXiv:1603.06807 (2016)
  20. 20.
    Singh, K., et al.: Capturing knowledge in semantically-typed relational patterns to enhance relation linking. In: Proceedings of the Knowledge Capture Conference, p. 31. ACM (2017)Google Scholar
  21. 21.
    Singh, K., et al.: Why reinvent the wheel: let’s build question answering systems together. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 1247–1256. International World Wide Web Conferences Steering Committee (2018)Google Scholar
  22. 22.
    Speck, R., Ngonga Ngomo, A.-C.: Ensemble learning for named entity recognition. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 519–534. Springer, Cham (2014). Scholar
  23. 23.
    Trivedi, P., Maheshwari, G., Dubey, M., Lehmann, J.: LC-QuAD: a corpus for complex question answering over knowledge graphs. In: d’Amato, C., et al. (eds.) ISWC 2017. LNCS, vol. 10588, pp. 210–218. Springer, Cham (2017). Scholar
  24. 24.
    Trudeau, R.J.: Introduction to Graph Theory (corrected, enlarged republication. ed.) (1993)Google Scholar
  25. 25.
    Unger, C., Bühmann, L., Lehmann, J., Ngonga Ngomo, A.-C., Gerber, D., Cimiano, P.: Template-based question answering over RDF data. In: Proceedings of the 21st International Conference on World Wide Web, pp. 639–648. ACM (2012)Google Scholar
  26. 26.
    Usbeck, R., et al.: AGDISTIS - graph-based disambiguation of named entities using linked data. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 457–471. Springer, Cham (2014). Scholar
  27. 27.
    Veyseh, A.P.B.: Cross-lingual question answering using common semantic space. In: TextGraphs@ NAACL-HLT, pp. 15–19 (2016)Google Scholar
  28. 28.
    Xu, K., Zhang, S., Feng, Y., Zhao, D.: Answering natural language questions via phrasal semantic parsing. In: Zong, C., Nie, J.Y., Zhao, D., Feng, Y. (eds.) Natural Language Processing and Chinese Computing. CCIS, vol. 496, pp. 333–344. Springer, Heidelberg (2014). Scholar
  29. 29.
    Yang, Y., Chang, M.-W.: S-mart: novel tree-based structured learning algorithms applied to tweet entity linking. In: ACL 2015 (2015)Google Scholar
  30. 30.
    Yih, W.-T., Chang, M.-W., He, X., Gao, J.: Semantic parsing via staged query graph generation: question answering with knowledge base. In: Proceedings of the 53rd ACL Conference, vol. 1, pp. 1321–1331 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mohnish Dubey
    • 1
    • 2
    Email author
  • Debayan Banerjee
    • 1
  • Debanjan Chaudhuri
    • 1
    • 2
  • Jens Lehmann
    • 1
    • 2
  1. 1.Smart Data Analytics Group (SDA)University of BonnBonnGermany
  2. 2.Fraunhofer IAISBonnGermany

Personalised recommendations