Skip to main content
Log in

An Overview of Utilizing Knowledge Bases in Neural Networks for Question Answering

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

Question Answering (QA) requires understanding of queries expressed in natural languages and identification of relevant information content to provide an answer. For closed-world QAs, information access is obtained by means of either context texts, or a Knowledge Base (KB), or both. KBs are human-generated schematic representations of world knowledge. The representational ability of neural networks to generalize world information makes it an important component of current QA research. In this paper, we study the neural networks and QA systems in the context of KBs. Specifically, we focus on surveying methods for KB embedding, how such embeddings are integrated into the neural networks, and the role such embeddings play in improving performance across different question-answering problems. Our study of multiple question answering methods finds that the neural networks are able to produce state-of-art results in different question answering domains, and inclusion of additional information via KB embeddings further improve the performance of such approaches. Further progress in QA can be improved by incorporating more powerful representations of KBs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Alsentzer, E., Murphy, J.R., Boag, W., Weng, W.H., Jin, D., Naumann, T., & McDermott, M. (2019). Publicly available clinical bert embeddings. CoRR, arXiv:1904.03323.

  • Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C.L., & Parikh, D. (2015). VQA: visual question answering. In ICCV (pp. 2425–2433).

  • Antoniou, G., & Van Harmelen, F. (2004). Web ontology language: Owl. In Handbook on ontologies (pp. 67–92): Springer.

  • Ashburner, M., Ball, C.A., Blake, J.A., Botstein, D., Butler, H., Cherry, J.M., Davis, A.P., Dolinski, K., Dwight, S.S., Eppig, J.T., & et al. (2000). Gene ontology: tool for the unification of biology. Nature Genetics, 25(1), 25–29.

    Article  Google Scholar 

  • Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z.G. (2007). Dbpedia: A nucleus for a web of open data. In ISWC (pp. 722–735): Springer.

  • Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In ICLR.

  • Balazevic, I., Allen, C., & Hospedales, T.M. (2019a). Hypernetwork knowledge graph embeddings. In ICANN (pp. 553–565).

  • Balazevic, I., Allen, C., & Hospedales, T.M. (2019b). Tucker: Tensor factorization for knowledge graph completion. In EMNLP-IJCNLP pp. 5184–5193.

  • Banerjee, P., Pal, K.K., Mitra, A., & Baral, C. (2019). Careful selection of knowledge to solve open book question answering. In ACL (pp. 6120–6129).

  • Bast, H., & Haussmann, E. (2015). More accurate question answering on freebase. In CIKM (pp. 1431–1440).

  • Berant, J., Chou, A., Frostig, R., & Liang, P. (2013). Semantic parsing on freebase from question-answer pairs. In EMNLP (pp. 1533–1544).

  • Bodenreider, O. (2004). The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Research, 32(Database-Issue), 267–270.

    Article  Google Scholar 

  • Bollacker, K.D., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD (pp. 1247–1250).

  • Bordes, A., Weston, J., Collobert, R., & Bengio, Y. (2011). Learning structured embeddings of knowledge bases. In AAAI.

  • Bordes, A., Usunier, N., García-Durán, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In NeurIPS (pp. 2787–2795).

  • Bordes, A., Chopra, S., & Weston, J. (2014). Question answering with subgraph embeddings. In EMNLP (pp. 615–620).

  • Bordes, A., Usunier, N., Chopra, S., & Weston, J. (2015). Large-scale simple question answering with memory networks. CoRR, arXiv:1506.02075.

  • Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Jr, E.R.H., & Mitchell, T.M. (2010). Toward an architecture for never-ending language learning. In AAAI.

  • Chandrahas, S.A, & Talukdar, P.P. (2018). Towards understanding the geometry of knowledge graph embeddings. In ACL (pp. 122–131).

  • Chandrasekaran, B., Josephson, J.R., & Benjamins, V.R. (1999). What are ontologies, and why do we need them? IEEE Intelligent Systems and Their Applications, 14(1), 20–26.

    Article  Google Scholar 

  • Dai, Z., Li, L., & Xu, W. (2016). CFO: Conditional focused neural question answering with large-scale knowledge bases. In ACL.

  • Dettmers, T., Minervini, P., Stenetorp, P., & Riedel, S. (2018). Convolutional 2d knowledge graph embeddings. In AAAI (pp. 1811–1818).

  • Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (pp. 4171–4186).

  • Diefenbach, D., Lopez, V., Singh, K., & Maret, P. (2018). Core techniques of question answering systems over knowledge bases: a survey. Knowledge and Information Systems, 55(3), 529–569.

    Article  Google Scholar 

  • Dong, L., Wei, F., Zhou, M., & Xu, K. (2015). Question answering over freebase with multi-column convolutional neural networks. In ACL (pp. 260–269).

  • Ebisu, T., & Ichise, R. (2018). Toruse: Knowledge graph embedding on a lie group. In AAAI (pp. 1819–1826).

  • Ehrlinger, L., & Wöß, W. (2016). Towards a definition of knowledge graphs. SEMANTiCS (Posters, Demos, suCCESS). Metallurgy - Proceedings, 48.

  • Fader, A., Zettlemoyer, L.S., & Etzioni, O. (2013). Paraphrase-driven learning for open question answering. In ACL (pp 1608–1618).

  • Fader, A., Zettlemoyer, L., & Etzioni, O. (2014). Open question answering over curated and extracted knowledge bases. In KDD (pp 1156–1165).

  • Fukui, A., Park, D.H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016). Multimodal compact bilinear pooling for visual question answering and visual grounding. In EMNLP (pp. 457–468).

  • Ganea, O., Bécigneul, G., & Hofmann, T. (2018). Hyperbolic entailment cones for learning hierarchical embeddings. In ICML (pp. 1632–1641).

  • García-Durán, A., Bordes, A., & Usunier, N. (2015). Composing relationships with translations. In EMNLP (pp. 286–290).

  • García-Durȧn, A., Bordes, A., Usunier, N., & Grandvalet, Y. (2016). Combining two and three-way embedding models for link prediction in knowledge bases. Journal of Artificial Intelligence Research, 55, 715–742.

    Article  Google Scholar 

  • Graves, A., Wayne, G., & Danihelka, I. (2014). Neural turing machines. CoRR, arXiv:1410.5401.

  • Gruber, T. (2009). Ontology. Encyclopedia of database systems, 1963–1965.

  • Gutiérrez-Basulto, V., & Schockaert, S. (2018). From knowledge graph embedding to ontology embedding? an analysis of the compatibility between vector space representations and rules. In Principles of Knowledge Representation and Reasoning (pp. 379–388).

  • Guu, K., Miller, J., & Liang, P. (2015). Traversing knowledge graphs in vector space. In EMNLP (pp. 318–327).

  • He, S., Liu, K., Ji, G., & Zhao, J. (2015). Learning to represent knowledge graphs with gaussian embedding. In CIKM (pp. 623–632): ACM.

  • He, X., & Golub, D. (2016). Character-level question answering with attention. In EMNLP (pp. 1598–1607).

  • Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. In ACL (pp. 328–339).

  • Huang, H., Zhu, C., Shen, Y., & Chen, W. (2018). Fusionnet: Fusing via fully-aware attention with application to machine comprehension. In ICLR.

  • Huang, K., Altosaar, J., & Ranganath, R. (2019). Clinicalbert: Modeling clinical notes and predicting hospital readmission. CoRR, arXiv:1904.05342.

  • Iyyer, M., Boyd-Graber, J.L., Claudino, L.M.B., & Socher, III. R.H.D. (2014). A neural network for factoid question answering over paragraphs. In EMNLP (pp. 633–644).

  • Jain, S. (2016). Question answering over knowledge base using factual memory networks. In Student Research Workshop, SRW@HLT-NAACL (pp. 109–115).

  • Ji, G., He, S., Xu, L., Liu, K., & Zhao, J. (2015). Knowledge graph embedding via dynamic mapping matrix. In ACL (pp. 687–696).

  • Kadlec, R., Bajgar, O., & Kleindienst, J. (2017). Knowledge base completion: Baselines strike back. In Workshop on Representation Learning for NLP, Rep4NLP@ACL (pp. 69–74).

  • Kafle, S., de Silva, N., & Dou, D. (2019). An overview of utilizing knowledge bases in neural networks for question answering. In IRI, IEEE (pp. 326–333).

  • Kazemi, V., & Elqursh, A. (2017). Show, ask, attend, and answer: A strong baseline for visual question answering. CoRR, arXiv:1704.03162.

  • Kazemi, S.M., & Poole, D. (2018). Simple embedding for link prediction in knowledge graphs. In NeurIPS (pp. 4289–4300.

  • Kratzwald, B., Eigenmann, A., & Feuerriegel, S. (2019). Rankqa: Neural question answering with answer re-ranking. In ACL (pp. 6076–6085).

  • Krompaß, D., Baier, S., & Tresp, V. (2015). Type-constrained representation learning in knowledge graphs. In ISWC (pp. 640–655): Springer.

  • Krótkiewicz, M., Wojtkiewicz, K., & Jodłowiec, M. (2018). Towards semantic knowledge base definition. In International scientific conference BCI 2018 Opole (pp. 218–239): Springer.

  • Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: a lite bert for self-supervised learning of language representations. CoRR, arXiv:1909.11942.

  • LeCun, Y., Bengio, Y., & Hinton, G.E. (2015). Deep learning. Nature, 521(7553), 436–444.

    Article  Google Scholar 

  • Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., & Kang, J. (2019). Biobert: pre-trained biomedical language representation model for biomedical text mining. CoRR, arXiv:1901.08746.

  • Li, H., Wang, P., Shen, C., & van den Hengel, A. (2019). Visual question answering as reading comprehension. In CVPR (pp. 6319–6328).

  • Liang, C., & Forbus, K.D. (2015). Learning plausible inferences from semantic web knowledge by combining analogical generalization with structured logistic regression. In AAAI (pp. 551–557).

  • Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., & Liu, S. (2015a). Modeling relation paths for representation learning of knowledge bases. In EMNLP (pp. 705–714).

  • Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015b). Learning entity and relation embeddings for knowledge graph completion. In AAAI (pp. 2181–2187).

  • Liu, H., & Singh, P. (2004). Conceptnet-a practical commonsense reasoning tool-kit. BT Technology journal, 22(4), 211–226.

    Article  Google Scholar 

  • Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). Roberta: a robustly optimized bert pretraining approach. CoRR, arXiv:1907.11692.

  • Lukovnikov, D., Fischer, A., Lehmann, J., & Auer, S. (2017). Neural network-based question answering over knowledge graphs on word and character level. In WWW (pp. 1211–1220).

  • Luo, Y., Wang, Q., Wang, B., & Guo, L. (2015). Context-dependent knowledge graph embedding. In EMNLP (pp. 1656–1661).

  • Ma, L., Sun, P., Lin, Z., & Wang, H. (2019). Composing knowledge graph embeddings via word embeddings. CoRR, arXiv:1909.03794.

  • Mihaylov, T., Clark, P., Khot, T., & Sabharwal, A. (2018). Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP (pp. 2381–2391).

  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In NeurIPS (pp. 3111–3119).

  • Miller, G.A. (1995). Wordnet: a lexical database for english. Communications of the ACM, 38 (11), 39–41.

    Article  Google Scholar 

  • Min, S., Zhong, V., Zettlemoyer, L., & Hajishirzi, H. (2019). Multi-hop reading comprehension through question decomposition and rescoring. In ACL (pp. 6097–6109).

  • Mitchell, T.M., Cohen, W.W., Jr, E.R.H., Talukdar, P.P., Yang, B., Betteridge, J., Carlson, A., Mishra, B.D., Gardner, M., Kisiel, B., Krishnamurthy, J., Lao, N., Mazaitis, K., Mohamed, T., Nakashole, N., Platanios, E.A., Ritter, A., Samadi, M., Settles, B., Wang, R.C., Wijaya, D., Gupta, A., Chen, X., Saparov, A., Greaves, M., & Welling, J. (2018). Never-ending learning. Communications of the ACM, 61(5), 103–115.

  • Neelakantan, A., Roth, B., & McCallum, A. (2015). Compositional vector space models for knowledge base completion. In ACL (pp. 156–166).

  • Nguyen, D.Q., Sirts, K., Qu, L., & Johnson, M. (2016a). Neighborhood mixture model for knowledge base completion. In CoNLL (pp. 40–50).

  • Nguyen, D.Q., Sirts, K., Qu, L., & Johnson, M. (2016b). Stranse: a novel embedding model of entities and relationships in knowledge bases. In NAACL-HLT (pp. 460–466).

  • Nickel, M., Tresp, V., & Kriegel, H. (2011). A three-way model for collective learning on multi-relational data. In ICML (pp. 809–816).

  • Nickel, M., Tresp, V., & Kriegel, H. (2012). Factorizing YAGO: scalable machine learning for linked data. In WWW (pp. 271–280).

  • Nickel, M., & Tresp, V. (2013). Logistic tensor factorization for multi-relational data. CoRR.

  • Nickel, M., Rosasco, L., & Poggio, T.A. (2016). Holographic embeddings of knowledge graphs. In AAAI (pp. 1955–1961).

  • Nickel, M., & Kiela, D. (2017). Poincaré embeddings for learning hierarchical representations. In NeurIPS (pp. 6338–6347).

  • Nickel, M., & Kiela, D. (2018). Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In ICML (pp. 3776– 3785).

  • Pan, B., Yang, Y., Li, H., Zhao, Z., Zhuang, Y., Cai, D., & He, X. (2018). Macnet: Transferring knowledge from machine comprehension to sequence-to-sequence models. In NeurIPS (pp. 6095–6105).

  • Park, C., Lee, C., Hong, L., Hwang, Y., Yoo, T., Jang, J., Hong, Y., Bae, K.H., & Kim, H.K. (2019). S2-net: Machine reading comprehension with sru-based self-matching networks. ETRI Journal.

  • Pennington, J., Socher, R., & Manning, C.D. (2014). Glove: Global vectors for word representation. In EMNLP (pp. 1532–1543).

  • Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8).

  • Rajpurkar, P., Zhang, J, Lopyrev, K., & Liang, P. (2016). Squad: 100, 000+ questions for machine comprehension of text. In EMNLP (pp. 2383–2392).

  • Reddy, S., Chen, D., & Manning, C.D. (2019). Coqa: A, conversational question answering challenge. TACL, 7, 249–266.

    Article  Google Scholar 

  • Riedel, S., Yao, L., McCallum, A., & Marlin, B.M. (2013). Relation extraction with matrix factorization and universal schemas. In NAACL-HLT (pp. 74–84).

  • Sala, F., Sa, C.D., Gu, A., & Ré, C. (2018). Representation tradeoffs for hyperbolic embeddings. In ICML (pp. 4457–4466).

  • Schlichtkrull, M.S., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018). Modeling relational data with graph convolutional networks. In ESWC (pp. 593–607).

  • Seaborne, A., & Prud’hommeaux, E. (2006). Sparql query language for rdf. W3C recommendation.

  • Shearer, R., Motik, B., & Horrocks, I. (2008). Hermit: A highly-efficient OWL reasoner. In Fifth OWLED Workshop on OWL: Experiences and Directions@ISWC.

  • Shen, Y., Huang, P., Gao, J., & Chen, W. (2017). Reasonet: Learning to stop reading in machine comprehension. In KDD (pp. 1047–1055).

  • Shi, B., & Weninger, T. (2017). Proje: Embedding projection for knowledge graph completion. In AAAI (pp. 1236–1242).

  • Singhal, A. (2012). Introducing the knowledge graph: things, not strings. Official google blog.

  • Socher, R., Chen, D., Manning, C.D., & Ng, A.Y. (2013). Reasoning with neural tensor networks for knowledge base completion. In NeurIPS (pp. 926–934).

  • Suchanek, F.M., Kasneci, G., & Weikum, G. (2007). Yago: a core of semantic knowledge. In WWW, ACM (pp. 697–706).

  • Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015). End-to-end memory networks. In NeurIPS (pp. 2440–2448).

  • Sun, Z., Deng, Z., Nie, J., & Tang, J. (2019). Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR.

  • Sutskever, I., Salakhutdinov, R., & Tenenbaum, J.B. (2009). Modelling relational data using bayesian clustered tensor factorization. In NeurIPS (pp. 1821–1828).

  • Thurston, W.P. (1982). Three dimensional manifolds, kleinian groups and hyperbolic geometry. Bulletin of the American Mathematical Society, 6(3), 357–381.

    Article  Google Scholar 

  • Toutanova, K., Lin, V., Yih, W., Poon, H., & Quirk, C. (2016). Compositional learning of embeddings for relation paths in knowledge base and text. In ACL.

  • Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., & Bouchard, G. (2016). Complex embeddings for simple link prediction. In ICML (pp. 2071–2080).

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In NeurIPS (pp. 5998–6008).

  • Vendrov, I., Kiros, R., Fidler, S., & Urtasun, R. (2016). Order-embeddings of images and language. In ACL.

  • Vilnis, L., Li, X., Murty, S., & McCallum, A. (2018). Probabilistic embedding of knowledge graphs with box lattice measures. In ACL (pp. 263–272).

  • Vinyals, O., Fortunato, M., & Jaitly, N. (2015). Pointer networks. In NeurIPS (pp. 2692–2700).

  • Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017). Gated self-matching networks for reading comprehension and question answering. In ACL (pp. 189–198).

  • Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014). Knowledge graph embedding by translating on hyperplanes. In AAAI, Citeseer (pp. 1112–1119).

  • Wang, Z., & Li, J. (2016). Text-enhanced representation learning for knowledge graph. In IJCAI, AAAI Press (pp. 1293–1299).

  • Wang, Z., Ng, P., Ma, X., Nallapati, R., & Xiang, B. (2019). Multi-passage bert: A globally normalized bert model for open-domain question answering. In EMNLP-IJCNLP (pp. 5881–5885).

  • West, R., Gabrilovich, E., Murphy, K., Sun, S., Gupta, R., & Lin, D. (2014). Knowledge base completion via search-based question answering. In WWW (pp. 515–526).

  • Weston, J., Chopra, S., & Bordes, A. (2015). Memory networks. In ICLR.

  • Wu, Q., Teney, D., Wang, P., Shen, C., Dick, A.R., & van den Hengel, A. (2017). Visual question answering: A survey of methods and datasets. Computer Vision and Image Understanding, 163, 21–40.

    Article  Google Scholar 

  • Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Philip, S.Y. (2020). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems.

  • Xu, K., Reddy, S., Feng, Y., Huang, S., & Zhao, D. (2016). Question answering on freebase via relation extraction and textual evidence. In ACL.

  • Yang, M., Duan, N., Zhou, M., & Rim, H. (2014). Joint relational embeddings for knowledge-based question answering. In EMNLP (pp. 645–650).

  • Yang, B., Yih, W., He, X., Gao, J., & Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. In ICLR.

  • Yang, P., Fang, H., & Lin, J. (2018). Anserini: Reproducible ranking baselines using lucene. Journal of Data and Information Quality (JDIQ), 10(4), 16.

    Google Scholar 

  • Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., & Lin, J. (2019). End-to-end open-domain question answering with bertserini. NAACL-HLT, 72.

  • Yao, X., & Durme, B.V. (2014). Information extraction over structured data: Question answering with freebase. In ACL (pp. 956–966).

  • Yao, X. (2015). Lean question answering over freebase from scratch. In NAACL-HLT (pp. 66–70).

  • Yih, W., He, X., & Meek, C. (2014). Semantic parsing for single-relation question answering. In ACL (pp. 643–648).

  • Yin, J., Jiang, X., Lu, Z., Shang, L., Li, H., & Li, X. (2016a). Neural generative question answering. In IJCAI (pp. 2972– 2978).

  • Yin, W., Yu, M., Xiang, B., Zhou, B., & Schütze, H. (2016b). Simple question answering by attentive convolutional neural network. In COLING (pp. 1746–1756).

  • Yu, A.W., Dohan, D., Luong, M., Zhao, R., Chen, K., Norouzi, M., & Le, Q.V. (2018). Qanet: Combining local convolution with global self-attention for reading comprehension. In ICLR.

  • Zadeh, A., Chan, M., Liang, P.P., Tong, E., & Morency, L. (2019). Social-iq: A question answering benchmark for artificial social intelligence. In CVPR (pp. 8807–8817).

  • Zhang, Z., Zhuang, F., Qu, M., Lin, F., & He, Q. (2018). Knowledge graph embedding with hierarchical relation structure. In EMNLP (pp. 3198–3207).

  • Zhang, S., Tay, Y., Yao, L., & Liu, Q. (2019). Quaternion knowledge graph embeddings. In NeurIPS (pp. 2731–2741).

  • Zhu, C., Zeng, M., & Huang, X. (2018). Sdnet: Contextualized attention-based deep network for conversational question answering. CoRR arXiv:1812.03593.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sabin Kafle.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kafle, S., de Silva, N. & Dou, D. An Overview of Utilizing Knowledge Bases in Neural Networks for Question Answering. Inf Syst Front 22, 1095–1111 (2020). https://doi.org/10.1007/s10796-020-10035-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-020-10035-2

Keywords

Navigation