Skip to main content
SpringerLink
Log in

Sequence Embedding for Zero or Low Resource Knowledge Graph Completion

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12681))

Included in the following conference series:

  • 2790 Accesses

Abstract

Knowledge graph completion (KGC) has been proposed to improve KGs by filling in missing links. Previous KGC approaches require a large number of training instances (entity and relation) and hold a closed-world assumption. The real case is that very few instances are available and KG evolve quickly with new entities and relations being added by the minute. The newly added cases are zero resource in training. In this work, we propose a Sequence Embedding with Adversarial learning approach (SEwA) for zero or low resource KGC. It transform the KGC into a sequence prediction problem by making full use of inherently link structure of knowledge graph and resource-easy-to-transfer feature of adversarial contextual embedding. Specifically, the triples \((<\!\!h,r,t\!\!>)\) and higher-order triples \((<\!\!h,p,t\!\!>)\) containing the paths \((p= r_1 \rightarrow \cdots \rightarrow r_n)\) are represented as word sequences and are encoded by pre-training model with multi head self-attention. The path is obtained by a non-parametric learning based on the one-class classification of the relation trees. The zero and low resources issues are further optimizes by adversarial learning. At last, our SEwA is evaluated by low resource datasets and open world datasets.

Supported by the Natural Science Foundation of Inner Mongolia in China (2020BS06005, 2018BS06001), the Inner Mongolia Discipline Inspection and Supervision Big Data Laboratory Open Project (IMDBD2020010), and the High-level Talents Scientific Research Foundation of Inner Mongolia University (21500-5195118).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This results are measured by the classic TransE model on the baseline dataset FB15K. FB15K is a subset of Freebase, Freebase is a subset of Wikidata.

  2. 2.

    The results come from 2016 release data in Freebase. The total number of entities in Freebase is 507, 480, 694, of which 271, 330, 531 entities occur only once and 124, 378, 009 entities occur 2 to 4 times. 15449 entities occur from 1257 to 97922175.

  3. 3.

    It is the only new parameters introduced during entity prediction fine-tuning.

  4. 4.

    Note: If a corrupted triple exists in the knowledge graph, it is also correct. It may be ranked above the test triple, but this should not be counted as an error because both triples are true.

References

  1. Balazevic, I., Allen, C., Hospedales, T.M.: Tucker: tensor factorization for knowledge graph completion. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, 3–7 November 2019, pp. 5184–5193 (2019)

    Google Scholar 

  2. Blockeel, H., Raedt, L.D.: Top-down induction of first-order logical decision trees. Artif. Intell. 101(1–2), 285–297 (1998)

    Article  MathSciNet  Google Scholar 

  3. Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held 5–8 December 2013, Lake Tahoe, Nevada, USA, pp. 2787–2795 (2013)

    Google Scholar 

  4. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA (2018)

    Google Scholar 

  5. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019)

    Google Scholar 

  6. Guu, K., Miller, J., Liang, P.: Traversing knowledge graphs in vector space. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, pp. 318–327 (2015)

    Google Scholar 

  7. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, 26–31 July 2015, Beijing, China, Volume 1: Long Papers, pp. 687–696 (2015)

    Google Scholar 

  8. Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, pp. 985–991 (2016)

    Google Scholar 

  9. Khot, T., Natarajan, S., Shavlik, J.W.: Relational one-class classification: A non-parametric approach. In: Brodley, C.E., Stone, P. (eds.) Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 27–31 July 2014, Québec City, Québec, Canada, pp. 2453–2459. AAAI Press (2014)

    Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)

    Google Scholar 

  11. Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, pp. 705–714 (2015)

    Google Scholar 

  12. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 25–30 January 2015, Austin, TX, USA, pp. 2181–2187 (2015)

    Google Scholar 

  13. Mehta, S., Rangwala, H., Ramakrishnan, N.: Low rank factorization for compact multi-head self-attention. CoRR abs/1912.00835 (2019)

    Google Scholar 

  14. Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.Q.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, 1–6 June 2018, Volume 2 (Short Papers), pp. 327–333 (2018)

    Google Scholar 

  15. Nickel, M., Rosasco, L., Poggio, T.A.: Holographic embeddings of knowledge graphs. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 12–17 February, 2016, Phoenix, Arizona, USA, pp. 1955–1961 (2016)

    Google Scholar 

  16. Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X.: Pre-trained models for natural language processing: A survey. CoRR abs/2003.08271 (2020)

    Google Scholar 

  17. Reiter, R.: On closed world data bases. Logic and Data Bases. In: 1977 Symposium on Logic and Data Bases, Centre d’études et de recherches de Toulouse, France, pp. 55–76 (1977)

    Google Scholar 

  18. Shah, H., Villmow, J., Ulges, A., Schwanecke, U., Shafait, F.: An open-world extension to knowledge graph completion models. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, 27 January – 1 February 2019, pp. 3044–3051 (2019)

    Google Scholar 

  19. Shi, B., Weninger, T.: Open-world knowledge graph completion. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, 2–7 February 2018, pp. 1957–1964. AAAI Press (2018)

    Google Scholar 

  20. Sun, Z., Deng, Z., Nie, J., Tang, J.: Rotate: knowledge graph embedding by relational rotation in complex space. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA (2019)

    Google Scholar 

  21. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, 19–24 June 2016, pp. 2071–2080 (2016)

    Google Scholar 

  22. Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 2962–2971. IEEE Computer Society (2017)

    Google Scholar 

  23. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp. 5998–6008 (2017)

    Google Scholar 

  24. Wang, P., Han, J., Li, C., Pan, R.: Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, 27 January – 1 February 2019, pp. 7152–7159 (2019)

    Google Scholar 

  25. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Québec City, Québec, Canada, pp. 1112–1119 (2014)

    Google Scholar 

  26. Xie, R., Liu, Z., Jia, J., Luan, H., Sun, M.: Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, USA, pp. 2659–2665 (2016)

    Google Scholar 

  27. Xiong, W., Yu, M., Chang, S., Guo, X., Wang, W.Y.: One-shot relational learning for knowledge graphs. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October – 4 November 2018, pp. 1980–1990 (2018)

    Google Scholar 

  28. Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. CoRR abs/1412.6575 (2014)

    Google Scholar 

  29. Yao, L., Mao, C., Luo, Y.: KG-BERT: BERT for knowledge graph completion. CoRR abs/1909.03193 (2019)

    Google Scholar 

  30. Zhang, J., Ding, Z., Li, W., Ogunbona, P.: Importance weighted adversarial nets for partial domain adaptation. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 8156–8164. IEEE Computer Society (2018)

    Google Scholar 

  31. Zhang, N., Deng, S., Sun, Z., Chen, J., Zhang, W., Chen, H.: Relation adversarial network for low resource knowledge graph completion. In: WWW 2020: The Web Conference 2020, Taipei, Taiwan, pp. 1–12 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Inner Mongolia University, Hohhot, 010021, China

    Zhijuan Du

  2. Inner Mongolia Discipline Inspection and Supervision Big Data Laboratory, Hohhot, 010015, China

    Zhijuan Du

Authors
  1. Zhijuan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Christian S. Jensen

  2. Singapore Management University, Singapore, Singapore

    Ee-Peng Lim

  3. Academia Sinica, Taipei, Taiwan

    De-Nian Yang

  4. The Pennsylvania State University, University Park, PA, USA

    Wang-Chien Lee

  5. National Chiao Tung University, Hsinchu, Taiwan

    Vincent S. Tseng

  6. Athens University of Economics and Business, Athens, Greece

    Vana Kalogeraki

  7. National Cheng Kung University, Tainan City, Taiwan

    Jen-Wei Huang

  8. National Tsing Hua University, Hsinchu, Taiwan

    Chih-Ya Shen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Du, Z. (2021). Sequence Embedding for Zero or Low Resource Knowledge Graph Completion. In: Jensen, C.S., et al. Database Systems for Advanced Applications. DASFAA 2021. Lecture Notes in Computer Science(), vol 12681. Springer, Cham. https://doi.org/10.1007/978-3-030-73194-6_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-73194-6_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-73193-9

  • Online ISBN: 978-3-030-73194-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation