Latent Relational Model for Relation Extraction

  • Gaetano RossielloEmail author
  • Alfio Gliozzo
  • Nicolas Fauceglia
  • Giovanni Semeraro
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11503)


Analogy is a fundamental component of the way we think and process thought. Solving a word analogy problem, such as mason is to stone as carpenter is to wood, requires capabilities in recognizing the implicit relations between the two word pairs. In this paper, we describe the analogy problem from a computational linguistics point of view and explore its use to address relation extraction tasks. We extend a relational model that has been shown to be effective in solving word analogies and adapt it to the relation extraction problem. Our experiments show that this approach outperforms the state-of-the-art methods on a relation extraction dataset, opening up a new research direction in discovering implicit relations in text through analogical reasoning.


Information extraction Distributional semantics 



This work was conducted during an internship at the IBM Thomas J. Watson Research Center in Yorktown Heights, NY, USA. We thank Anastas Stoyanovsky, Steven Pritko and Gabe Hart, software engineers at the IBM Watson Groups in Pittsburgh and Denver, USA, for helping and inspiring us during the “Fast Domain Adaptation in IBM Watson Discovery” project.


  1. 1.
    Agichtein, E., Gravano, L.: Snowball: extracting relations from large plain-text collections. In: ACM DL, pp. 85–94 (2000)Google Scholar
  2. 2.
    Bollacker, K.D., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD Conference, pp. 1247–1250. ACM (2008)Google Scholar
  3. 3.
    Church, K.W.: Word2vec. Nat. Lang. Eng. 23(1), 155–162 (2017)CrossRefGoogle Scholar
  4. 4.
    Deerwester, S.C., Dumais, S.T., Landauer, T.K., Furnas, G.W., Harshman, R.A.: Indexing by latent semantic analysis. JASIS 41(6), 391–407 (1990)CrossRefGoogle Scholar
  5. 5.
    Gladkova, A., Drozd, A., Matsuoka, S.: Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t. In: SRW@HLT-NAACL, pp. 8–15. The Association for Computational Linguistics (2016)Google Scholar
  6. 6.
    Glass, M., Gliozzo, A., Hassanzadeh, O., Mihindukulasooriya, N., Rossiello, G.: Inducing implicit relations from text using distantly supervised deep nets. In: Vrandečić, D., et al. (eds.) ISWC 2018. LNCS, vol. 11136, pp. 38–55. Springer, Cham (2018). Scholar
  7. 7.
    Gliozzo, A.M., Strapparava, C.: Semantic Domains in Computational Linguistics. Springer, Heidelberg (2009). Scholar
  8. 8.
    Halko, N., Martinsson, P., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review 53(2), 217–288 (2011)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Harris, Z.: Distributional structure. Word 10(23), 146–162 (1954)CrossRefGoogle Scholar
  10. 10.
    Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: COLING, pp. 539–545 (1992)Google Scholar
  11. 11.
    Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L.S., Weld, D.S.: Knowledge-based weak supervision for information extraction of overlapping relations. In: ACL, pp. 541–550. The Association for Computer Linguistics (2011)Google Scholar
  12. 12.
    Jiang, J., Zhai, C.: A systematic exploration of the feature space for relation extraction. In: HLT-NAACL, pp. 113–120. The Association for Computational Linguistics (2007)Google Scholar
  13. 13.
    Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis. Discourse Processes 25(2–3), 259–284 (1998)CrossRefGoogle Scholar
  14. 14.
    Levy, O., Goldberg, Y.: Linguistic regularities in sparse and explicit word representations. In: CoNLL, pp. 171–180. ACL (2014)Google Scholar
  15. 15.
    Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: ACL. The Association for Computer Linguistics (2016)Google Scholar
  16. 16.
    Linzen, T.: Issues in evaluating semantic spaces using word analogies. In: RepEval@ACL, pp. 13–18. Association for Computational Linguistics (2016)Google Scholar
  17. 17.
    Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)zbMATHGoogle Scholar
  18. 18.
    Mausam, Schmitz, M., Soderland, S., Bart, R., Etzioni, O.: Open language learning for information extraction. In: EMNLP-CoNLL, pp. 523–534. ACL (2012)Google Scholar
  19. 19.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)Google Scholar
  20. 20.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: ACL/IJCNLP, pp. 1003–1011. The Association for Computer Linguistics (2009)Google Scholar
  21. 21.
    Nguyen, T.H., Grishman, R.: Relation extraction: perspective from convolutional neural networks. In: VS@HLT-NAACL, pp. 39–48. The Association for Computational Linguistics (2015)Google Scholar
  22. 22.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP, pp. 1532–1543. ACL (2014)Google Scholar
  23. 23.
    Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6323, pp. 148–163. Springer, Heidelberg (2010). Scholar
  24. 24.
    Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: HLT-NAACL, pp. 74–84. The Association for Computational Linguistics (2013)Google Scholar
  25. 25.
    Sahlgren, M.: An introduction to random indexing (2005)Google Scholar
  26. 26.
    Sun, L., Han, X.: A feature-enriched tree kernel for relation extraction. In: ACL, vol. 2, pp. 61–67. The Association for Computer Linguistics (2014)Google Scholar
  27. 27.
    Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: EMNLP-CoNLL, pp. 455–465. ACL (2012)Google Scholar
  28. 28.
    Turney, P.D.: Similarity of semantic relations. Comput. Linguist. 32(3), 379–416 (2006)CrossRefGoogle Scholar
  29. 29.
    Turney, P.D., Littman, M.L.: Corpus-based learning of analogies and semantic relations. Mach. Learn. 60(1–3), 251–278 (2005)CrossRefGoogle Scholar
  30. 30.
    Verga, P., McCallum, A.: Row-less universal schema. In: AKBC@NAACL-HLT, pp. 63–68. The Association for Computer Linguistics (2016)Google Scholar
  31. 31.
    Vylomova, E., Rimell, L., Cohn, T., Baldwin, T.: Take and took, gaggle and goose, book and read: evaluating the utility of vector differences for lexical relation learning. In: ACL. The Association for Computer Linguistics (2016)Google Scholar
  32. 32.
    Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: EMNLP, pp. 1753–1762. The Association for Computational Linguistics (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Gaetano Rossiello
    • 1
    Email author
  • Alfio Gliozzo
    • 2
  • Nicolas Fauceglia
    • 2
  • Giovanni Semeraro
    • 1
  1. 1.Department of Computer ScienceUniversity of BariBariItaly
  2. 2.IBM Research AIYorktown HeightsUSA

Personalised recommendations