Semantic Capture Analysis in Word Embedding Vectors Using Convolutional Neural Network

  • Raúl Navarro-AlmanzaEmail author
  • Guillermo LiceaEmail author
  • Reyes Juárez-RamírezEmail author
  • Olivia MendozaEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 569)


The semantic relation detection among entities from unstructured text is an important task in automatic knowledge construction to discover new knowledge. Word embeddings have been successful in capturing semantic relations among entities in unstructured text. In this work we propose to use WordNet as a knowledge base to extract semantic relations among entities and measure how well word embeddings vectors capture semantic regularities by themselves, using state-of-art classification model to detect semantic relations. We present semantic relation capture f-measure score in word embedding vectors of 94.9%, the semantic relations addressed in this work are taxonomic relations (hypernym-hyponym) and part-of relations (holonym-meronym).


Semantic regularities Word embedding Relation classification Convolutional neural network 



This research was supported/partially supported by MyDCI (Maestría y Doctorado en Ciencias e Ingeniería).


  1. 1.
    Bengio, Y., et al.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003). doi: 10.1162/153244303322533223. ISSN: 15324435. arXiv:1301.3781v3
  2. 2.
    Caraballo, S.A.: Automatic construction of a hypernym-labeled noun hierarchy from text. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, ACL 1999, pp. 120–126. Association for Computational Linguistics, Stroudsburg (1999). ISBN: 1558606093.
  3. 3.
    Chen, Y.-N., et al.: Learning semantic hierarchy with distributed representations for unsupervised spoken language understanding, September 2015Google Scholar
  4. 4.
    Colace, F., et al.: Terminological ontology learning and population using latent Dirichlet allocation. J. Vis. Lang. Comput. 25(6), 818–826 (2014). doi: 10.1016/j.jvlc.2014.11.001. ISSN: 1045926X. CrossRefGoogle Scholar
  5. 5.
    Fan, M., et al.: Probabilistic belief embedding for large-scale knowledge population. Cogn. Comput. 8(6), 1–16 (2016). doi: 10.1007/s12559-016-9425-5. ISSN: 18669964. arXiv:1505.02433v4 CrossRefGoogle Scholar
  6. 6.
    Fu, R., et al.: Learning semantic hierarchies: a continuous vector space approach. IEEE Trans. Audio Speech Lang. Process. 23(3), 461–471 (2015). doi: 10.1109/TASLP.2014.2377580. ISSN: 15587916CrossRefGoogle Scholar
  7. 7.
    Fu, R., et al.: Learning semantic hierarchies via word embeddings. In: ACL, pp. 1199–1209 (2014)Google Scholar
  8. 8.
    Hendrickx, I., et al.: SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, DEW 2009, pp. 94–99. Association for Computational Linguistics, Stroudsburg (2009). ISBN: 978-1-932432-31-2.
  9. 9.
    Hyland, S.L., Karaletsos, T., Rätsch, G.: A generative model of words and relationships from multiple sources. In: Proceedings of the 30th Conference on Artificial Intelligence (AAAI 2016), p. 8 (2016). arXiv:1510.00259
  10. 10.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization, pp. 1–15. arXiv:1412.6980 [cs.LG] (2014)
  11. 11.
    Komninos, A.: Dependency based embeddings for sentence classification tasks. In: Naacl 2016, pp. 1490–1500 (2016)Google Scholar
  12. 12.
    Mikolov, T., Yih, W.-T., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of NAACL-HLT, pp. 746–751, June 2013.
  13. 13.
    Mikolov, T., et al.: Efficient Estimation of Word Representations in Vector Space (2013).
  14. 14.
    Qin, P., Xu, W., Guo, J.: An empirical convolutional neural network approach for semantic relation classification. Neurocomputing 190, 1–9 (2016). doi: 10.1016/j.neucom.2015.12.091. ISSN: 18728286CrossRefGoogle Scholar
  15. 15.
    Rios-Alvarado, A.B., Lopez-Arevalo, I., Sosa-Sosa, V.J.: Learning concept hierarchies from textual resources for ontologies construction. Expert Syst. Appl. 40(15), 5907–5915 (2013). doi: 10.1016/j.eswa.2013.05.005. ISSN: 09574174CrossRefGoogle Scholar
  16. 16.
    dos Santos, C.N., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. In: ACL 2015, vol. 3, pp. 626–634 (2015). arXiv:1504.06580v2.
  17. 17.
    Takase, S., Okazaki, N., Inui, K.: Modeling semantic compositionality of relational patterns. Eng. Appl. Artif. Intell. 50, 256–264 (2016). doi: 10.1016/j.engappai.2016.01.027. ISSN: 09521976CrossRefGoogle Scholar
  18. 18.
    Xiong, S., Ji, D.: Exploiting flexible-constrained K-means clustering with word embedding for aspect-phrase grouping. Inf. Sci. 367–368, 689–699 (2016). doi: 10.1016/j.ins.2016.07.002. ISSN: 00200255CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Universidad Autónoma de Baja CaliforniaTijuanaMexico

Personalised recommendations