Abstract
Learning ontological relations is an important step on the way to automatically developing ontologies. This paper introduces a novel way to exploit WordNet [16], the combination of pre-trained word embeddings and deep neural networks for the task of ontological relation classification. The data from WordNet and the knowledge encapsulated in the pre-trained word vectors are combined into an enriched dataset. In this dataset a pair of terms that are linked in WordNet through some ontological relation are represented by their word embeddings. A Deep Neural Network uses this dataset to learn the classification of ontological relations based on the word embeddings. The implementation of this approach has yielded encouraging results, which should help the ontology learning research community develop tools for ontological relation extraction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The “words” are actually terms, which could be multi-word terms. For the sake of simplicity, we use “word” to refer to term/lemma in this article.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
For the hypernym class, we did not take all the available data in this experiment; this was done in order to make the data set more balanced with respect to the rest of the considered classes.
References
Agarap, A.F.: Deep learning using rectified linear units (ReLU). arXiv preprint arXiv:1803.08375 (2018)
Dauphin, Y.N., de Vries, H., Bengio, Y.: Equilibrated adaptive learning rates for non-convex optimization. arXiv preprint arXiv:1502.04390 (2015)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
Bui, V.T., Nguyen, P.T., Pham, V.L., Ngo, T.Q.: A neural network model for efficient antonymy-synonymy classification by exploiting co-occurrence contexts and word-structure patterns. Int. J. Intell. Eng. Syst. 13(1), 156–166 (2020)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dunne, R.A., Campbell, N.A.: On the pairing of the softmax activation and cross-entropy penalty functions and the derivation of the softmax activation function. In: Proceedings of the 8th Australian Conference on the Neural Networks, Melbourne, vol. 181, p. 185. Citeseer (1997)
Gábor, K., Buscaldi, D., Schumann, A.K., QasemiZadeh, B., Zargayouna, H., Charnois, T.: Semeval-2018 task 7: semantic relation extraction and classification in scientific papers. In: Proceedings of the 12th International Workshop on Semantic Evaluation, pp. 679–688 (2018)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)
Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: Proceedings of the 14th Conference on Computational Linguistics, vol. 2, pp. 539–545. Association for Computational Linguistics (1992)
Hendrickx, I., et al.: SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 5th International Workshop on Semantic Evaluation, pp. 33–38. Association for Computational Linguistics, Uppsala (2010). https://www.aclweb.org/anthology/S10-1006
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Li, Q., Li, L., Wang, W., Li, Q., Zhong, J.: A comprehensive exploration of semantic relation extraction via pre-trained CNNs. Knowl.-Based Syst. 194, 105488 (2020). https://doi.org/10.1016/j.knosys.2020.105488. ISSN 0950-7051
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Miller, G.A.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Navigli, R., Ponzetto, S.P.: Babelnet: the automatic construction, evaluation and application of a wide-coverage multilingual semantic network. Artif. Intell. 193, 217–250 (2012)
Qin, P., Xu, W., Guo, J.: An empirical convolutional neural network approach for semantic relation classification. Neurocomputing 190, 1–9 (2016)
dos Santos, C., Tan, M., Xiang, B., Zhou, B.: Attentive pooling networks. arXiv preprint arXiv:1602.03609 (2016)
Shijia, E., Jia, S., Xiang, Y.: Study on the Chinese word semantic relation classification with word embedding. In: National CCF Conference on Natural Language Processing and Chinese Computing, pp. 849–855. Springer, Cham (2017)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wu, S., He, Y.: Enriching pre-trained language model with entity information for relation classification. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 2361–2364 (2019)
Wu, Y., Zhang, M.: Overview of the NLPCC 2017 shared task: Chinese word semantic relation classification. In: National CCF Conference on Natural Language Processing and Chinese Computing, pp. 919–925. Springer, Cham (2017)
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, The 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335–2344. Dublin City University and Association for Computational Linguistics, Dublin, August 2014. https://www.aclweb.org/anthology/C14-1220
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Khadir, A.C., Guessoum, A., Aliane, H. (2021). Ontological Relation Classification Using WordNet, Word Embeddings and Deep Neural Networks. In: Chikhi, S., Amine, A., Chaoui, A., Saidouni, D., Kholladi, M. (eds) Modelling and Implementation of Complex Systems. MISC 2020. Lecture Notes in Networks and Systems, vol 156. Springer, Cham. https://doi.org/10.1007/978-3-030-58861-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-58861-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58860-1
Online ISBN: 978-3-030-58861-8
eBook Packages: EngineeringEngineering (R0)