Obtaining Word Embedding from Existing Classification Model
This paper introduces a new technique to inspect relations between classes in a classification model. The method is built on the assumption that it is easier to distinguish some classes than others. The harder the distinction is, the more similar the objects are. Simple application demonstrating this approach was implemented and obtained class representations in a vector space are discussed. Created representation can be treated as word embedding where the words are represented by the classes. As an addition, potential usages and characteristics are discussed including a knowledge base.
KeywordsUnsupervised learning Artificial intelligence Word embedding Word2vec CNN
This work was supported by the BUT project FIT-S-17-4014 and the IT4IXS: IT4Innovations Excellence in Science project (LQ1602).
- 1.Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
- 2.Kottur, S., Vedantam, R., Moura, J.M.F., Parikh, D.: Visual word2vec (vis-w2v): learning visually grounded word embeddings using abstract scenes. CoRR abs/1511.07067 (2015)Google Scholar
- 3.Xu, R., Lu, J., Xiong, C., Yang, Z., Corso, J.J.: Improving word representations via global visual context. In: NIPS Workshop on Learning Semantics (2014)Google Scholar
- 4.Lazaridou, A., Baroni, M., et al.: Combining language and vision with a multimodal skip-gram model. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 153–163 (2015)Google Scholar
- 5.Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.O. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc., New York (2012)Google Scholar
- 6.Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)Google Scholar
- 7.Baroni, M., Joulin, A., Jabri, A., Kruszewski, G., Lazaridou, A., Simonic, K., Mikolov, T.: Commai: evaluating the first steps towards a useful general AI. CoRR abs/1701.08954 (2017)Google Scholar