Obtaining Word Embedding from Existing Classification Model

Conference paper
First Online:
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 736)

Abstract

This paper introduces a new technique to inspect relations between classes in a classification model. The method is built on the assumption that it is easier to distinguish some classes than others. The harder the distinction is, the more similar the objects are. Simple application demonstrating this approach was implemented and obtained class representations in a vector space are discussed. Created representation can be treated as word embedding where the words are represented by the classes. As an addition, potential usages and characteristics are discussed including a knowledge base.

Keywords

Unsupervised learning Artificial intelligence Word embedding Word2vec CNN 
This is a preview of subscription content, log in to check access

Notes

Acknowledgment

This work was supported by the BUT project FIT-S-17-4014 and the IT4IXS: IT4Innovations Excellence in Science project (LQ1602).

References

  1. 1.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  2. 2.
    Kottur, S., Vedantam, R., Moura, J.M.F., Parikh, D.: Visual word2vec (vis-w2v): learning visually grounded word embeddings using abstract scenes. CoRR abs/1511.07067 (2015)Google Scholar
  3. 3.
    Xu, R., Lu, J., Xiong, C., Yang, Z., Corso, J.J.: Improving word representations via global visual context. In: NIPS Workshop on Learning Semantics (2014)Google Scholar
  4. 4.
    Lazaridou, A., Baroni, M., et al.: Combining language and vision with a multimodal skip-gram model. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 153–163 (2015)Google Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.O. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc., New York (2012)Google Scholar
  6. 6.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)Google Scholar
  7. 7.
    Baroni, M., Joulin, A., Jabri, A., Kruszewski, G., Lazaridou, A., Simonic, K., Mikolov, T.: Commai: evaluating the first steps towards a useful general AI. CoRR abs/1701.08954 (2017)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.FIT, IT4Innovations Centre of ExcellenceBrno University of TechnologyBrnoCzech Republic

Personalised recommendations