Skip to main content

Exploiting Synonymy and Hypernymy to Learn Efficient Meaning Representations

  • 2026 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10075)

Abstract

Word representation learning methods such as word2vec usually associate one vector per word; however, in order to face polysemy problems, it’s important to produce distributed representations for each meaning, not for each surface form of a word. In this paper, we propose an extension for the existing AutoExtend model, an auto-encoder architecture that utilises synonymy relations to learn sense representations. We introduce a new layer in the architecture to exploit hypernymy relations predominantly present in existing ontologies. We evaluate the quality of the obtained vectors on word-sense disambiguation tasks and show that the use of the hypernymy relation leads to improvements of 1.2 % accuracy on Senseval-3 and 0.8 % on Semeval-2007 English lexical sample tasks, compared to the original model.

Keywords

  • Sense embedding
  • Semantic relation
  • Auto-encoder
  • Hypernymy
  • Word-sense disambiguation

This work was supported by JSPS KAKENHI Grants Number 15H02754 and 16K12546.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-49304-6_17
  • Chapter length: 7 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-49304-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Fig. 1.

References

  1. Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R.M., Makhoul, J.: Fast and robust neural network joint models for statistical machine translation. In: ACL (1), pp. 1370–1380. Citeseer (2014)

    Google Scholar 

  2. Jauhar, S.K., Dyer, C., Hovy, E.: Ontologically grounded multi-sense representation learning for semantic vector space models. In: Proceedings NAACL, vol. 1 (2015)

    Google Scholar 

  3. Mihalcea, R., Chklovski, T.A., Kilgarriff, A.: The senseval-3 english lexical sample task. In: Association for Computational Linguistics (2004)

    Google Scholar 

  4. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp. 3111–3119 (2013)

    Google Scholar 

  5. Miller, G.A.: WORDNET: A lexical database for english. In: Human Language Technology, Proceedings of a Workshop held at Plainsboro, New Jerey, USA, March 8–11, 1994. http://acl.ldc.upenn.edu/H/H94/H94-1111.pdf

  6. Neelakantan, A., Shankar, J., Passos, A., McCallum, A.: Efficient non-parametric estimation of multiple embeddings per word in vector space (2015). arXiv preprint: arXiv:1504.06654

  7. Pradhan, S.S., Loper, E., Dligach, D., Palmer, M.: Semeval-2007 task 17: English lexical sample, srl and all words. In: Proceedings of the 4th International Workshop on Semantic Evaluations, pp. 87–92. ACL (2007)

    Google Scholar 

  8. Qiu, S., Cui, Q., Bian, J., Gao, B., Liu, T.Y.: Co-learning of word representations and morpheme representations. In: COLING, pp. 141–150 (2014)

    Google Scholar 

  9. Reisinger, J., Mooney, R.J.: Multi-prototype vector-space models of word meaning. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 109–117. Association for Computational Linguistics (2010)

    Google Scholar 

  10. Rothe, S., Schütze, H.: Autoextend: Extending word embeddings to embeddings for synsets and lexemes. In: Proceedings of the ACL (2015)

    Google Scholar 

  11. Turney, P.D., Littman, M.L.: Measuring praise and criticism: Inference of semantic orientation from association. ACM Trans. Inf. Syst. 21(4), 315–346 (2003). http://doi.acm.org/10.1145/944012.944013

    CrossRef  Google Scholar 

  12. Xu, C., Bai, Y., Bian, J., Gao, B., Wang, G., Liu, X., Liu, T.Y.: Rc-net: A general framework for incorporating knowledge into word representations. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1219–1228. ACM (2014)

    Google Scholar 

  13. Zhong, Z., Ng, H.T.: It makes sense: A wide-coverage word sense disambiguation system for free text. In: Proceedings of the ACL 2010 System Demonstrations, pp. 78–83. Association for Computational Linguistics (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas Perianin .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Perianin, T., Senuma, H., Aizawa, A. (2016). Exploiting Synonymy and Hypernymy to Learn Efficient Meaning Representations. In: Morishima, A., Rauber, A., Liew, C. (eds) Digital Libraries: Knowledge, Information, and Data in an Open Access Society. ICADL 2016. Lecture Notes in Computer Science(), vol 10075. Springer, Cham. https://doi.org/10.1007/978-3-319-49304-6_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49304-6_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49303-9

  • Online ISBN: 978-3-319-49304-6

  • eBook Packages: Computer ScienceComputer Science (R0)