Advertisement

A Compositional Distributional Inclusion Hypothesis

  • Dimitri Kartsaklis
  • Mehrnoosh Sadrzadeh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10054)

Abstract

The distributional inclusion hypothesis provides a pragmatic way of evaluating entailment between word vectors as represented in a distributional model of meaning. In this paper, we extend this hypothesis to the realm of compositional distributional semantics, where meanings of phrases and sentences are computed by composing their word vectors. We present a theoretical analysis for how feature inclusion is interpreted under each composition operator, and propose a measure for evaluating entailment at the phrase/sentence level. We perform experiments on four entailment datasets, showing that intersective composition in conjunction with our proposed measure achieves the highest performance.

Keywords

Computational linguistics Artificial intelligence Natural language processing Textual entailment Inclusion hypothesis Compositionality Distributional models 

References

  1. 1.
    Balkır, E., Kartsaklis, D., Sadrzadeh, M.: Sentence entailment in compositional distributional semantics. In: Proceedings of the International Symposium on Artificial Intelligence and Mathematics (ISAIM). Fort Lauderdale, FL., (January 2016)Google Scholar
  2. 2.
    Baroni, M., Bernardi, R., Do, N.Q., Shan, C.C.: Entailment above the word level in distributional semantics. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 23–32. Association for Computational Linguistics, Avignon, France, April 2012. http://www.aclweb.org/anthology/E12-1004
  3. 3.
    Baroni, M., Zamparelli, R.: Nouns are vectors, adjectives are matrices: representing adjective-noun constructions in semantic space. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1183–1193. Association for Computational Linguistics, Cambridge, MA, October 2010. http://www.aclweb.org/anthology/D10-1115
  4. 4.
    Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, ACL 1996, pp. 310–318. Association for Computational Linguistics, Stroudsburg, PA, USA (1996). http://dx.doi.org/10.3115/981863.981904
  5. 5.
    Cheng, J., Kartsaklis, D.: Syntax-aware multi-sense word embeddings for deep compositional models of meaning. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1531–1542. Association for Computational Linguistics, Lisbon, Portugal, September 2015. http://aclweb.org/anthology/D15-1177
  6. 6.
    Clarke, D.: Context-theoretic semantics for natural language: an overview. In: Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pp. 112–119. Association for Computational Linguistics, Athens, Greece, March 2009. http://www.aclweb.org/anthology/W09-0215
  7. 7.
    Coecke, B., Sadrzadeh, M., Clark, S.: Mathematical foundations for a compositional distributional model of meaning. Linguist. Anal. 36, 345–384 (2010)Google Scholar
  8. 8.
    Dagan, I., Lee, L., Pereira, F.C.N.: Similarity-based models of word cooccurrence probabilities. Mach. Learn. 34(1–3), 43–69 (1999). doi: 10.1023/A:1007537716579 CrossRefzbMATHGoogle Scholar
  9. 9.
    Geffet, M., Dagan, I.: The distributional inclusion hypotheses and lexical entailment. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pp. 107–114. Association for Computational Linguistics, Ann Arbor, Michigan, June 2005. http://www.aclweb.org/anthology/P05-1014
  10. 10.
    Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1394–1404. Association for Computational Linguistics (2011)Google Scholar
  11. 11.
    Herbelot, A., Ganesalingam, M.: Measuring semantic content in distributional vectors. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 440–445. Association for Computational Linguistics (2013)Google Scholar
  12. 12.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 655–665. Association for Computational Linguistics, Baltimore, Maryland, June 2014. http://www.aclweb.org/anthology/P14-1062
  13. 13.
    Kartsaklis, D., Sadrzadeh, M., Pulman, S.: A unified sentence space for categorical distributional-compositional semantics: theory and experiments. In: COLING 2012, 24th International Conference on Computational Linguistics, Proceedings of the Conference: Posters, 8–15 December 2012, Mumbai, India, pp. 549–558 (2012)Google Scholar
  14. 14.
    Kotlerman, L., Dagan, I., Szpektor, I., Zhitomirsky-Geffet, M.: Directional distributional similarity for lexical inference. Nat. Lang. Eng. 16(04), 359–389 (2010)CrossRefGoogle Scholar
  15. 15.
    Lee, L.: Measures of distributional similarity. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics, pp. 25–32 (1999)Google Scholar
  16. 16.
    Lin, D.: An information-theoretic definition of similarity. In: Proceedings of the International Conference on Machine Learning, pp. 296–304 (1998)Google Scholar
  17. 17.
    MacCartney, B., Manning, C.D.: Natural logic for textual inference. In: ACL Workshop on Textual Entailment and Paraphrasing. Association for Computational Linguistics (2007)Google Scholar
  18. 18.
    Milajevs, D., Kartsaklis, D., Sadrzadeh, M., Purver, M.: Evaluating neural word representations in tensor-based compositional settings. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 708–719. Association for Computational Linguistics, Doha, Qatar, October 2014. http://www.aclweb.org/anthology/D14-1079
  19. 19.
    Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cogn. Sci. 34(8), 1388–1439 (2010)CrossRefGoogle Scholar
  20. 20.
    Powers, D.M.: Evaluation: from precision, recall and f-measure to ROC, informedness, markedness and correlation. J. Mach. Learn. Technol. 2(1), 37–63 (2011)MathSciNetGoogle Scholar
  21. 21.
    Socher, R., Huval, B., Manning, C.A.N.: Semantic compositionality through recursive matrix-vector spaces. In: 2012 Conference on Empirical Methods in Natural Language Processing (2012)Google Scholar
  22. 22.
    Turney, P.D., Pantel, P.: From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37(1), 141–188 (2010)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Weeds, J., Weir, D., McCarthy, D.: Characterising measures of lexical distributional similarity. In: Proceedings of the 20th International Conference on Computational Linguistics, no. 1015. Association for Computational Linguistics (2004)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2016

Authors and Affiliations

  1. 1.School of Electronic Engineering and Computer ScienceQueen Mary University of LondonLondonUK

Personalised recommendations