Advertisement

Distributional Sentence Entailment Using Density Matrices

  • Esma Balkir
  • Mehrnoosh Sadrzadeh
  • Bob Coecke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9541)

Abstract

Categorical compositional distributional model of Clark, Coecke, and Sadrzadeh suggests a way to combine grammatical composition of the formal, type logical models with the corpus based, empirical word representations of distributional semantics. This paper contributes to the project by expanding the model to also capture entailment relations. This is achieved by extending the representations of words from points in meaning space to density operators, which are probability distributions on the subspaces of the space. A symmetric measure of similarity and an asymmetric measure of entailment is defined, where lexical entailment is measured using von Neumann entropy, the quantum variant of Kullback-Leibler divergence. Lexical entailment, combined with the composition map on word representations, provides a method to obtain entailment relations on the level of sentences. Truth theoretic and corpus-based examples are provided.

Keywords

Target Word Pure State Density Matrice Relative Entropy Monoidal Category 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Abramsky, S., Coecke, B.: A categorical semantics of quantum protocols. In: Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, pp. 415–425. IEEE Computer Science Press (2004). arXiv:quant-ph/0402130
  2. 2.
    Balkır, E.: Using density matrices in a compositional distributional model of meaning. Master’s thesis, University of Oxford (2014)Google Scholar
  3. 3.
    Baroni, M., Bernardi, R., Do, N-Q., Shan, C-C.: Entailment above the word level in distributional semantics. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 23–32. Association of Computational Linguists (2012)Google Scholar
  4. 4.
    Beltagy, I., Chau, C., Boleda, G., Garrette, D., Erk, K., Mooney, R.: Montague meets markov: deep semantics with probabilistic logical form. In: Second Joint Conference on Lexical and Computational Semantics, vol. 1, pp. 11–21. Association of Computational Linguists (2013)Google Scholar
  5. 5.
    Blacoe, W., Kashefi, E., Lapata, M.: A quantum-theoretic approach to distributional semantics. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 847–857 (2013)Google Scholar
  6. 6.
    Bruza, P.D., Cole, R.: Quantum logic of semantic space: an exploratory investigation of context effects in practical reasoning. In: We Will Show Them: Essays in Honour of Dov Gabbay, pp. 339–361 (2005)Google Scholar
  7. 7.
    Clark, S., Coecke, B., Sadrzadeh, M.: A compositional distributional model of meaning. In: Proceedings of the Second Symposium on Quantum Interaction (QI-2008), pp. 133–140 (2008)Google Scholar
  8. 8.
    Clarke, D.: Context-theoretic semantics for natural language: an overview. In: Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pp. 112–119. Association for Computational Linguistics (2009)Google Scholar
  9. 9.
    Coecke, B.: Quantum picturalism. Contemp. Phys. 51(1), 59–83 (2010)CrossRefGoogle Scholar
  10. 10.
    Coecke, B., Sadrzadeh, M., Clark, S.: Mathematical foundations for a compositional distributional model of meaning. Linguist. Anal. 36 (2010)Google Scholar
  11. 11.
    Firth, John R.: A Synopsis of Linguistic Theory, 1930–1955. Studies in Linguistic, Analysis, pp. 1–32 (1957)Google Scholar
  12. 12.
    Geffet, M., Dagan, I.: The distributional inclusion hypotheses and lexical entailment. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 107–114. Association for Computational Linguistics (2005)Google Scholar
  13. 13.
    Glickman, O., Dagan, I., Koppel, M.: Web based probabilistic textual entailment. In: Proceedings of the PASCAL Challenges Workshop on Recognizing Textual Entailment (2005)Google Scholar
  14. 14.
    Grefenstette, E.: Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics. Ph.D. thesis, University of Oxford (2013)Google Scholar
  15. 15.
    Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1394–1404. Association for Computational Linguistics (2011)Google Scholar
  16. 16.
    Herbelot, A., Ganesalingam, M.: Measuring semantic content in distributional vectors. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 440–445. Association for Computational Linguistics (2013)Google Scholar
  17. 17.
    Kotlerman, L., Dagan, I., Szpektor, I., Zhitomirsky-Geffet, M.: Directional distributional similarity for lexical inference. Nat. Lang. Eng. 16(04), 359–389 (2010)CrossRefGoogle Scholar
  18. 18.
    Lambek, J.: Type grammars as pregroups. Grammars 4(1), 21–39 (2001)CrossRefMATHGoogle Scholar
  19. 19.
    Lenci, A., Benotto, G.: Identifying hypernyms in distributional semantic spaces. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics, vol. 2, pp. 75–79. Association for Computational Linguistics (2012)Google Scholar
  20. 20.
    Lin, D.: An information-theoretic definition of similarity. In: Proceedings of the International Conference on Machine Learning, pp. 296–304 (1998)Google Scholar
  21. 21.
    MacCartney, B., Manning, C.D.: Natural logic for textual inference. In: ACL Workshop on Textual Entailment and Paraphrasing, Association for Computational Linguistics (2007)Google Scholar
  22. 22.
    Piedeleu, R., Kartsaklis, D., Coecke, B., Sadrzadeh, M.: Open system categorical quantum semantics in natural language processing (2015). arXiv:1502.00831
  23. 23.
    Santus, E., Lenci, A., Lu, Q., Walde, S.S.I.: Chasing hypernyms in vector spaces with entropy. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, vol. 2, pp. 38–42 (2014)Google Scholar
  24. 24.
    Selinger, P.: Dagger compact closed categories and completely positive maps. Electron. Notes Theoret. Comput. Sci. 170, 139–163 (2007)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Sordoni, A., Nie, J-Y., Bengio, Y.: Modeling term dependencies with quantum language models for ir. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 653–662. Association for Computational Linguistics (2013)Google Scholar
  26. 26.
    Van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, New York (2004)CrossRefMATHGoogle Scholar
  27. 27.
    Weeds, J., Weir, D., McCarthy, D.: Characterising measures of lexical distributional similarity. In: Proceedings of the 20th International Conference on Computational Linguistics, Number 1015. Association for Computational Linguistics (2004)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2016

Authors and Affiliations

  1. 1.Queen Mary University of LondonLondonUK
  2. 2.University of OxfordOxfordUK

Personalised recommendations