Advertisement

A Framework for Distributional Formal Semantics

  • Noortje J. VenhuizenEmail author
  • Petra Hendriks
  • Matthew W. Crocker
  • Harm Brouwer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11541)

Abstract

Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at such a unified framework has, however, proven extremely challenging. One reason for this is that formal and distributional semantics operate on a fundamentally different ‘representational currency’: formal semantics defines meaning in terms of models of the world, whereas distributional semantics defines meaning in terms of linguistic co-occurrence. Here, we pursue an alternative approach by deriving a vector space model that defines meaning in a distributed manner relative to formal models of the world. We will show that the resulting Distributional Formal Semantics offers probabilistic distributed representations that are also inherently compositional, and that naturally capture quantification and entailment. We moreover show that, when used as part of a neural network model, these representations allow for capturing incremental meaning construction and probabilistic inferencing. This framework thus lays the groundwork for an integrated distributional and formal approach to meaning.

Keywords

Distributionality Compositionality Probability Inference Incrementality 

References

  1. 1.
    Baroni, M., Bernardi, R., Zamparelli, R.: Frege in space: a program of compositional distributional semantics. Linguist. Issues Lang. Technol. (LiLT) 9, 241–346 (2014)Google Scholar
  2. 2.
    Baroni, M., Zamparelli, R.: Nouns are vectors, adjectives are matrices: representing adjective-noun constructions in semantic space. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1183–1193. Association for Computational Linguistics (2010)Google Scholar
  3. 3.
    Boleda, G., Herbelot, A.: Formal distributional semantics: introduction to the special issue. Comput. Linguist. 42(4), 619–635 (2016)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bos, J., Basile, V., Evang, K., Venhuizen, N.J., Bjerva, J.: The Groningen Meaning Bank. In: Ide, N., Pustejovsky, J. (eds.) Handbook of Linguistic Annotation, pp. 463–496. Springer, Dordrecht (2017).  https://doi.org/10.1007/978-94-024-0881-2_18CrossRefGoogle Scholar
  5. 5.
    Brouwer, H., Crocker, M.W., Venhuizen, N.J., Hoeks, J.C.J.: A neurocomputational model of the N400 and the P600 in language processing. Cogn. Sci. 41, 1318–1352 (2017).  https://doi.org/10.1111/cogs.12461CrossRefGoogle Scholar
  6. 6.
    Calvillo, J., Brouwer, H., Crocker, M.W.: Connectionist semantic systematicity in language production. In: Papafragou, A., Grodner, D., Mirman, D., Trueswell, J.C. (eds.) Proceedings of the 38th Annual Conference of the Cognitive Science Society, Austin, TX, pp. 2555–3560 (2016)Google Scholar
  7. 7.
    Coecke, M.S.B., Clark, S.: Mathematical foundations for a compositional distributed model of meaning. In: Lambek Festschrift, Linguistic Analysis, vol. 36 (2010)Google Scholar
  8. 8.
    Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)CrossRefGoogle Scholar
  9. 9.
    Erk, K.: What do you know about an alligator when you know the company it keeps? Semant. Pragmat. 9(17), 1–63 (2016).  https://doi.org/10.3765/sp.9.17CrossRefGoogle Scholar
  10. 10.
    Firth, J.R.: A synopsis of linguistic theory, 1930–1955. In: Studies in linguistic analysis. Philological Society, Oxford (1957)Google Scholar
  11. 11.
    Frank, S.L., Haselager, W.F.G., van Rooij, I.: Connectionist semantic systematicity. Cognition 110(3), 358–379 (2009)CrossRefGoogle Scholar
  12. 12.
    Frank, S.L., Koppen, M., Noordman, L.G.M., Vonk, W.: Modeling knowledge-based inferences in story comprehension. Cogn. Sci. 27(6), 875–910 (2003)CrossRefGoogle Scholar
  13. 13.
    Frank, S.L., Vigliocco, G.: Sentence comprehension as mental simulation: an information-theoretic perspective. Information 2(4), 672–696 (2011)CrossRefGoogle Scholar
  14. 14.
    Frege, G.: Über Sinn und Bedeutung. Zeitschrift für Philosophie und philosophische Kritik 100, 25–50 (1892)Google Scholar
  15. 15.
    Golden, R.M., Rumelhart, D.E.: A parallel distributed processing model of story comprehension and recall. Discourse Process. 16(3), 203–237 (1993)CrossRefGoogle Scholar
  16. 16.
    Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1394–1404. Association for Computational Linguistics (2011)Google Scholar
  17. 17.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  18. 18.
    Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211–240 (1997)CrossRefGoogle Scholar
  19. 19.
    Rohde, D.L.T.: A connectionist model of sentence comprehension and production. Ph.D. thesis, Carnegie Mellon University (2002)Google Scholar
  20. 20.
    Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1201–1211. Association for Computational Linguistics (2012)Google Scholar
  21. 21.
    Venhuizen, N.J., Bos, J., Hendriks, P., Brouwer, H.: Discourse semantics with information structure. J. Semant. 35(1), 127–169 (2018).  https://doi.org/10.1093/jos/ffx017CrossRefGoogle Scholar
  22. 22.
    Venhuizen, N.J., Crocker, M.W., Brouwer, H.: Expectation-based comprehension: modeling the interaction of world knowledge and linguistic experience. Discourse Process. 56(3), 229–255 (2019).  https://doi.org/10.1080/0163853X.2018.1448677CrossRefGoogle Scholar
  23. 23.
    Wanzare, L.D., Zarcone, A., Thater, S., Pinkal, M.: DeScript: a crowdsourced corpus for the acquisition of high-quality script knowledge. In: The International Conference on Language Resources and Evaluation (2016)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Noortje J. Venhuizen
    • 1
    Email author
  • Petra Hendriks
    • 2
  • Matthew W. Crocker
    • 1
  • Harm Brouwer
    • 1
  1. 1.Saarland UniversitySaarbrückenGermany
  2. 2.University of GroningenGroningenThe Netherlands

Personalised recommendations