Abstract
Formal semantics and distributional semantics offer complementary strengths in capturing the meaning of natural language. As such, a considerable amount of research has sought to unify them, either by augmenting formal semantic systems with a distributional component, or by defining a formal system on top of distributed representations. Arriving at such a unified framework has, however, proven extremely challenging. One reason for this is that formal and distributional semantics operate on a fundamentally different ‘representational currency’: formal semantics defines meaning in terms of models of the world, whereas distributional semantics defines meaning in terms of linguistic co-occurrence. Here, we pursue an alternative approach by deriving a vector space model that defines meaning in a distributed manner relative to formal models of the world. We will show that the resulting Distributional Formal Semantics offers probabilistic distributed representations that are also inherently compositional, and that naturally capture quantification and entailment. We moreover show that, when used as part of a neural network model, these representations allow for capturing incremental meaning construction and probabilistic inferencing. This framework thus lays the groundwork for an integrated distributional and formal approach to meaning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
dfs-tools is publicly available at http://github.com/hbrouwer/dfs-tools under the Apache License, version 2.0.
- 2.
cf. The Legend of Zelda: A Link to the Past (Nintendo, 1992).
- 3.
While a constraint is a well-formed formula that specifies its truth-conditions relative to the Light World (\(LV_M\)), its complement specifies its falsehood-conditions relative to the Dark World (\(DV_M\)); e.g., the Light Word constraint \(\forall x. sleep(x)\) can be proven to be violated if \(\exists x. sleep(x)\) is satisfied in the Dark World. See the appendix for a full set of translation rules.
- 4.
The sampling of inconsistent models strongly depends on the interdependency of the constraints in \(\mathcal {C}\) and can be prevented by defining \(\mathcal {C}\) in such a way that all combinations of propositions are explicitly handled.
- 5.
The specification of the world described here, including the definition of the language \(\mathcal {L}\), is available as part of dfs-tools (see Footnote 1).
- 6.
For real-valued vectors, we can calculate the probability of vector \(\mathbf {v}(a)\) as follows: \(P(a) = \sum _i \mathbf {v}_i(a) / |\mathcal {M}|\).
- 7.
Multidimensional scaling from 100 into 3 dimensions necessarily results in a significant loss of information. Therefore, distances between points in the meaning space shown in Fig. 2 should be interpreted with care.
References
Baroni, M., Bernardi, R., Zamparelli, R.: Frege in space: a program of compositional distributional semantics. Linguist. Issues Lang. Technol. (LiLT) 9, 241–346 (2014)
Baroni, M., Zamparelli, R.: Nouns are vectors, adjectives are matrices: representing adjective-noun constructions in semantic space. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1183–1193. Association for Computational Linguistics (2010)
Boleda, G., Herbelot, A.: Formal distributional semantics: introduction to the special issue. Comput. Linguist. 42(4), 619–635 (2016)
Bos, J., Basile, V., Evang, K., Venhuizen, N.J., Bjerva, J.: The Groningen Meaning Bank. In: Ide, N., Pustejovsky, J. (eds.) Handbook of Linguistic Annotation, pp. 463–496. Springer, Dordrecht (2017). https://doi.org/10.1007/978-94-024-0881-2_18
Brouwer, H., Crocker, M.W., Venhuizen, N.J., Hoeks, J.C.J.: A neurocomputational model of the N400 and the P600 in language processing. Cogn. Sci. 41, 1318–1352 (2017). https://doi.org/10.1111/cogs.12461
Calvillo, J., Brouwer, H., Crocker, M.W.: Connectionist semantic systematicity in language production. In: Papafragou, A., Grodner, D., Mirman, D., Trueswell, J.C. (eds.) Proceedings of the 38th Annual Conference of the Cognitive Science Society, Austin, TX, pp. 2555–3560 (2016)
Coecke, M.S.B., Clark, S.: Mathematical foundations for a compositional distributed model of meaning. In: Lambek Festschrift, Linguistic Analysis, vol. 36 (2010)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
Erk, K.: What do you know about an alligator when you know the company it keeps? Semant. Pragmat. 9(17), 1–63 (2016). https://doi.org/10.3765/sp.9.17
Firth, J.R.: A synopsis of linguistic theory, 1930–1955. In: Studies in linguistic analysis. Philological Society, Oxford (1957)
Frank, S.L., Haselager, W.F.G., van Rooij, I.: Connectionist semantic systematicity. Cognition 110(3), 358–379 (2009)
Frank, S.L., Koppen, M., Noordman, L.G.M., Vonk, W.: Modeling knowledge-based inferences in story comprehension. Cogn. Sci. 27(6), 875–910 (2003)
Frank, S.L., Vigliocco, G.: Sentence comprehension as mental simulation: an information-theoretic perspective. Information 2(4), 672–696 (2011)
Frege, G.: Über Sinn und Bedeutung. Zeitschrift für Philosophie und philosophische Kritik 100, 25–50 (1892)
Golden, R.M., Rumelhart, D.E.: A parallel distributed processing model of story comprehension and recall. Discourse Process. 16(3), 203–237 (1993)
Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1394–1404. Association for Computational Linguistics (2011)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211–240 (1997)
Rohde, D.L.T.: A connectionist model of sentence comprehension and production. Ph.D. thesis, Carnegie Mellon University (2002)
Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1201–1211. Association for Computational Linguistics (2012)
Venhuizen, N.J., Bos, J., Hendriks, P., Brouwer, H.: Discourse semantics with information structure. J. Semant. 35(1), 127–169 (2018). https://doi.org/10.1093/jos/ffx017
Venhuizen, N.J., Crocker, M.W., Brouwer, H.: Expectation-based comprehension: modeling the interaction of world knowledge and linguistic experience. Discourse Process. 56(3), 229–255 (2019). https://doi.org/10.1080/0163853X.2018.1448677
Wanzare, L.D., Zarcone, A., Thater, S., Pinkal, M.: DeScript: a crowdsourced corpus for the acquisition of high-quality script knowledge. In: The International Conference on Language Resources and Evaluation (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
The complement of any well-formed formula is found by recursively applying the following translations, where \(\phi '\) is the complement of \(\phi \):
Rights and permissions
Copyright information
© 2019 Springer-Verlag GmbH Germany, part of Springer Nature
About this paper
Cite this paper
Venhuizen, N.J., Hendriks, P., Crocker, M.W., Brouwer, H. (2019). A Framework for Distributional Formal Semantics. In: Iemhoff, R., Moortgat, M., de Queiroz, R. (eds) Logic, Language, Information, and Computation. WoLLIC 2019. Lecture Notes in Computer Science(), vol 11541. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-59533-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-662-59533-6_39
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-59532-9
Online ISBN: 978-3-662-59533-6
eBook Packages: Computer ScienceComputer Science (R0)