Abstract
Formal semantic theories are designed to explain how it is possible to produce and understand an infinite number of sentences on the basis of a finite lexicon and a finite number of composition rules. According to this architecture, language comprehension completely proceeds in a bottom-up fashion only driven by linear linguistic input thereby leaving no room for a predictive component which allows to make expectations about upcoming words. This is in stark contrast to neurophysiological research in the past decades on online semantic processing which has provided ample evidence that prediction plays indeed an indispensable role in language comprehension (the brain as a prediction machine, [Ber10]). In this article, we present an extension of formal semantic theory that allows to make predictions of upcoming words. The basic intuition is: predictions are based on incomplete information. Drawing (defeasible) conclusions based on such information can be modeled by default reasoning. Since predictions can go wrong, a second strategy for retracting wrong guesses is needed in order to integrate (unexpected) words into the prior context. This is modeled by belief revision. We model both processing stages, making predictions about upcoming words and integrating them into the prior context, and relate the models to the empirical findings in neurophysiological research.
We thank our anonymous reviewers who helped to improve this paper by providing an elaborated and constructive feedback to former versions. The research was supported by the German Science Foundation (DFG) funding the Collaborative Research Center 991.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
‘Prediction’ must not be understood as a conscious or strategic process. Rather, prediction is understood as the unconscious activation of semantic properties of upcoming words prior to their occurrence, [FK99, 487].
- 2.
Cloze probability: participants in an offline norming task are presented sentence frames like that in (1) and are asked to fill in the dots with the first word that comes to their mind. The proportion, ranging from 0 to 1, of respondents supplying a particular word is defined as the cloze probability of this word in that context.
- 3.
Note that the pre-activated features used to predict upcoming words cannot simply be part of information about arguments, say, of verbs or common nouns. For example, ‘board’ in isolation does not prime (semantic features of) ‘ferry’ as opposed to (semantic features of) other semantically possible arguments like ‘gondola’ or ‘airplane’.
- 4.
An event-related potential (ERP) is the measured brain response that is the direct result of a specific sensory, cognitive, or motor event. An ERP component is a portion of an ERP waveform that has a characteristic shape, timing and amplitude distribution across the scalp and a well-characterized pattern of sensitivity to experimental manipulations or neural source, [KF11, LPP08]. It is important to note that the common statement that a word does not elicit an ERP component (which will be used in this paper as well) is a simplification. It is meant that it does not trigger a brain response that significantly differs from the baseline response triggered by some control word.
- 5.
Cloze probabilities: ‘play’: 91%; ‘move’: 31% and ‘look’: 3% in both contexts.
- 6.
Section 4 discusses alternative interpretations of the results..
- 7.
The ordering \(\ge _N\) depends both on the kind of context and the comprehender. The dependency on the context corresponds to the distinction between strongly constraining and weakly constraining contexts. In a strongly constraining context there are more expectations than in a weakly constraining context. The dependency on a comprehender is illustrated by the following example concerning the moral value system of a comprehender. [BHN+09] presented examples like ‘I think euthanasia is an acceptable course of action’ to members of a relatively strict Dutch Christian party and to non-Christian respondents with sufficiently contrasting moral value systems. The result was that for both groups there was an enhanced N400 though it was larger for members of the strict Dutch Christian party.
- 8.
Note that [Pet07] allows unrooted frames as well, but such frames are of no interest for our purpose.
- 9.
The reason for distinguishing \(\varDelta _D\) and \(\varDelta _E\) will become clear if a ranking on the set \(\varDelta _D\) of default rules using System Z is defined. See below for details.
- 10.
According to rule \(r_0\), an expectation w.r.t. to the theme argument of ‘plant’ does not include sortal information. Thus, there is no bias towards any tropical tree in the context of \(A_0\). For example, both ‘palm’ and ‘eucalyptus’ are equally expected. However, if ‘palm’ is the best completion one may argue that this information is already activated prior to the encounter of the argument. Thus, rule \(r_0\) seems to apply to weakly constraining and not to strongly constraining contexts. However, if sortal information is part of the consequent of the default rule, alternatives (‘eucalyptus’) to the best completion (‘palm’) are excluded. E.g., rule \(r_0\) becomes \(r_{00}\).
(i) \(\quad r_{00} \quad A_0 \Rightarrow B_0 \, \,\wedge \) resort:driveway:adornment:sort=palm.
Using \(r_{00}\), \(r_1\) becomes redundant because upon encountering ‘palm’ no new information needs to be added. Rule \(r_1\) is replaced by the following rule for the sort ‘eucalyptus’.
\(\begin{array}{ll} \mathrm{(ii)} \quad r_1: &{} A_0 \wedge { resort{:}driveway{:}adornment{:}sort{=}eucalyptus}\Rightarrow \\ &{} B_0 \wedge { resort{:}driveway{:}adornment{:}sort{=}eucalyptus}.\\ \end{array}\)
An open empirical question is the relation between N400 effects both in strongly constraining and weakly constraining contexts for ‘palm’ and ‘eucalyptus’, i.e. two concepts that are of the same type, here ‘plant’, but also of the same category. here ‘tree’, and that both fulfill the conditions specified in the consequent of rule \(r_0\).
- 11.
One may argue that rules \(r_1-r_3\) are strict and not defeasible. For example, a palm is a tree and not a flower. However, in the present context we are interested in the way a comprehender uses information, both top-down and bottom-up, to build a semantic representation of a constituent. What matters, therefore, is the relation between the various rules he uses (the priority ordering) and not the status of an individual rule as defeasible or strict. For example, rule \(r_2\) has a higher priority than rules \(r_0\) and \(r_1\) because it describes a situation which is assumed to be less normal. In addition, not all conjuncts in the consequent of a rule are non-defeasible, given the antecedent. For example, the tropics are only normally the habitat of palms, but they grow in moderate habitats as well (e.g., in botanical gardens in Europe).
- 12.
These restrictions are due to the fact that we do not have any information about the way, say, orchids (tropical flowers) or palms whose habitat are not the tropics are semantically processed online. Additional experimental data is needed to tackle this question.
- 13.
This section owes much to the review article [BFH12].
References
Van Berkum, J.J.A.: The brain is a prediction machine that cares about good and bad - any implications for neuropragmatics? Ital. J. Linguist. 22, 181–208 (2010)
Brouwer, H., Fitz, H., Hoeks, J.: Getting real about semantic illusions: rethinking the functional role of the P600 in language comprehension. Brain Res. 1446, 127–143 (2012)
Baggio, G., Hagoort, P.: The balance between memory and unification in semantics: a dynamic account of the N400. Lang. Cogn. Process. 26(9), 1338–1367 (2011)
Van Berkum, J.J.A., Holleman, B., Nieuwland, M., Otten, M., Murre, J.: Right or wrong? the brain’s fast response to morally objectionable statements. Psychol. Sci. 20, 1092–1099 (2009)
Boutilier, C.: Conditional logics of normality: a modal approach. Artif. Intell. 68(1), 87–154 (1994)
Burkhardt, P.: Inferential bridging relations reveal distinct neural mechanisms: evidence from event-related brain potentials. Brain Lang. 98(2), 159–168 (2006)
DeLong, K.A., Quante, L., Kutas, M.: Predictability, plausibility, and two late ERP positivities during written sentence comprehension. Neuropsychologia 61, 150–162 (2014)
DeLong, K.A., Urbach, T., Kutas, M.: Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nat. Neurosci. 8, 1117–1121 (2005)
Federmeier, K.D., Kutas, M.: A rose by any other name: long-term memory structure and sentence processing. J. Mem. Lang. 41, 469 (1999)
Federmeier, K.D., Wlotko, E.W., De Ochoa-Dewald, E., Kutas, M.: Multiple effects of sentential constraint on word processing. Brain Res. 1146, 75–84 (2007)
Goldszmidt, M., Pearl, J.: System-Z+: a formalism for reasoning with variable-strength defaults. In: Dean, T.L., McKeown, K. (eds.) Proceedings of the 9th National Conference on Artificial Intelligence, Anaheim, CA, USA, 14–19 July, Vol. 1, pp. 399–404. AAAI Press/The MIT Press (1991)
Goldszmidt, M., Pearl, J.: Rank-based systems: a simple approach to belief revision, belief update, and reasoning about evidence and actions. In: Nebel, B., Rich, C., Swartout, W.R. (eds.) Proceedings of the 3rd International Conference on Principles of Knowledge Representation and Reasoning (KR 1992), pp. 661–672. Morgan Kaufmann (1992)
Gouvea, A.C., Phillips, C., Kazanina, N., Poeppel, D.: The linguistic processes underlying the P600. Lang. Cogn. Process. 25(2), 149–188 (2010)
Kutas, M., Federmeier, K.: Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annu. Rev. Psychol. 62(1), 621–647 (2011)
Kaan, E., Harris, A., Gibson, E., Holcomb, P.: The P600 as an index of syntactic integration difficulty. Lang. Cogn. Process. 15(2), 159–201 (2000)
Lau, E.F., Phillips, C., Poeppel, D.: A cortical network for semantics: (de)constructing the N400. Nat. Rev. Neurosci. 9, 920–933 (2008)
Pearl, J., System, Z.: A natural ordering of defaults with tractable applications to nonmonotonic reasoning. In: Parikh, R. (ed.) Proceedings of the 3rd Conference on Theoretical Aspects of Reasoning about Knowledge, Pacific Grove, CA, March 1990, pp. 121–135. Morgan Kaufmann (1990)
Petersen, W.: Representation of concepts as frames. In: Skilters, J., Toccafondi, F., Stemberger, G. (eds.) Complex Cognition and Qualitative Science, The Baltic International Yearbook of Cognition, Logic and Communication, vol. 2, pp. 151–170. University of Latvia (2007)
Petersen, W., Osswald, T.: Concept composition in frames: focusing on genitive constructions. In: Gamerschlag, T., Gerland, D., Osswald, R., Petersen, W. (eds.) Frames and Concept Types. Studies in Linguistics and Philosophy, vol. 94, pp. 243–266. Springer, Cham (2014)
Pylkkänen, L.: N400. http://www.psych.nyu.edu/pylkkanen/Neural_Bases/07_slides/18_N400.pdf. Accessed 10 Sept 2015
Regel, S., Gunter, T.C., Friederici, A.D.: Isn’t it ironic? an electrophysiological exploration of figurative language processing. J. Cogn. Neurosci. 23(2), 277–293 (2011)
Schumacher, P.B.: Content and context in incremental processing: “the ham sandwich" revisited. Philos. Stud. 168(1), 151–165 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer-Verlag GmbH Germany
About this paper
Cite this paper
Naumann, R., Petersen, W. (2017). Semantic Predictions in Natural Language Processing, Default Reasoning and Belief Revision. In: Hansen, H., Murray, S., Sadrzadeh, M., Zeevat, H. (eds) Logic, Language, and Computation. TbiLLC 2015. Lecture Notes in Computer Science(), vol 10148. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-54332-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-662-54332-0_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-54331-3
Online ISBN: 978-3-662-54332-0
eBook Packages: Computer ScienceComputer Science (R0)