Abstract
We propose a model of natural language inference which identifies valid inferences by their lexical and syntactic features, without full semantic interpretation. We extend past work in natural logic, which has focused on semantic containment and monotonicity, by incorporating both semantic exclusion and implicativity. Our model decomposes an inference problem into a sequence of atomic edits linking premise to hypothesis; predicts a lexical entailment relation for each edit; propagates these relations upward through a semantic composition tree according to properties of intermediate nodes; and joins the resulting entailment relations across the edit sequence. A computational implementation of the model achieves 70 % accuracy and 89 % precision on the FraCaS test suite. Moreover, including this model as a component in an existing system yields significant performance gains on the Recognizing Textual Entailment challenge.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
We use \(\overline {x}\) to denote the complement of set x in universe U; thus \(x \cap \overline {x} = \emptyset\) and \(x \cup \overline {x} = U\).
- 4.
Our model can easily be revised to accommodate vacuous expressions and relations between them, but then becomes somewhat unwieldy. The assumption of non-vacuity is closely related to the assumption of existential import in traditional logic. For a defense of existential import in natural language semantics, see (Böttner 1988).
- 5.
We describe relations R and S as duals under negation iff \(\forall x, y: { \langle x, y \rangle } \in R \Leftrightarrow { \langle \overline{x}, \overline{y} \rangle } \in S\). Thus ⊏ and ⊐ are dual; | and \(\mathrel{\smallsmile}\) are dual; and ≡,
, and # are self-dual. The significance of this duality will become apparent in Sect. 5. - 6.
Two sets selected uniformly at random from 2U are overwhelmingly likely to belong to # (for large |U|).
- 7.
That is, all functional types whose final output is a truth value. If we assume a type system whose basic types are e (entities) and t (truth values), then this includes most of the functional types encountered in semantic analysis: e→t (common nouns, adjectives, and intransitive verbs), e→ e→t (transitive verbs), (e→t) → (e→t) (adverbs), (e→t) → (e→t) → t (binary generalized quantifiers), and so on.
- 8.
Assuming the expressions are non-vacuous, and belong to the same semantic type.
- 9.
In Tarskian relation algebra, this operation is known as relation composition, and is often represented by a semi-colon: R ; S. To avoid confusion with semantic composition (Sect. 5), we prefer to use the term join for this operation, by analogy to the database JOIN operation (also commonly represented by ⋈).
- 10.
We use this notation as shorthand for the union ≡∪⊏∪⊐∪ | ∪ #. To be precise, the result of this join is not identical with this union, but is a subset of it, since the union contains some pairs of sets (e.g. 〈U∖a,U∖a〉, for any |a|=1) which cannot participate in the | relation. However, the approximation makes little practical difference.
- 11.
Some union relations hold intrinsic interest. For example, in the three-way formulation of the NLI task described in Sect. 2, the three classes can be identified as ⋃{≡,⊏},
, and \(\bigcup\{\sqsupset, \mathrel{\smallsmile}, \#\}\). - 12.
That is, the relations in \(\mathfrak{B}\) plus 9 union relations. Note that this closure fails to include most of the 120 possible union relations. Perhaps surprisingly, the unions ⋃{≡,⊏} and
mentioned in footnote 11 do not appear. - 13.
In fact, computer experiments show that if relations are selected uniformly at random from \(\mathfrak{B}\), it requires on average just five joins to reach •.
- 14.
For compactness, we omit the union notation here; thus ⊏| # stands for ⋃{⊏,|,#}.
- 15.
Note that most antonym pairs do not belong to the
relation, since they typically do not exclude the middle. - 16.
Some of these assertions assume the non-vacuity (Sect. 2) of the predicates to which the quantifiers are applied.
- 17.
Indeed, the official definition of the RTE task explicitly specifies that tense be ignored.
- 18.
At least for practical purposes. The projection of
and | as | depends on the assumption of non-vacuity, and \(\mathrel{\smallsmile}\) is actually projected as ⋃{≡,⊏,⊐,|,#}, which we approximate by #, as described in Sect. 3. - 19.
Consider the verbal construct is married to: is married to a German |is married to a non-German, is married to a German |is married to an Italian, is married to a European #is married to a non-German. The AuContraire system (Ritter et al. 2008) includes an intriguing approach to identifying such functional phrases automatically.
- 20.
We use “factives” as an umbrella term embracing counterfactives and nonfactives along with factives proper.
- 21.
Of course, the implicatives may carry presuppositions as well ( he managed to escape → it was hard to escape), but these implications are not activated by a simple deletion, as with the factives.
- 22.
The order of edits can be significant, if one edit affects the projectivity properties of the context for another edit. In practice, we typically find that different edit orders lead to the same final result (albeit via different intermediate steps), or at worst to a result which is compatible with, though less informative than, the desired result. But in principle, edit sequences involving lexical items with unusual properties—not exhibited, so far as we are aware, by any natural language expressions—could lead to incompatible results. Thus we lack any formal guarantee of soundness.
- 23.
However, some inferences can be enabled by auxiliary premises encoded as lexical entailment relations. For example, men ⊏ mortal can enable the classic syllogism Socrates is a man ⊏ Socrates is mortal.
- 24.
We neglect edits involving auxiliaries and morphology, which simply yield the ≡ relation.
- 25.
Our evaluation excluded multi-premise problems, which constitute about 44 % of the test suite.
, and # are self-dual. The significance of this duality will become apparent in Sect. 
, and 
mentioned in footnote 
relation, since they typically do not exclude the middle.
and | as | depends on the assumption of non-vacuity, and References
Bos, J., & Markert, K. (2005). Recognising textual entailment with logical inference. In Proceedings of the conference on human language technology and empirical methods in natural language processing (pp. 628–635). Vancouver: Association for Computational Linguistics.
Böttner, M. (1988). A note on existential import. Studia Logica, 47(1), 35–40.
Chambers, N., Cer, D., Grenager, T., Hall, D., Kiddon, C., MacCartney, B., de Marneffe, M.-C., Ramage, D., Yeh, E., & Manning, C. D. (2007). Learning alignments and leveraging natural logic. In Proceedings of the ACL-07 workshop on textual entailment and paraphrasing (pp. 165–170). Prague: Association for Computational Linguistics.
Condoravdi, C., Crouch, D., de Paiva, V., Stolle, R., & Bobrow, D. (2003). Entailment, intensionality and text understanding. In Proceedings of the HLT-NAACL 2003 workshop on text meaning (pp. 38–45). Morristown: Association for Computational Linguistics.
Cooper, R., et al. (1996). Using the framework (Technical Report LRE 62-051 D-16). The FraCaS Consortium.
Dagan, I., Glickman, O., & Magnini, B. (2006). The PASCAL recognising textual entailment challenge. In J. Quiñonero-Candela, C. E. Rasmussen, F. Sinz, O. Bousquet, & B. Schölkopf (Eds.), Machine learning challenges. Evaluating predictive uncertainty, visual object classification, and recognising textual entailment (Vol. 3944, pp. 177–190). Berlin: Springer.
Fyodorov, Y., Winter, Y., & Francez, N. (2000). A natural logic inference system. In Proceedings of the 2nd international workshop on inference in computational semantics (ICoS-2), Germany: Dagstuhl.
Giampiccolo, D., Magnini, B., Dagan, I., & Dolan, B. (2007). The third PASCAL recognizing textual entailment challenge. In Proceedings of the ACL-07 workshop on textual entailment and paraphrasing (pp. 1–9). Prague: Association for Computational Linguistics
Glickman, O., Dagan, I., & Koppel, M. (2005). Web based probabilistic textual entailment. In Proceedings of the PASCAL challenges workshop on recognizing textual entailment. http://u.cs.biu.ac.il/~nlp/RTE1/Proceedings/glickman_et_al.pdf.
Hickl, A., Williams, J., Bensley, J., Roberts, K., Rink, B., & Shi, Y. (2006). Recognizing textual entailment with LCC’s GROUNDHOG system. In Proceedings of the second PASCAL challenges workshop on recognizing textual entailment, Venice, Italy, PASCAL (pp. 137–142).
Lakoff, G. (1970). Linguistics and natural logic. Synthese, 22, 151–271.
MacCartney, B. (2009). Natural language inference. Ph.D. thesis, Stanford University.
MacCartney, B., & Manning, C. D. (2008). Modeling semantic containment and exclusion in natural language inference. In Proceedings of the 22nd international conference on computational linguistics (COLING-08) (pp. 521–528). Manchester: Association for Computational Linguistics.
MacCartney, B., Grenager, T., de Marneffe, M.-C., Cer, D., & Manning, C. D. (2006). Learning to recognize features of valid textual entailments. In Proceedings of the human language technology conference of the North American chapter of the Association of Computational Linguistics (pp. 41–48). New York: Association for Computational Linguistics.
Nairn, R., Condoravdi, C., & Karttunen, L. (2006). Computing relative polarity for textual inference. In Proceedings of the fifth international workshop on inference in computational semantics (ICoS-5) (pp. 67–76).
MacCartney, B., Galley, M., & Manning, C. D. (2008). A phrase-based alignment model for natural language inference. In Proceedings of the conference on empirical methods in natural language processing (pp. 802–811). Honolulu: Association for Computational Linguistics.
Ritter, A., Downey, D., Soderland, S., & Etzioni, O. (2008). It’s a contradiction—no, it’s not: A case study using functional relations. In Proceedings of the conference on empirical methods in natural language processing (pp. 11–20). Honolulu: Association for Computational Linguistics.
Sánchez Valencia, V. (1991). Studies on natural logic and categorial grammar. Ph.D. thesis, Univ. Amsterdam.
Sukkarieh, J. (2001). Quasi-NL knowledge representation for structurally-based inferences. In Proceedings of the 3rd international workshop on inference in computational semantics (ICoS-3), Siena, Italy.
van Benthem, J. (1988). The semantics of variety in categorial grammars. In W. Buszkowski, W. Marciszewski, & J. van Benthem (Eds.), Categorial grammar (pp. 33–55). Amsterdam: Benjamins.
van Benthem, J. (1991). Studies in logic: Vol. 130. Language in action: Categories, lambdas and dynamic logic. Amsterdam: North-Holland.
van Benthem, J. (2008). A brief history of natural logic (Technical Report PP-2008-05). Institute for Logic, Language & Computation. http://www.illc.uva.nl/Publications/ResearchReports/PP-2008-05.text.pdf.
van der Sandt, R. A. (1992). Presupposition projection as anaphora resolution. Journal of Semantics, 9(4), 333–377.
van Eijck, J. (2005). Natural logic for natural language. http://homepages.cwi.nl/~jve/papers/05/nlnl/NLNL.pdf.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
MacCartney, B., Manning, C.D. (2014). Natural Logic and Natural Language Inference. In: Bunt, H., Bos, J., Pulman, S. (eds) Computing Meaning. Text, Speech and Language Technology, vol 47. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7284-7_8
Download citation
DOI: https://doi.org/10.1007/978-94-007-7284-7_8
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-7283-0
Online ISBN: 978-94-007-7284-7
eBook Packages: Computer ScienceComputer Science (R0)