Language Modelling with Dynamic Syntax
In this paper we introduce a system for the robust analysis of English using the apporach of Dynamic Syntax, in which the syntactic process is modelled as the word-by-word construction of a semantic representation. We argue that the inherent incrementality of the approach, in contrast with the essentially static assumptions of standard generative grammar, has clear advantages for the task of language modelling. To demonstrate its potential we show that this syntactic approach consistently outperforms a standard trigram model in word recovery tasks on parsable sentences. Furthermore, these results are achieved without recourse to hand-prepared training data.
KeywordsMutual Information Syntactic Structure Transition Rule British National Corpus Lexical Rule
Unable to display preview. Download preview PDF.
- 2.Kempson, R., Meyer-Viol, W., Gabbay, D.: Dynamic Syntax: the flow of language understanding. Blackwell, Oxford (2001)Google Scholar
- 4.Chomsky, N.: Syntactic structures. Mouton, The Hague (1957)Google Scholar
- 6.Jackendoff, R.: Semantic structures. MIT Press, Cambridge (1990)Google Scholar
- 7.Goldberg, A.E.: Constructions: a construction grammar approach to argument structure. University of Chicago Press, Chicago (1995)Google Scholar
- 8.Kilgarriff, A., Tugwell, D.: Sketching words. In: Corréard, M.-H. (ed.) Lexicography and Natural Language Processing: A Festschrift in Honour of B.T.S. Atkins, EURALEX, pp. 125–137Google Scholar
- 9.Chelba, C., Jelinek, F.: Exploiting syntactic structure for language modeling. In: Proceedings of 36th ACL and 17th COLING, pp. 225–231 (1998)Google Scholar
- 11.Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proceedings of 34th ACL, pp. 310–318 (1996)Google Scholar