Strong Systematicity in Sentence Processing by an Echo State Network
For neural networks to be considered as realistic models of human linguistic behavior, they must be able to display the level of systematicity that is present in language. This paper investigates the systematic capacities of a sentence-processing Echo State Network. The network is trained on sentences in which particular nouns occur only as subjects and others only as objects. It is then tested on novel sentences in which these roles are reversed. Results show that the network displays so-called strong systematicity.
KeywordsNoun Phrase Relative Clause Sentence Processing Test Sentence Main Clause
Unable to display preview. Download preview PDF.
- 6.Aizawa, K.: The systematicity arguments. Kluwer Academic Publishers, Dordrecht (2003)Google Scholar
- 11.Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in neural information processing systems, vol. 15, pp. 593–600. MIT Press, Cambridge (2003)Google Scholar
- 13.Frank, S.L.: Learn more by training less: systematicity in sentence processing by recurrent networks. Connection Science (in press)Google Scholar
- 16.Hadley, R.F.: On the proper treatment of semantic systematicity 14, 145–172 (2004)Google Scholar
- 17.Frank, S.L., Haselager, W.F.G.: Robust semantic systematicity and distributed representations in a connectionist model of sentence comprehension. In: Miyake, N., Sun, R. (eds.) Proceedings of the 28th Annual Conference of the Cognitive Science Society, Lawrence Erlbaum, Mahwah (in press)Google Scholar