Strong Systematicity in Sentence Processing by an Echo State Network

  • Stefan L. Frank
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)


For neural networks to be considered as realistic models of human linguistic behavior, they must be able to display the level of systematicity that is present in language. This paper investigates the systematic capacities of a sentence-processing Echo State Network. The network is trained on sentences in which particular nouns occur only as subjects and others only as objects. It is then tested on novel sentences in which these roles are reversed. Results show that the network displays so-called strong systematicity.


Noun Phrase Relative Clause Sentence Processing Test Sentence Main Clause 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988)CrossRefGoogle Scholar
  2. 2.
    Chalmers, D.J.: Connectionism and compositionality: why Fodor and Pylyshyn were wrong. Philosophical Psychology 6(3), 305–319 (1993)CrossRefGoogle Scholar
  3. 3.
    Hadley, R.F.: Systematicity in connectionist language learning. Mind & Language 9(3), 247–272 (1994)CrossRefGoogle Scholar
  4. 4.
    Niklasson, L.F., Van Gelder, T.: On being systematically connectionist. Mind & Language 9, 288–302 (1994)CrossRefGoogle Scholar
  5. 5.
    Fodor, J.A., McLaughlin, B.: Connectionism and the problem of systematicity: Why Smolensky’s solution does not work. Cognition 35, 183–204 (1990)CrossRefGoogle Scholar
  6. 6.
    Aizawa, K.: The systematicity arguments. Kluwer Academic Publishers, Dordrecht (2003)Google Scholar
  7. 7.
    Christiansen, M.H., Chater, N.: Generalization and connectionist language learning. Mind & Language 9(3), 273–287 (1994)CrossRefGoogle Scholar
  8. 8.
    Hadley, R.F., Rotaru-Varga, A., Arnold, D.V., Cardei, V.C.: Syntactic systematicity arising from semantic predictions in a Hebbian-competetive network. Connection Science 13(1), 73–94 (2001)MATHCrossRefGoogle Scholar
  9. 9.
    Bodén, M.: Generalization by symbolic abstraction in cascaded recurrent networks. Neurocomputing 57, 87–104 (2004)CrossRefGoogle Scholar
  10. 10.
    Elman, J.L.: Finding structure in time. Cognitive Science 14, 179–211 (1990)CrossRefGoogle Scholar
  11. 11.
    Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in neural information processing systems, vol. 15, pp. 593–600. MIT Press, Cambridge (2003)Google Scholar
  12. 12.
    Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)CrossRefGoogle Scholar
  13. 13.
    Frank, S.L.: Learn more by training less: systematicity in sentence processing by recurrent networks. Connection Science (in press)Google Scholar
  14. 14.
    Van der Velde, F., Van der Voort van der Kleij, G.T., De Kamps, M.: Lack of combinatorial productivity in language processing with simple recurrent networks. Connection Science 16(1), 21–46 (2004)CrossRefGoogle Scholar
  15. 15.
    Hadley, R.F.: Systematicity revisited: reply to Christiansen and Chater and Niklasson and van Gelder. Mind & Language 9(4), 431–444 (1994)CrossRefGoogle Scholar
  16. 16.
    Hadley, R.F.: On the proper treatment of semantic systematicity 14, 145–172 (2004)Google Scholar
  17. 17.
    Frank, S.L., Haselager, W.F.G.: Robust semantic systematicity and distributed representations in a connectionist model of sentence comprehension. In: Miyake, N., Sun, R. (eds.) Proceedings of the 28th Annual Conference of the Cognitive Science Society, Lawrence Erlbaum, Mahwah (in press)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Stefan L. Frank
    • 1
  1. 1.Nijmegen Institute for Cognition and InformationRadboud University NijmegenNijmegenThe Netherlands

Personalised recommendations