Advertisement

Holistic Symbol Processing and the Sequential RAAM: An Evaluation

  • James A. Hammerton
  • Barry L. Kalman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1778)

Abstract

In recent years connectionist researchers have demonstrated many examples of holistic symbol processing, where symbolic structures are operated upon as a whole by a neural network, by using a connectionist compositional representation of the structures. In this paper the ability of the Sequential RAAM (SRAAM) to generate representations that support holistic symbol processing is evaluated by attempting to perform Chalmers’ syntactic transformations using it. It is found that the SRAAM requires a much larger hidden layer for this task than the RAAM and that it tends to cluster its hidden layer states close together, leading to a representation that is fragile to noise. The lessons for connectionism and holistic symbol processing are discussed and possible methods for improving the SRAAM’s performance suggested.

Keywords

Hide Layer Unit Representation Hide Unit Active Sentence Connectionist Representation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Balogh, I.L.: An analysis of a connectionist internal representation: Do RAAM networks produce truly distributed representations? PhD thesis, New Mexico State University (1994)Google Scholar
  2. 2.
    Blank, D.S., Meeden, L.A., Marshall, J.B.: Exploring the symbolic/subsymbolic continuum: A case study of RAAM. In: Dinsmore, J. (ed.) Symbolic and Connectionist Paradigms: Closing the Gap, pp. 113–148. Lawrence Erlbaum Associates, Hillsdale (1992)Google Scholar
  3. 3.
    Callan, R.E., Palmer-Brown, D.: (S)RAAM: An analytical technique for fast and reliable derivation of connectionist symbol structure representations. Connection Science 9(2), 139–159 (1997)Google Scholar
  4. 4.
    Chalmers, D.J.: Syntactic transformations on distributed representations. Connection Science 2(1–2), 53–62 (1990); Reprinted in [16], pp. 46–55Google Scholar
  5. 5.
    Chrisman, L.: Learning recursive distributed representations for holistic computation. Connection Science 3(4), 345–366 (1991)CrossRefGoogle Scholar
  6. 6.
    French, R.M.: Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference. In: Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, Atlanta, GA, pp. 335–340. Lawrence Erlbaum Associates, Hillsdale (1994)Google Scholar
  7. 7.
    Hammerton, J.A.: Exploiting Holistic Computation: An evaluation of the Sequential RAAM. PhD thesis, School of Computer Science, The University of Birmingham, UK (1998)Google Scholar
  8. 8.
    Hammerton, J.A.: Holistic computation: Reconstructing a muddled concept. Connection Science 10(1), 3–19 (1998)CrossRefGoogle Scholar
  9. 9.
    Ho, E.K.S., Chan, L.W.: Confluent preorder parser as a finite state automata. In: Proceedings of International Conference on Artificial Neural Networks ICANN 1996, Bochum, Germany, pp. 899–904. Springer, Berlin (1996)Google Scholar
  10. 10.
    Kalman, B.L., Kwasny, S.C.: High performance training of feedforward and simple recurrent networks. Neurocomputing 14, 63–83 (1997)CrossRefGoogle Scholar
  11. 11.
    Kohonen, T.: The self-organising map. Proceedings of the IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  12. 12.
    Kwasny, S.C., Kalman, B.L.: Tail-recursive distributed representations and simple recurrent networks. Connection Science 7(1), 61–80 (1995)CrossRefGoogle Scholar
  13. 13.
    Niklasson, L., Sharkey, N.E.: Systematicity and generalization in compositional connectionist representations. In: Dorffner, G. (ed.) Neural Networks and a New Artificial Intelligence, pp. 217–232. Thomson Computer Press, London (1997)Google Scholar
  14. 14.
    Plate, T.A.: Distributed Representations and Nested Compositional Structure. PhD thesis, University of Toronto (1994)Google Scholar
  15. 15.
    Pollack, J.B.: Recursive distributed representations. Artificial Intelligence 46(1-2), 77–105 (1990)CrossRefGoogle Scholar
  16. 16.
    Sharkey, N.E. (ed.): Connectionist Natural Language Processing: Readings from Connection Science. Intellect, Oxford (1992)Google Scholar
  17. 17.
    Sperduti, A.: Labeling RAAM. Technical Report TR-93-029, International Computer Science Institute, Berkeley, California (1993)Google Scholar
  18. 18.
    Sperduti, A.: On Some Stability Properties of the LRAAM Model. Technical Report TR-93-031, International Computer Science Institute, Berkeley, California (1993)Google Scholar
  19. 19.
    Weber, V.: Connectionist unification with a distributed representation. In: Proceedings of the International Joint Conference on Neural Networks– IJCNN 1992, Beijing, China, pp. 555–560. IEEE, Piscataway (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • James A. Hammerton
    • 1
  • Barry L. Kalman
    • 2
  1. 1.School of Computer ScienceThe University of BirminghamUK
  2. 2.Department of Computer ScienceWashington UniversitySt Louis

Personalised recommendations