Recursive hetero-associative memories for translation
This paper presents a modification of Pollack's RAAM (Recursive Auto-Associative Memory), called a Recursive Hetero-Associative Memory (RHAM), and shows that it is capable of learning simple translation tasks, by building a state-Space representation of each input string and unfolding it to obtain the corresponding output string. RHAM-based translators are computationally more powerful and easier to train than their corresponding double-RAAM counterparts in the literature.
KeywordsHide Neuron Recurrent Neural Network Hide Unit Input String Empty String
Unable to display preview. Download preview PDF.
- Chalmers, D.J. (1990) “Syntactic Transformations on Distributed Representations”, Connection Science 2, 53–62.Google Scholar
- Chrisman, L. (1991) “Learning Recursive Distributed Representations for Holistic Computation”, Connection Science 3:4, 345–366.Google Scholar
- Hopcroft, J.E., Ullman, J.D. (1979) Introduction to automata theory, languages and computation. Reading, Massachussets: Addison-Wesley.Google Scholar
- Neco, R., Forcada, M.L. (1996) “Beyond Mealy machines: Learning translators with recurrent neural networks”, Proc. World Congress on Neural Networks (San Diego, Calif., Sept. 1996), p. 408–411.Google Scholar
- Pollack, J.B. (1990) “Recursive distributed representations”, Artificial Intelligence 46, 77–105.Google Scholar
- Rumelhart, D.E., Hinton, G.E., Williams, R.J. (1986) “Learning internal representations by error propagation”. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition (D.E. Rumelhart and J.L. McClelland, eds.), Vol. 1, Chapter 8, Cambridge, MA: MIT Press.Google Scholar
- Salomaa, A. (1987) Formal Languages, Boston, Massachusetts: Academic Press.Google Scholar
- Unnikrishnan, K.P., Venugopal, K.P. (1994) “Alopex: A Correlation-Based Algorithm for Feedforward and Recurrent Neural Networks”, Neural Computation 6, 469–490.Google Scholar
- Williams, R.J., Zipser, D. (1989) “A learning algorithm for continually running fully recurrent neural networks”, Neural Comp. 1, 270–280.Google Scholar