Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning
A novel neural network model is presented that learns by trial-and-error to reproduce complex sensory-motor sequences. One subnetwork, corresponding to the prefrontal cortex (PFC), is responsible for generating unique patterns of activity that represent the continuous state of sequence execution. A second subnetwork, corresponding to the striatum, associates these state-encoding patterns with the correct response at each point in the sequence execution. From a neuroscience perspective, the model is based on the known cortical and subcortical anatomy of the primate oculomotor system. From a theoretical perspective, the architecture is similar to that of a finite automaton in which outputs and state transitions are generated as a function of inputs and the current state. Simulation results for complex sequence reproduction and sequence discrimination are presented.
Unable to display preview. Download preview PDF.
- Barto AG (1990) Connectionist learning for control: an overview. In: Miller WT, Sutton RS, Werbos PJ (eds) Neural Networks for Control. MIT Press, Cambridge MA, pp 1–58Google Scholar
- Dominey PF (1993) Models of spatial accuracy and conditional behavior in oculomotor control. PhD Thesis, Dept of Computer Science, University of Southern CaliforniaGoogle Scholar
- Dominey PF, Arbib MA, Joseph JP (1995) A model of corticostriatal plasticity for learning oculomotor associations and sequences. J Cog Neuroscience 7:311–336Google Scholar
- Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. In: Vernon B Mountcastle (eds) Handbook of physiology. The Nervous system. Vol. V, Chap 9. 9: 373–417Google Scholar
- Keele SW, Jennings P, Jones S, Caulton D, Cohen A (1995) On the modularity of sequence representation. J Motor Behavior 27:17–30Google Scholar
- Kühn R, van Hemmen JL (1992) Temporal association. In: Domanay E, Hemmen van JL, Schulten K (eds) Physics of neural networks. Springer, Berlin Heidelberg New York, pp 213–280Google Scholar
- Rolls ET, Williams GV (1987) Sensory and movement-related neuronal activity in different regions of the primate striatum. In: Schneider JS, Lidsky TI (eds) Basal ganglia and motor behavior. Hans Huner, Toronto, pp 37–59Google Scholar
- Schlag J, Schlag-Rey M (1984) Visuomotor functions of central thalamus in monkey. II. Unit activity related to visual events, targeting, and fixation. J Neurophysiol 51:1175–1195Google Scholar
- Stadler MA (1995) The role of attention in implicit learning. J Exp Psychology: Learning, Mem, Cognition (in press)Google Scholar
- Weitzenfield A (1991) NSL neural simulation language version 2. 1. (Technical Report 91-05). Center for Neural Engineering, University of Southern CaliforniaGoogle Scholar