Advertisement

Biological Cybernetics

, Volume 71, Issue 6, pp 469–480 | Cite as

STORE working memory networks for storage and recall of arbitrary temporal sequences

  • Gary Bradski
  • Gail A. Carpenter
  • Stephen Grossberg
Article

Abstract

Neural network models of working memory, called “sustained temporal order recurrent” (STORE) models, are described. They encode the invariant temporal order of sequential events in short-term memory (STM) in a way that mimics cognitive data about working memory, including primacy, recency, and bowed order and error gradients. As new items are presented, the pattern of previously stored items remains invariant in the sense that relative activations remain constant through time. This invariant temporal order code enables all possible groupings of sequential events to be stably learned and remembered in real time, even as new events perturb the system. Such competence is needed to design self-organizing temporal recognition and planning systems in which any subsequence of events may need to be categorized in order to control and predict future behavior or external events. STORE models show how arbitrary event sequences may be invariantly stored, including repeated events. A preprocessor interacts with the working memory to represent event repeats in spatially separate locations. It is shown why at least two processing levels are needed to invariantly store events presented with variable durations and interstimulus intervals. It is also shown how network parameters control the type and shape of primacy, recency, or bowed temporal order gradients that will be stored.

Keywords

Neural Network Model Temporal Order Sequential Event Temporal Sequence Interstimulus Interval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Atkinson RC, Shiffrin RM (1971) The control of short term memory. Sci Am, August:82–90Google Scholar
  2. Baddeley AD (1986) Working memory. Clarendon Press, OxfordGoogle Scholar
  3. Boardman I, Bullock D (1991) A neural network model of serial order recall from short-term memory. In: Proceedings of the international joint conference on neural networks, Seattle, II:879–884 IEEE Service Center, Piscataway, NJGoogle Scholar
  4. Boardman I, Cohen MA, Grossberg S (1993) Variable rate working memories for phonetic categorization and invariant speech perception. In: Proceedings of the international joint conference on neural networks, Portland, III:2–5. IEEE Service Center, Piscataway, NJGoogle Scholar
  5. Bradski G, Carpenter GA, Grossberg S (1991) Working memory networks for learning multiple groupings of temporally ordered events: applications to 3-D visual object recognition. In: Proceedings of the international joint conference on neural networks, Seattle, 1: 723–728. IEEE Service Center, Piscataway, NJGoogle Scholar
  6. Bradski G, Carpenter, GA, Grossberg, S (1992) Working memory networks for learning temporal order with application to 3-D visual object recognition. Neural Comp 4:270–286Google Scholar
  7. Carpenter GA, Grossberg S (eds) (1991) Pattern recognition by self-organizing neural networks. MIT Press, Cambridge, MAGoogle Scholar
  8. Carpenter GA, Grossberg S (1992) A self-organizing neural network for supervised learning, recognition, and prediction. IEEE Commun 30:38–49CrossRefGoogle Scholar
  9. Carpenter GA, Grossberg S, Reynolds JH (1991) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Networks 4:565–588CrossRefGoogle Scholar
  10. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Networks 3:698–713CrossRefGoogle Scholar
  11. Cohen MA, Grossberg S (1986) Neural dynamics of speech and language coding: developmental programs, perceptual grouping, and competition for short term memory. Hum Neurobiol 5:1–22PubMedGoogle Scholar
  12. Elman JL (1990) Finding structure in time. Cogn 14:179–211CrossRefGoogle Scholar
  13. Gaudiano P, Grossberg S (1991) Vector associative maps: unsupervised real-time error-based learning and control of movement trajectories. Neural Networks 4:147–183CrossRefGoogle Scholar
  14. Grossberg S (1970) Some networks that can learn, remember, and reproduce any number of complicated space-time patterns. II. Stud Appl Math 49:135–166Google Scholar
  15. Grossberg S (1973) Contour enhancement, short term memory, and constancies in reverberating neural networks. Stud Appl Math 52:217–257Google Scholar
  16. Grossberg S (1976) Adaptive pattern classification and universal recoding. I. Parallel development and coding of neural feature detectors. Biol Cybern 23:121–134CrossRefPubMedGoogle Scholar
  17. Grossberg S (1978a) A theory of human memory: self-organization and performance of sensory-motor codes, maps, and plans. In: Rosen R, Snell F (Eds) Progress in theoretical biology, Vol 5. Academic Press, New York, pp 233–374Google Scholar
  18. Grossberg S (1987b) Behavioral contrast in short term memory: serial binary memory models or parallel continuous memory models? J Math Psychol 17:199–219CrossRefGoogle Scholar
  19. Grossberg S (1982) Studies of mind and brain: neural principles of learning, perception, development, cognition, and motor control. Reidel Press, BostonGoogle Scholar
  20. Grossberg S (1986) The adaptive self-organization of serial order in behavior: speech, language, and motor control. Schwab EC, Nusbaum, HC (Eds) Pattern recognition by humans and machines, Vol 1. Speech perception. Academic Press, New York, pp 187–294. Reprinted in: Grossberg S (eds) The adaptive brain. II. Vision, Speech, language, and motor control. Elsevier/North-Holland, Amsterdam, pp 311–400Google Scholar
  21. Grossberg S, Kuperstein M (1989) Neural dynamics of adaptive sensory-motor control: expanded edition. Pergamon Press, Elmsford, NYGoogle Scholar
  22. Grossberg S, Pepe J (1971) Spiking threshold and overarousal effects in serial learning. J Stat Phys 3:95–125CrossRefGoogle Scholar
  23. Grossberg S, Stone GO (1986a) Neural dynamics of attention switching and temporal order information in short term memory. Memory Cogn 14:451–468Google Scholar
  24. Grossberg S, Stone GO (1986b) Neural dynamics of word recognition and recall: attentional priming, learning, and resonance. Psychol Rev 93:46–74CrossRefPubMedGoogle Scholar
  25. Gutfreund H, Mezard M (1988) Processing of temporal sequences in neural networks. Physiol Rev Lett 61:235–238CrossRefGoogle Scholar
  26. Guyon I, Personnaz L, Nadal JP, Dreyfus G (1988) Storage retrieval of complex sequences in neural networks. Physiol Rev A 38:6365–6372CrossRefGoogle Scholar
  27. Healy AF (1975) Separating item from order information in short-term memory. J Verbal Learn Verbal Behav 13:644–655CrossRefGoogle Scholar
  28. Jordan MI (1986) Serial order: a parallel distributed processing approach. (Institute for Cognitive Science, Report 8604) University of California, San DiegoGoogle Scholar
  29. Lee C, Estes WK (1981) Item and order information in short-term memory: evidence for multilevel perturbation process J Exp Psychol Hum Learn Memory 1:149–169Google Scholar
  30. Miller JL (1981) Effects of speaking rate on segmental distinctions In: Eimas PD, Miller JL (Eds) Perspectives on the study of speech. Lawrence Erlbaum, Hillsdale, NJGoogle Scholar
  31. Miller JL, Liberman AM (1979) Some effects of later-occurring information on the perception of stop consonant and semivowel. Percept Psychophys 25:457–465PubMedGoogle Scholar
  32. Ratcliff, R (1978) A theory of memory retrieval. Psychol Rev 85:59–108CrossRefGoogle Scholar
  33. Reeves A, Sperling G (1986) Attention gating in short-term visual memory. Psychol Rev 93:180–206CrossRefPubMedGoogle Scholar
  34. Repp BH (1980) A range-frequency effect on perception of silence in speech. Haskins Laboratories Status Report on Speech Research, SR-61:151–165Google Scholar
  35. Repp BH (1983) Bi directional contrast effects in the perception of VC-CV sequences. Percept Psychophys 33:147–155PubMedGoogle Scholar
  36. Repp B, Liberman A, Eccardt T, Pestsky D (1978) Perceptual integration of acoustic cues for stop fricative and affiricate manner. J Exp Psychol Hum Percept Perform 4:621–637CrossRefPubMedGoogle Scholar
  37. Schreter Z, Pfeifer R (1989) Short-term memory/long-term memory interactions in connectionist simulations of psychological experiments on list learning. In: Personnaz L, Dreyfus G (Eds) Neural networks from models to applications. IDSET, ParisGoogle Scholar
  38. Seibert MC (1991) Neural networks for machine vision: learning three-dimensional object recognition. PhD Dissertation, Boston UniversityGoogle Scholar
  39. Seibert MC, Waxman AM (1990a) Learning aspect graph representations from view sequences. In: Touretzky DS (Eds) Advances in neural information processing systems 2. Morgan Kaufmann, San Mateo, CA, pp 258–265Google Scholar
  40. Seibert MC, Waxman AM (1990b) Learning aspect graph representations of 3D objects in a neural network. In: Caudill M (eds) Proceedings of the international joint conference on neural networks, Washington, DC, 2. Lawrence Erlbaum, Hillsdale, NJ, 233–236Google Scholar
  41. Tartter VC, Kat D, Samuel AG, Repp BH (1983) Perception of intervocalic stop consonants: the contributions of closure duration and formant transitions. J Acoust Soc Am 73:715–725CrossRefGoogle Scholar
  42. Wang D, Arbib MA (1990) Complex temporal sequence learning based on short-term memory. Proc IEE 78:1536–1543CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 1994

Authors and Affiliations

  • Gary Bradski
    • 1
  • Gail A. Carpenter
    • 1
  • Stephen Grossberg
    • 1
  1. 1.Center for Adaptive Systems and Department of Cognitive and Neural SystemsBoston UniversityBostonUSA

Personalised recommendations