On learning and recognition of temporal sequences of correlated patterns

Abstract

Starting from prescriptions for the learning and retrieval of correlated time-independent patterns in spin glass models we present a class of learning rules for the implementation of temporal sequences of patterns into spin models with randomly delayed interactions. From biological motivation a learning process is devised in such a way that, while learning, the spin-dynamics and the dynamics in the interactions are coupled and take place simultaneously. After training, such a system allows good retrieval of strongly noisy versions of complex sequences of highly correlated patterns such as words considered to be temporal sequences of letters, without making use of multispin coupling. However, the storage of given sequences as stable limit cycles of the network remains in many cases somewhat imperfect during the considered learning time. The small number of errors which are left, lead to a gradually decreasing recognition quality unless the noisy sequence to be recognized is continuously presented to the system with small amplitude via additional receptors. By the use of local receptor fields one may also recognize unlearnt combinations of fragments of the learnt sequences. Perfect learning and retrieval of such sequences is guaranteed under certain conditions by a coupling matrix analogous to the pseudoinverse solution [11] for static patterns. Two different types of networks are investigated in computer simulations of finite systems: (I) a one-layered fully connected network and (II) an extension of a so-called index model proposed by Schmitz et al. [26] for time-independent patterns, which consists of two layers of neurons with connections only inbetween the layers. In the second model the transitions between two patterns of a sequence are perfectly sharp, one reason why this model may be more interesting for applications.