Abstract
The properties of time series, generated by continuous valued feed-forward networks in which the next input vector is determined from past output values, are studied. Asymptotic solutions developed suggest that the typical stable behavior is (quasi) periodic with attractor dimension that is limited by the number of hidden units, independent of the details of the weights. The results are robust under additive noise, except for expected noise-induced effects – attractor broadening and loss of phase coherence at large times. These effects, however, are moderated by the size of the network N.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H.D.I. Abarbanel, R. Brown, J.J. Sidorowich and L.S. Tsimring, The analysis of observed chaotic data in physical systems, Reviews of Modern Physics 65 (1993) 1331.
M. Abramowitz and C.A. Stegun (eds.), Handbook of Mathematical Functions (Dover, 1972).
R.L. Devaney, An Introduction to Chaotic Dynamical Systems (Addison-Wesley, 1985).
L. Ein-Dor and I. Kanter, Time series generation by multilayer networks, Physical Review E 57 (1998) 6564.
E. Eisenstein, I. Kanter, D.A. Kessler and W. Kinzel, Generation and prediction of time series by a neural network, Physical Review Letters 74 (1995) 6.
J.A. Hertz, A. Krogh and R.G. Palmer, Introduction to the Theory of Neural Computation (Addison-Wesley, 1991).
I. Kanter, D.A. Kessler, A. Priel and E. Eisenstein, Analytical study of time series generation by feed-forward networks, Physical Review Letters 75 (1995) 2614.
R. Metzler, W. Kinzel and I. Kanter, Interacting neural networks, Physical Review E 62 (2000) 2555.
C. Moore, Generalized shifts: undecidabilty and unpredictability in dynamical systems, Nonlinearity 4 (1991) 199.
A. Priel, Dynamic and static properties of neural networks with feedback, Ph.D. Thesis, Bar-Ilan University, Israel (1999). See URL: http://www.biu.ac.il/faculty/priel
A. Priel and I. Kanter, Long-term properties of time series generated by a perceptron with various transfer functions, Physical Review E 59 (1999) 3368.
A. Priel and I. Kanter, Robust chaos generation by a perceptron, Europhysics Letters 51 (2000) 230.
A. Priel, I. Kanter and D.A. Kessler, Analytical study of the interplay between architecture and predictability, in: Proceedings of Neural Information Processing Systems 10, eds. M.I. Jordan, S.A. Solla and M. Kearns (MIT Press, 1998).
A. Priel, I. Kanter and D.A. Kessler, Noisy time series generation by feed-forward networks, Journal of Physics A 31 (1998) 1189.
T. Sauer, J.A. Yorke and M. Casdagli, Embedology, Journal of Statistical Physics 65 (1991) 579.
M. Schroder and W. Kinzel, Limit cycles of a perceptron, Journal of Physics A 31 (1998) 2967.
H.T. Siegelmann, B.G. Horne and C.L. Giles, Computational capabilities of recurrent NARX neural networks, IEEE Transactions on Systems, Man and Cybernetics 27 (1997) 208.
H. Tong, Non-Linear Time Series (Oxford University Press, 1990).
A. Waibel, Modular construction of time-delay neural networks for speech recognition, Neural Computation 1 (1989) 39.
T.L.H. Watkin, A. Rau and M. Biehl, The statistical mechanics of learning a rule, Reviews of Modern Physics 65 (1993) 499.
A.S. Weigand and N.A. Gershenfeld (eds.), Time Series Prediction, Santa Fe Proceedings (Addison-Wesley, 1994).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Priel, A., Kanter, I. Time Series Generation by Recurrent Neural Networks. Annals of Mathematics and Artificial Intelligence 39, 315–332 (2003). https://doi.org/10.1023/A:1024620813258
Issue Date:
DOI: https://doi.org/10.1023/A:1024620813258