Dynamics of Neural Networks - Some Qualitative Properties
All neural networks, both natural and artificial, are characterized by two kinds of dynamics. The first one is concerned with what we would call “learning dynamics”, in fact the sequential (discrete time) dynamics of the choice of synaptic weights. The second one is the intrinsic dynamics of the neural network viewed as a dynamical system after the weights have been established via learning. The paper deals with the second kind of dynamics. Since the emergent computational capabilities of a recurrent neural network can be achieved provided it has suitable dynamical properties when viewed as a system with several equilibria, the paper deals with those qualitative properties connected to the achievement of such dynamical properties, more precisely the gradient like behavior. In the case of the neural networks with delays, these aspects are reformulated in accordance with the state of the art of the theory of delay dynamical systems.
KeywordsNeural Network Recurrent Neural Network Qualitative Property Synaptic Weight Invariance Principle
Unable to display preview. Download preview PDF.
- 2.Danciu, D.: Qualitative behavior of the time delay Hopfield type neural networks with time varying stimulus. Annals of The University of Craiova, Series: Electrical Engineering (Automatics, Computers, Electronics) 26, 72–82 (2002)Google Scholar
- 3.Danciu, D., Răsvan, V.: On Popov-type Stability Criteria for Neural Networks. Electronic Journal on Qualitative Theory of Differential Equations (Proc. 6th Coll. Qualitative Theory of Differential Equations QTDE) 23 (2000), http://www.math.u-szeged.hu/ejqtde/6/623.pdf
- 4.Danciu, D., Răsvan, V.: Gradient-like behaviour for Hopfield-type neural networks with delay. In: Proc. of The 3rd Int. Workshop on Intelligent Control Systems ICS’2001, pp. 20–24. Printech, Bucharest (2001)Google Scholar
- 8.Gelig, A.K.: Dynamics of pulse systems and neural networks (in Russian). Leningrad Univ. Publishing House, Leningrad (1982)Google Scholar
- 9.Gelig, A.K., Leonov, G.A., Yakubovich, V.A.: Stability of the systems with non-unique equilibrium points (in Russian). Nauka, Moscow (1978)Google Scholar
- 11.Halanay, A.: Invariant manifolds for systems with time lag. In: Hale, J.K., La Salle, J.P. (eds.) Differential and dynamical systems, pp. 199–213. Acad. Press, New York (1967)Google Scholar
- 12.Halanay, A., Răsvan, V.: Applications of Liapunov Methods to Stability. Kluwer Academic Publishers, Dordrecht (1993)Google Scholar
- 15.Kitamura, S., Hirai, K., Nishimura, M.: Stability of a Control System with Several Nonlinear Elements and Time Lags. Technology Repts. Osaka Univ. 17, 93–102 (1967)Google Scholar
- 19.Nishimura, M., Kitamura, S.: A Lyapunov Functional for Systems with Multiple Non-linearities and Time Lags. Technology Repts. Osaka Univ. 19, 83–88 (1969)Google Scholar
- 21.Osareh, A.: Automated Identification of Diabetic Retinal Exudates and the Optic Disc. PhD Thesis, Bristol University, Faculty of Engineering, Department of Computer Science (2004)Google Scholar
- 24.Răsvan, V., Danciu, D.: Neural networks - global behavior versus delay. Periodica Politechnica. Trans. Autom. Control and Comp. Sci. 49(63), 11–14 (2004)Google Scholar