Dynamics of Neural Networks - Some Qualitative Properties

  • Daniela Danciu
  • Vladimir Răsvan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)


All neural networks, both natural and artificial, are characterized by two kinds of dynamics. The first one is concerned with what we would call “learning dynamics”, in fact the sequential (discrete time) dynamics of the choice of synaptic weights. The second one is the intrinsic dynamics of the neural network viewed as a dynamical system after the weights have been established via learning. The paper deals with the second kind of dynamics. Since the emergent computational capabilities of a recurrent neural network can be achieved provided it has suitable dynamical properties when viewed as a system with several equilibria, the paper deals with those qualitative properties connected to the achievement of such dynamical properties, more precisely the gradient like behavior. In the case of the neural networks with delays, these aspects are reformulated in accordance with the state of the art of the theory of delay dynamical systems.


Neural Network Recurrent Neural Network Qualitative Property Synaptic Weight Invariance Principle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bélair, J., Campbell, S.A., van den Driessche, P.: Frustration, stability and delay induced oscillations in a neural network model. SIAM J. Appl. Math. 56, 254–265 (1996)CrossRefGoogle Scholar
  2. 2.
    Danciu, D.: Qualitative behavior of the time delay Hopfield type neural networks with time varying stimulus. Annals of The University of Craiova, Series: Electrical Engineering (Automatics, Computers, Electronics) 26, 72–82 (2002)Google Scholar
  3. 3.
    Danciu, D., Răsvan, V.: On Popov-type Stability Criteria for Neural Networks. Electronic Journal on Qualitative Theory of Differential Equations (Proc. 6th Coll. Qualitative Theory of Differential Equations QTDE) 23 (2000),
  4. 4.
    Danciu, D., Răsvan, V.: Gradient-like behaviour for Hopfield-type neural networks with delay. In: Proc. of The 3rd Int. Workshop on Intelligent Control Systems ICS’2001, pp. 20–24. Printech, Bucharest (2001)Google Scholar
  5. 5.
    Driessche, P.: Global attractivity in delayed Hopfield neural networks. SIAM J. Appl. Math. 58, 1878–1890 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Fink, W.: Neural attractor network for application in visual field data classification. Phys. Med. Biol. 49, 2799–2809 (2004)CrossRefGoogle Scholar
  7. 7.
    Fortuna, L., Balya, D., Zarandy, A.: Cellular Neural Networks. IEEE Circuits and Systems Magazine 4, 6–21 (2001)CrossRefGoogle Scholar
  8. 8.
    Gelig, A.K.: Dynamics of pulse systems and neural networks (in Russian). Leningrad Univ. Publishing House, Leningrad (1982)Google Scholar
  9. 9.
    Gelig, A.K., Leonov, G.A., Yakubovich, V.A.: Stability of the systems with non-unique equilibrium points (in Russian). Nauka, Moscow (1978)Google Scholar
  10. 10.
    Gopalsamy, K., He, X.: Stability in asymmetric Hopfield nets with transmission delays. Physica D 76, 344–358 (1994)zbMATHMathSciNetGoogle Scholar
  11. 11.
    Halanay, A.: Invariant manifolds for systems with time lag. In: Hale, J.K., La Salle, J.P. (eds.) Differential and dynamical systems, pp. 199–213. Acad. Press, New York (1967)Google Scholar
  12. 12.
    Halanay, A., Răsvan, V.: Applications of Liapunov Methods to Stability. Kluwer Academic Publishers, Dordrecht (1993)Google Scholar
  13. 13.
    Hale, J.K., Verduyn Lunel, S.M.: Introduction to Functional Differential Equations. Springer, Heidelberg (1993)zbMATHGoogle Scholar
  14. 14.
    Kharitonov, V.L., Zhabko, A.P.: Lyapunov-Krasovskii approach to the robust stability analysis of time-delay systems. Automatica 39, 15–20 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Kitamura, S., Hirai, K., Nishimura, M.: Stability of a Control System with Several Nonlinear Elements and Time Lags. Technology Repts. Osaka Univ. 17, 93–102 (1967)Google Scholar
  16. 16.
    König, P., Schillen, J.B.: Stimulus dependent assembly formation of oscillatory responses: I. Synchronization. Neural Computation 3, 155–166 (1991)CrossRefGoogle Scholar
  17. 17.
    Kopell, N.: We got the rhythm: dynamical systems of the nervous system. Notices AMS 47, 6–16 (2000)zbMATHMathSciNetGoogle Scholar
  18. 18.
    Leonov, G.A., Reitmann, V., Smirnova, V.B.: Pendulum like Feedback Systems. Teubner Verlag, Leipzig (1992)zbMATHGoogle Scholar
  19. 19.
    Nishimura, M., Kitamura, S.: A Lyapunov Functional for Systems with Multiple Non-linearities and Time Lags. Technology Repts. Osaka Univ. 19, 83–88 (1969)Google Scholar
  20. 20.
    Osareh, A., Mirmehdi, M., Thomas, B., Markham, R.: Automatic Identification of Diabetic Retinal Exudates in Digital Colour Images. British Journal of Ophthalmology 87(10), 1220–1223 (2003)CrossRefGoogle Scholar
  21. 21.
    Osareh, A.: Automated Identification of Diabetic Retinal Exudates and the Optic Disc. PhD Thesis, Bristol University, Faculty of Engineering, Department of Computer Science (2004)Google Scholar
  22. 22.
    Popov, V.M.: Monotonicity and Mutability. Journ. Diff. Eqs. 31, 337–358 (1979)CrossRefzbMATHGoogle Scholar
  23. 23.
    Răsvan, V.: Dynamical Systems with Several Equilibria and Natural Liapunov Functions. Archivum mathematicum 34, 207–215 (1998)zbMATHGoogle Scholar
  24. 24.
    Răsvan, V., Danciu, D.: Neural networks - global behavior versus delay. Periodica Politechnica. Trans. Autom. Control and Comp. Sci. 49(63), 11–14 (2004)Google Scholar
  25. 25.
    Saperstone, S.: Semidynamical Systems in Infinite Dimensional Spaces. Springer, Heidelberg (1981)zbMATHGoogle Scholar
  26. 26.
    Yi, Z.: Global exponential stability and periodic solutions of delay Hopfield neural netorks. Int. Journ. Syst. Sci. 27, 227–231 (1996)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Daniela Danciu
    • 1
  • Vladimir Răsvan
    • 1
  1. 1.University of Craiova, Department of Automatic Control, 13, A.I. Cuza str., 200585 - CraiovaRomania

Personalised recommendations