Neural Processing Letters

, Volume 28, Issue 1, pp 35–47 | Cite as

Stability and Chaos of a Class of Learning Algorithms for ICA Neural Networks

  • Jian Cheng Lv
  • Kok Kiong Tan
  • Zhang Yi
  • Sunan Huang


Independent component analysis (ICA) neural networks can estimate independent components from the mixed signal. The dynamical behavior of the learning algorithms for ICA neural networks is crucial to effectively apply these networks to practical applications. The paper presents the stability and chaotic dynamical behavior of a class of ICA learning algorithms with constant learning rates. Some invariant sets are obtained so that the non-divergence of these algorithms can be guaranteed. In these invariant sets, the stability and chaotic behaviors are analyzed. The conditions for stability and chaos are derived. Lyapunov exponents and bifurcation diagrams are presented to illustrate the existence of chaotic behavior.


Independent component analysis Dynamical behavior Bifurcation and chaos Lyapunov exponents 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Comon P (1994) Independent component analysis - a new concept. Signal Process 36: 287–314zbMATHCrossRefGoogle Scholar
  2. 2.
    Jutten C, Herault J (1991) Blind separation of sources part I an adaptive algorithm based on neuromimetic architecture. Signal Process 24: 1–10zbMATHCrossRefGoogle Scholar
  3. 3.
    Bell A, Sejnowski TJ (1997) Edges are the independent components of natural scenes. In: Advances in neural information processing 9 (NIPS 96). MIT Press, pp 831–837Google Scholar
  4. 4.
    Karhunen J, Hyvärinen A, Vigario R, Hurri J, Oja E (1997) Applications of neural blind separation to signal and image processing. In: Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP 97). Munich, Germany, pp 131–134Google Scholar
  5. 5.
    Kotani M, Ozawa S (2005) Feature extraction using independent components of each category. Neural Process Lett 22: 113–124CrossRefGoogle Scholar
  6. 6.
    Donoho D (1981) On minimum entropy deconvolution. In: Applied Time Series Analysis II. Academic Press, pp 565–608Google Scholar
  7. 7.
    Cichocki A, Thawonmas R (2000) On-line algorithms for blind signal extraction of arbitrarily distributed, but temporally correlated sources using second order statistics. Neural Process Lett 12: 91–98zbMATHCrossRefGoogle Scholar
  8. 8.
    Hyvärinen A, Oja E (1997) Simple neuron models for independent component analysis. Int J Neural Syst 7(6): 671–687CrossRefGoogle Scholar
  9. 9.
    Hyvärinen A, Oja E (1998) Independent component analysis by general non-linear hebbian-like learning Rues. Signal Process 64(3): 301–313zbMATHCrossRefGoogle Scholar
  10. 10.
    Ilin A, Valpola H (2005) On the effect of the form of the posterior approximation in variational learning of ICA models. Neural Process Lett 22: 183–204CrossRefGoogle Scholar
  11. 11.
    Karhunen J, Oja E, Wang L, Vigario R, Joutsensalo J (1997) A class of neural networks for independent component analysis. IEEE Trans Neural Netw 8(3): 486–504CrossRefGoogle Scholar
  12. 12.
    Karhunen J (2005) A resampling test for the total independence of stationary time series: application to the performance evaluation of ICA algorithms. Neural Process Lett 22: 311–324CrossRefGoogle Scholar
  13. 13.
    Mollah MNH, Eguchi S, Minami M (2007) Robust prewhitening for ICA by minimizing β-divergence and its application to fastICA. Neural Process lett 25: 91–110CrossRefGoogle Scholar
  14. 14.
    Oja E, Yuan Z (2006) The fastICA algorithm revisited: convergence analysis. IEEE Trans Neural Netw 17(6): 1370–1381CrossRefGoogle Scholar
  15. 15.
    Wang X, Tian L (2006) Bifurcation analysis and linear control of the Newton-Leipnik system. Chaos Solitons Fractals 27: 31–38zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    AGIZA HN (1999) On the analysis of stability, bifurcation, chaos and chaos control of Kopel Map. Chaos Solitons Fractals 10: 1909–1916zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Cheng Z, Lin Y, Cao J (2006) Dynamical behaviors of a partial-dependent predator–prey system. Chaos Solitons Fractals 28: 67–75CrossRefMathSciNetGoogle Scholar
  18. 18.
    Jing Z, Yang J (2006) Bifurcation and chaos in discrete-time predator–prey system. Chaos Solitons Fractals 27: 259–277zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Li C, Chen G (2004) Chaos in the fractional order Chen system and its control. Chaos Solitons Fractals 22: 549–554zbMATHCrossRefGoogle Scholar
  20. 20.
    Lv JC, Yi Z (2007) Stability and chaos of LMSER PCA learning algorithm. Chaos Solitons Fractals 32: 1440–1447zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Lv JC, Yi Z, Tan KK (2007) Global convergence of GHA learning algorithm with nonzero-approaching learning rates. IEEE Trans Neural Netw 18(6): 1557–1571CrossRefGoogle Scholar
  22. 22.
    Lv JC, Yi Z, Tan KK (2006) Global convergence of Oja’s PCA learning algorithm with a non-zero-approaching adaptive learning rate. Theor Comput Sci 367: 286–307zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Lv JC, Yi Z, Tan KK (2006) Convergence analysis of Xu’s LMSER learning algorithm via deterministic discrete time system method. Neurocomputing 70: 362–372CrossRefGoogle Scholar
  24. 24.
    Xu L, Oja E, Suen CY (1992) Modified hebbian learning for curve and surface fitting. Neural Netw 5: 441–457CrossRefGoogle Scholar
  25. 25.
    Yi Z, Ye M, Lv JC, Tan KK (2005) Convergence analysis of a deterministic discrete time system of Oja’s PCA learning algorithm. IEEE Trans Neural Netw 16(6): 1318–1328CrossRefGoogle Scholar
  26. 26.
    Zufiria PJ (2002) On the discrete-time dynamic of the basic Hebbian neural-network nodes. IEEE Trans Neural Netw 13(6): 1342–1352CrossRefGoogle Scholar
  27. 27.
    Yi Z, Tan KK (2004) Convergence analysis of recurrent neural networks. Kluwer Academic PublishersGoogle Scholar
  28. 28.
    Dror G, Tsodyks M (2000) Chaos in neural networks with dynamical synapses. Neurocomputing 32: 365–370CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2008

Authors and Affiliations

  • Jian Cheng Lv
    • 1
  • Kok Kiong Tan
    • 1
  • Zhang Yi
    • 2
  • Sunan Huang
    • 1
  1. 1.Department of Electrical and Computer EngineeringNational University of SingaporeSingaporeSingapore
  2. 2.Computational Intelligence Laboratory, School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduPeople’s Republic of China

Personalised recommendations