Dynamics of Learning In Hierarchical Models – Singularity and Milnor Attractor

  • Shun-ichi AmariEmail author
  • Tomoko Ozeki
  • Florent Cousseau
  • Haikun Wei
Conference paper


We study the dynamics of learning in a hierarchical model such as multilayer perceptron. Such a model includes singularities, which affects its dynamics seriously. The Milnor type attractors appear, because of the singularity. We will show its trajectories explicitly, and present the topological nature of the singularities.


Dynamics of learning On-line learning Multilayer-perceptron Singularity Milnor attractor 


  1. 1.
    Cousseau, F., Ozeki, T., Amari, S.: Dynamics of learning in multilayer perceptrons near singularities. IEEE. Trans. Neural. Netw. 19 (2008) 1313–1328.CrossRefPubMedGoogle Scholar
  2. 2.
    Amari, S.: Natural gradient works efficiently in learning. Neural. Comput. 10 (1998) 251–276.CrossRefGoogle Scholar
  3. 3.
    Amari, S., Park, H., Ozeki, T.: Singularities affect dynamics of learning in neuromanifolds. Neural. Comput. 18 (2006) 1007–1065.CrossRefPubMedGoogle Scholar
  4. 4.
    Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural. Netw. 13 (2000) 317–327.CrossRefPubMedGoogle Scholar
  5. 5.
    Wei, H., Zhang, J., Cousseau, F., Ozeki, T., Amari, S.: Dynamics of learning near singularities in layered networks. Neural. Comput. 20 (2008) 813–843.CrossRefPubMedGoogle Scholar
  6. 6.
    Wei, H., Amari, S.: Dynamics of learning near singularities in radial basis function networks. Neural. Netw. 21 (2008) 989–1005.CrossRefPubMedGoogle Scholar
  7. 7.
    Milnor, J.: On the Concept of Attractor. Commun. Math. Phys. 99 (1985) 177–195.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Shun-ichi Amari
    • 1
    Email author
  • Tomoko Ozeki
    • 2
  • Florent Cousseau
    • 3
  • Haikun Wei
    • 4
  1. 1.RIKEN Brain Science InstituteSaitamaJapan
  2. 2.Department of Human and Information ScienceTOKAI UniversityKanagawaJapan
  3. 3.Graduate School of Frontier SciencesThe University of TokyoChibaJapan
  4. 4.Research Institute of AutomationSoutheast UniversityNanjingChina

Personalised recommendations