Advertisement

Flexible Architecture of Self Organizing Maps for Changing Environments

  • Rodrigo Salas
  • Héctor Allende
  • Sebastián Moreno
  • Carolina Saavedra
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3773)

Abstract

Catastrophic Interference is a well known problem of Artificial Neural Networks (ANN) learning algorithms where the ANN forget useful knowledge while learning from new data. Furthermore the structure of most neural models must be chosen in advance.

In this paper we introduce a hybrid algorithm called Flexible Architecture of Self Organizing Maps (FASOM) that overcomes the Catastrophic Interference and preserves the topology of Clustered data in changing environments. The model consists in K receptive fields of self organizing maps. Each Receptive Field projects high-dimensional data of the input space onto a neuron position in a low-dimensional output space grid by dynamically adapting its structure to a specific region of the input space.

Furthermore the FASOM model automatically finds the number of maps and prototypes needed to successfully adapt to the data. The model has the capability of both growing its structure when novel clusters appears and gradually forgets when the data volume is reduced in its receptive fields.

Finally we show the capabilities of our model with experimental results using synthetic sequential data sets and real world data.

Keywords

Catastrophic Interference Artificial Neural Networks Self Organizing Maps Pattern Recognition 

References

  1. 1.
    Ans, B.: Sequential learning in distributed neural networks without catastrophic forgetting: A single and realistic self-refreshing memory can do it. Neural Information Processing - Letters and Reviews 4(2), 27–37 (2004)Google Scholar
  2. 2.
    Bauer, H., Villmann, T.: Growing a hypercubical output space in a self-organizing feature map. IEEE Trans. on Neural Networks 8(2), 226–233 (1997)CrossRefGoogle Scholar
  3. 3.
    Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: ordering, convergence properties and energy functions. Biological Cybernetics 67, 47–55 (1992)zbMATHCrossRefGoogle Scholar
  4. 4.
    French, R.: Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences 3, 128–135 (1999)CrossRefGoogle Scholar
  5. 5.
    Fritzke, B.: Growing cell structures - a self-organizing network for unsupervised and supervised learning. Neural Networks 7(9), 1441–1460 (1994)CrossRefGoogle Scholar
  6. 6.
    Grossberg, S.: Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Reidel Press, Boston (1982)zbMATHGoogle Scholar
  7. 7.
    Jain, A.K., Dubes, R.C.: Algorithms for clustering data. Prentice-Hall, Englewood Cliffs (1988)zbMATHGoogle Scholar
  8. 8.
    Kohonen, T.: Self-Organizing Maps, 3rd extended edn. Springer Series in Information Sciences, vol. 30. Springer Verlag, Berlin, Heidelberg (2001)zbMATHGoogle Scholar
  9. 9.
    Maulik, U., Bandyopadhyay, S.: Performance evaluation. IEEE. Trans. on Pattern Analysis and Machine Intelligence 24(12), 1650–1654 (2002)CrossRefGoogle Scholar
  10. 10.
    McCloskey, M., Cohen, N.: Catastrophic interference in connectionist networks: The sequential learning problem. The psychology of Learning and Motivation 24, 109–164 (1989)CrossRefGoogle Scholar
  11. 11.
    McQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and probability, vol. 1, pp. 281–297 (1967)Google Scholar
  12. 12.
    Moreno, S., Allende, H., Rogel, C., Salas, R.: Robust growing hierarchical self organizing map. In: Cabestany, J., Prieto, A.G., Sandoval, F. (eds.) IWANN 2005. LNCS, vol. 3512, pp. 341–348. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Saavedra, C., Allende, H., Moreno, S., Salas, R.: K-dynamical self organizing maps, To appear in Lecture Notes in Computer Science (November 2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Rodrigo Salas
    • 1
    • 2
  • Héctor Allende
    • 2
    • 3
  • Sebastián Moreno
    • 2
  • Carolina Saavedra
    • 2
  1. 1.Departamento de ComputaciónUniversidad de ValparaísoValparaísoChile
  2. 2.Dept. de InformáticaUniversidad Técnica Federico Santa MaríaValparaísoChile
  3. 3.Facultad de Ciencia y TecnologíaUniversidad Adolfo Ibañez 

Personalised recommendations