Abstract
Catastrophic Interference is a well known problem of Artificial Neural Networks (ANN) learning algorithms where the ANN forget useful knowledge while learning from new data. Furthermore the structure of most neural models must be chosen in advance.
In this paper we introduce a hybrid algorithm called Flexible Architecture of Self Organizing Maps (FASOM) that overcomes the Catastrophic Interference and preserves the topology of Clustered data in changing environments. The model consists in K receptive fields of self organizing maps. Each Receptive Field projects high-dimensional data of the input space onto a neuron position in a low-dimensional output space grid by dynamically adapting its structure to a specific region of the input space.
Furthermore the FASOM model automatically finds the number of maps and prototypes needed to successfully adapt to the data. The model has the capability of both growing its structure when novel clusters appears and gradually forgets when the data volume is reduced in its receptive fields.
Finally we show the capabilities of our model with experimental results using synthetic sequential data sets and real world data.
Keywords
- Catastrophic Interference
- Artificial Neural Networks
- Self Organizing Maps
- Pattern Recognition
This work was supported in part by Research Grant Fondecyt 1040365 and 7050205, DGIP-UTFSM, BMBF-CHL 03-Z13 from German Ministry of Education, DIPUV-22/2004 and CID-04/2003
Chapter PDF
References
Ans, B.: Sequential learning in distributed neural networks without catastrophic forgetting: A single and realistic self-refreshing memory can do it. Neural Information Processing - Letters and Reviews 4(2), 27–37 (2004)
Bauer, H., Villmann, T.: Growing a hypercubical output space in a self-organizing feature map. IEEE Trans. on Neural Networks 8(2), 226–233 (1997)
Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps: ordering, convergence properties and energy functions. Biological Cybernetics 67, 47–55 (1992)
French, R.: Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences 3, 128–135 (1999)
Fritzke, B.: Growing cell structures - a self-organizing network for unsupervised and supervised learning. Neural Networks 7(9), 1441–1460 (1994)
Grossberg, S.: Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Reidel Press, Boston (1982)
Jain, A.K., Dubes, R.C.: Algorithms for clustering data. Prentice-Hall, Englewood Cliffs (1988)
Kohonen, T.: Self-Organizing Maps, 3rd extended edn. Springer Series in Information Sciences, vol. 30. Springer Verlag, Berlin, Heidelberg (2001)
Maulik, U., Bandyopadhyay, S.: Performance evaluation. IEEE. Trans. on Pattern Analysis and Machine Intelligence 24(12), 1650–1654 (2002)
McCloskey, M., Cohen, N.: Catastrophic interference in connectionist networks: The sequential learning problem. The psychology of Learning and Motivation 24, 109–164 (1989)
McQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and probability, vol. 1, pp. 281–297 (1967)
Moreno, S., Allende, H., Rogel, C., Salas, R.: Robust growing hierarchical self organizing map. In: Cabestany, J., Prieto, A.G., Sandoval, F. (eds.) IWANN 2005. LNCS, vol. 3512, pp. 341–348. Springer, Heidelberg (2005)
Saavedra, C., Allende, H., Moreno, S., Salas, R.: K-dynamical self organizing maps, To appear in Lecture Notes in Computer Science (November 2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Salas, R., Allende, H., Moreno, S., Saavedra, C. (2005). Flexible Architecture of Self Organizing Maps for Changing Environments. In: Sanfeliu, A., Cortés, M.L. (eds) Progress in Pattern Recognition, Image Analysis and Applications. CIARP 2005. Lecture Notes in Computer Science, vol 3773. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11578079_67
Download citation
DOI: https://doi.org/10.1007/11578079_67
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29850-2
Online ISBN: 978-3-540-32242-9
eBook Packages: Computer ScienceComputer Science (R0)
Publish with us
-
Published in cooperation with
http://www.iapr.org/
