Abstract
This paper describes a neural network architecture which has been developed specifically to investigate and alleviate the effects of catastrophic interference. This is the tendency of certain types of feed forward network to forget what they have learned when required to learn a second pattern set which overlaps significantly in content with the first. This work considers a neural network architecture which performs a pattern separated representation of the inputs and develops an attractor dynamic representation, which is subsequently associated with the original pattern. The paper then describes an excitatory and inhibitory function which ensures only the top firing neurons are retained. The paper considers the biological plausibility of this network and reports a series of experiments designed to evaluate the neural networks ability to recall patterns after learning a second data set, as well as the time to relearn the original data set.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
McCloskey M, and Cohen N Catastrophic Interference in Connectionist Networks. The Sequential learning problem, In G H Bower (ed), The Psychology and Learning of Motivation, Vol 24, 109–164, NY: Academic Press. 1989
Kortge, C. A, Episodic memory in connectionist networks, 1993, Proceedings of the Cognitive Science Society, pp 764–771, Hillsdale, NJ, Erlbaum.
French, R.M Semi distributed representations and catastrophic forgetting in connectionist networks, 1992, Connection Science, 4, 365–377
Sloman, S.A & Rummelhart D E, Reducing interference in distributed memories through episodic gating, In Healy. A, Kosslyn. S & Shiffrin. R (eds) Essays in honour of W.K Estes, Hillsdale, NJ: Erlbaum 1992
McRae, K & Hetherington, P.A. Catastrophic Interference is eliminated in pretrained networks, 1993, Proceedings of the Fifteenth Annual Conference of the Cognitive Society, 723–728, Hillsdale, NJ, Erlbaum
French, R.M, Pseudo Recurrent Connectionist Networks: An approach to the “sensitivity-stability” dilemma, Connection Science, 1997, 9(4), 353–379
Kanerva, P Sparse distributed memory, Cambridge, MA: Bradford books, 1989
Kruschke, J. ALCOVE: An exemplar based model of category learning, 1992, Psychology Review, 9, 22–44
Ans B, Rousset S, French. R M, & Musca, S Preventing Catastrophic Interference in multiple sequence learning using coupled reverberating Elman Networks, 2002, Proceedings of the 24th Annual Conference of the Cognitive Science Society, NJ: LEA
Kortge, C A Episodic memory in connectionist networks, 1993, In proceedings of the 12th Annual meeting of the Cognitive Science Society, 764–771
Ramon y Cajal, Histology, 10th edition, Baltimore Wood, 1937
Scoville, W B & Milner B, Loss of recent memory after bilateral hippocampal lesions, Journal of Neuro Chemistry, 1957, Feb 20(1): 11–21.
Macleod P, Plunkett P, Rolls E Introduction to connectionist modelling of cognitive processes, Oxford University Press, 1998.
Hebb, D The Organisation of Behaviour, Wiley & Sons, New York, 1949
Marr, D A theory of Cerebellar Cortex, Journal Physiology, 1969, 202, 437–470
McNaughton, B L & Morris, R G M Hippocampal synaptic enhancement and information storage within a distributed memory system, Trends in neurosciences, 1987, 10(10), 408–415
Hopfield J J, Neurons with graded responses have collective computational properties like those of two state neurons, Proceedings of the National academy of sciences, 81, 3088–3092, 1984
Kosko, B Hidden patterns in combined adaptive knowledge neural networks, 1988, International journal of approximate reasoning, 2, 337–393
Yonelinas, A P The nature of recollection and familiarity: A review of 30 years of research, 2002, Journal of memory and language, 46, 441–517
Blake, CL & Merz, C J, UCI Repository pf machine learning databases [http:www.ics.uci.edu/~mlearn/mlrepository.html], Irvine, CA University of California, Department of information and computer science, 1998
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag London Limited
About this paper
Cite this paper
Dale Addison, J.F., Arevian, G.Z., MacIntyre, J. (2006). A biologically motivated neural network architecture for the avoidance of catastrophic interference. In: Bramer, M., Coenen, F., Allen, T. (eds) Research and Development in Intelligent Systems XXII. SGAI 2005. Springer, London. https://doi.org/10.1007/978-1-84628-226-3_12
Download citation
DOI: https://doi.org/10.1007/978-1-84628-226-3_12
Publisher Name: Springer, London
Print ISBN: 978-1-84628-225-6
Online ISBN: 978-1-84628-226-3
eBook Packages: Computer ScienceComputer Science (R0)