Advertisement

A biologically motivated neural network architecture for the avoidance of catastrophic interference

  • J F Dale Addison
  • Garen Z Arevian
  • John MacIntyre

Abstract

This paper describes a neural network architecture which has been developed specifically to investigate and alleviate the effects of catastrophic interference. This is the tendency of certain types of feed forward network to forget what they have learned when required to learn a second pattern set which overlaps significantly in content with the first. This work considers a neural network architecture which performs a pattern separated representation of the inputs and develops an attractor dynamic representation, which is subsequently associated with the original pattern. The paper then describes an excitatory and inhibitory function which ensures only the top firing neurons are retained. The paper considers the biological plausibility of this network and reports a series of experiments designed to evaluate the neural networks ability to recall patterns after learning a second data set, as well as the time to relearn the original data set.

Keywords

Entorhinal Cortex Mossy Fibre Neural Network Architecture Transfer Layer Back Propagation Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    McCloskey M, and Cohen N Catastrophic Interference in Connectionist Networks. The Sequential learning problem, In G H Bower (ed), The Psychology and Learning of Motivation, Vol 24, 109–164, NY: Academic Press. 1989Google Scholar
  2. 2.
    Kortge, C. A, Episodic memory in connectionist networks, 1993, Proceedings of the Cognitive Science Society, pp 764–771, Hillsdale, NJ, Erlbaum.Google Scholar
  3. 3.
    French, R.M Semi distributed representations and catastrophic forgetting in connectionist networks, 1992, Connection Science, 4, 365–377CrossRefGoogle Scholar
  4. 4.
    Sloman, S.A & Rummelhart D E, Reducing interference in distributed memories through episodic gating, In Healy. A, Kosslyn. S & Shiffrin. R (eds) Essays in honour of W.K Estes, Hillsdale, NJ: Erlbaum 1992Google Scholar
  5. 5.
    McRae, K & Hetherington, P.A. Catastrophic Interference is eliminated in pretrained networks, 1993, Proceedings of the Fifteenth Annual Conference of the Cognitive Society, 723–728, Hillsdale, NJ, ErlbaumGoogle Scholar
  6. 6.
    French, R.M, Pseudo Recurrent Connectionist Networks: An approach to the “sensitivity-stability” dilemma, Connection Science, 1997, 9(4), 353–379CrossRefGoogle Scholar
  7. 7.
    Kanerva, P Sparse distributed memory, Cambridge, MA: Bradford books, 1989Google Scholar
  8. 8.
    Kruschke, J. ALCOVE: An exemplar based model of category learning, 1992, Psychology Review, 9, 22–44CrossRefGoogle Scholar
  9. 9.
    Ans B, Rousset S, French. R M, & Musca, S Preventing Catastrophic Interference in multiple sequence learning using coupled reverberating Elman Networks, 2002, Proceedings of the 24th Annual Conference of the Cognitive Science Society, NJ: LEAGoogle Scholar
  10. 10.
    Kortge, C A Episodic memory in connectionist networks, 1993, In proceedings of the 12th Annual meeting of the Cognitive Science Society, 764–771Google Scholar
  11. 11.
    Ramon y Cajal, Histology, 10th edition, Baltimore Wood, 1937Google Scholar
  12. 12.
    Scoville, W B & Milner B, Loss of recent memory after bilateral hippocampal lesions, Journal of Neuro Chemistry, 1957, Feb 20(1): 11–21.Google Scholar
  13. 13.
    Macleod P, Plunkett P, Rolls E Introduction to connectionist modelling of cognitive processes, Oxford University Press, 1998.Google Scholar
  14. 14.
    Hebb, D The Organisation of Behaviour, Wiley & Sons, New York, 1949Google Scholar
  15. 15.
    Marr, D A theory of Cerebellar Cortex, Journal Physiology, 1969, 202, 437–470Google Scholar
  16. 16.
    McNaughton, B L & Morris, R G M Hippocampal synaptic enhancement and information storage within a distributed memory system, Trends in neurosciences, 1987, 10(10), 408–415CrossRefGoogle Scholar
  17. 17.
    Hopfield J J, Neurons with graded responses have collective computational properties like those of two state neurons, Proceedings of the National academy of sciences, 81, 3088–3092, 1984CrossRefGoogle Scholar
  18. 18.
    Kosko, B Hidden patterns in combined adaptive knowledge neural networks, 1988, International journal of approximate reasoning, 2, 337–393CrossRefGoogle Scholar
  19. 19.
    Yonelinas, A P The nature of recollection and familiarity: A review of 30 years of research, 2002, Journal of memory and language, 46, 441–517CrossRefGoogle Scholar
  20. 20.
    Blake, CL & Merz, C J, UCI Repository pf machine learning databases [http:www.ics.uci.edu/~mlearn/mlrepository.html], Irvine, CA University of California, Department of information and computer science, 1998Google Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • J F Dale Addison
    • 1
  • Garen Z Arevian
    • 1
  • John MacIntyre
    • 1
  1. 1.School of Computing and TechnologyUniversity of SunderlandSunderlandEngland

Personalised recommendations