Advertisement

Cognitive Neurodynamics

, Volume 8, Issue 4, pp 299–311 | Cite as

Post and pre-compensatory Hebbian learning for categorisation

  • Christian R. HuyckEmail author
  • Ian G. Mitchell
Research Article

Abstract

A system with some degree of biological plausibility is developed to categorise items from a widely used machine learning benchmark. The system uses fatiguing leaky integrate and fire neurons, a relatively coarse point model that roughly duplicates biological spiking properties; this allows spontaneous firing based on hypo-fatigue so that neurons not directly stimulated by the environment may be included in the circuit. A novel compensatory Hebbian learning algorithm is used that considers the total synaptic weight coming into a neuron. The network is unsupervised and entirely self-organising. This is relatively effective as a machine learning algorithm, categorising with just neurons, and the performance is comparable with a Kohonen map. However the learning algorithm is not stable, and behaviour decays as length of training increases. Variables including learning rate, inhibition and topology are explored leading to stable systems driven by the environment. The model is thus a reasonable next step toward a full neural memory model.

Keywords

Compensatory Hebbian learning Categorisation Spontaneous neural spiking Neural fatigue Point neural model Self-organisation 

Notes

Acknowledgments

Thanks to Zhijun Yang and Dan Diaper for comments on this paper.

References

  1. Abbott L (1999) Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res 50:303–304Google Scholar
  2. Ackley D, Hinton G, Sejnowski T (1985) A learning algorithm for Boltzmann machines. Cogn Sci 9:147–169CrossRefGoogle Scholar
  3. Amit D (1989) Modelling brain function: the world of attractor neural networks. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  4. Athitsos V, Sclaroff S (2004) Boosting nearest neighbor classifiers for multiclass recognition. Technical report, Boston UniversityGoogle Scholar
  5. Bache K, Lichman M (2013) UCI machine learning repository. School of Information and Computer Science, University of California, Irvine. http://archive.ics.uci.edu/ml
  6. Bi G, Poo M (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18:24:10464–10472Google Scholar
  7. Bienenstock E, Cooper L, Munro P (1982) Theory for the development of neuron selectivity: orientation specificity and binocular interaction in the visual cortex. J Neurosci 2:1:32–48Google Scholar
  8. Brette R, Rudolph M, Carnevale T, Hines M, Beeman D, Bower J, Diesmann M, Morrison A, Goodman P, Harris F, Zirpe M, Natschalager T, Pecevski D, Ermentrout B, Djurfeldt M, Lansner A, Rochel O, Vieville T, Muller E, Dafison A, ElBoustani S, Destexhe A (2007) Simulation of networks of spiking neurons: a review of tools and strategies. J Comput Neurosci 23:349–398PubMedCentralPubMedCrossRefGoogle Scholar
  9. Bush D, Philippides A, Husbands P, O’Shea M (2010) Reconciling the stdp and bcm models of synaptic plasticity in a spiking recurrent neural network. Neural Comput 22:2059–2085PubMedCrossRefGoogle Scholar
  10. Freedman D, Riesenhuber M, Poggio T, Miller E (2001) Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291:312–316PubMedCrossRefGoogle Scholar
  11. Fyfe C (2005) Hebbian learning and negative feedback networks. Springer, BerlinGoogle Scholar
  12. Hebb D (1949) The organization of behavior. Wiley, LondonGoogle Scholar
  13. Hodgkin A, Huxley A (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544PubMedCentralPubMedGoogle Scholar
  14. Hsu D, Tan A, Hsu M, Beggs J (2007) A simple spontaneously active hebbian learning model: homeostasis of activity and connectivity, and consequences for learning and epileptogensis. Phys Rev E 76:041909CrossRefGoogle Scholar
  15. Huyck C (2007) Creating hierarchical categories using cell assemblies. Connect Sci 19:1:1–24CrossRefGoogle Scholar
  16. Huyck C, Mitchell I (2013) Compensatory hebbian learning for categorisation in simulated biological neural nets. Biol Inspir Cogn Arch 6:3–7Google Scholar
  17. Huyck C, Orengo V (2005) Information retrieval and categorisation using a cell assembly network. Neural Comput Appl 14:282–289CrossRefGoogle Scholar
  18. Huyck C, Parvizi A (2012) Parameter values and fatigue mechanisms for flif neurons. J Syst Cybern Inf 10:4:80–86Google Scholar
  19. Izhikevich E (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 15:5:1063–1070CrossRefGoogle Scholar
  20. Izhikevich E, Desai N (2003) Relating stdp to bcm. Neural Comput 15:1511–1523PubMedCrossRefGoogle Scholar
  21. Kohn A (2007) Visual adaptation: Physiology, mechanisms, and functional benefits. J Neurophysiol 97:3155–3164PubMedCrossRefGoogle Scholar
  22. Kohonen T (1997) Self-organizing maps. Springer, LondonCrossRefGoogle Scholar
  23. McCulloch W, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133CrossRefGoogle Scholar
  24. Mitchell I, Huyck C (2013) Self organising maps with a point neuron model. In 17th international conference on cognitive and neural systemsGoogle Scholar
  25. O’Reilly R (1996) The Leabra Model of Neural Interactions and Learning in the Neocortex. PhD thesis, Carnegie Mellon University, Pittsburgh, PAGoogle Scholar
  26. Wehrens R, Buydens L (2007) Self- and super-organizing maps in R: the Kohonen package. J Stat Softw 21(5):1–9Google Scholar
  27. Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1:67–82CrossRefGoogle Scholar
  28. Wu Q, Maguire L, Glackin B, Belatreche A (2006) Learning under weight constraints in networks of temporal encoding spiking neurons. Neurocomputing 69:1912–1922CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  1. 1.Department of Computer ScienceMiddlesex UniversityLondonUK

Personalised recommendations