Soft Computing

, Volume 11, Issue 3, pp 239–248 | Cite as

Advances in Design and Application of Spiking Neural Networks

  • Ammar Belatreche
  • Liam P. Maguire
  • Martin McGinnity
Focus
First Online:

Abstract

This paper presents new findings in the design and application of biologically plausible neural networks based on spiking neuron models, which represent a more plausible model of real biological neurons where time is considered as an important feature for information encoding and processing in the brain. The design approach consists of an evolutionary strategy based supervised training algorithm, newly developed by the authors, and the use of different biologically plausible neuronal models. A dynamic synapse (DS) based neuron model, a biologically more detailed model, and the spike response model (SRM) are investigated in order to demonstrate the efficacy of the proposed approach and to further our understanding of the computing capabilities of the nervous system. Unlike the conventional synapse, represented as a static entity with a fixed weight, employed in conventional and SRM-based neural networks, a DS is weightless and its strength changes upon the arrival of incoming input spikes. Therefore its efficacy depends on the temporal structure of the impinging spike trains. In the proposed approach, the training of the network free parameters is achieved using an evolutionary strategy where, instead of binary encoding, real values are used to encode the static and DS parameters which underlie the learning process. The results show that spiking neural networks based on both types of synapse are capable of learning non-linearly separable data by means of spatio-temporal encoding. Furthermore, a comparison of the obtained performance with classical neural networks (multi-layer perceptrons) is presented.

Keywords

Spiking neurons Spike response model Integrate-and-fire model Dynamic synapse Evolutionary strategy Temporal coding Supervised learning 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Gerstner W (1995) Time structure of the activity in neural network models. Phys Rev E 51(1):738–758CrossRefGoogle Scholar
  2. 2.
    Gerstner W, Kistler W (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, CambridgeGoogle Scholar
  3. 3.
    Maass W, Bishop CM (eds) (1999) Pulsed neural networks. The MIT Press, CambridgeMATHGoogle Scholar
  4. 4.
    Hopfield JJ (1995) Pattern recognition computation using action potential timing for stimulus representation. Nature 376:33–36CrossRefPubMedGoogle Scholar
  5. 5.
    Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10:1659–1671CrossRefGoogle Scholar
  6. 6.
    Thorpe SJ, Fize F, Marlot C (1996) Speed of processing in the human visual system. Nature 381:520–522CrossRefPubMedGoogle Scholar
  7. 7.
    Thorpe S, Delorme A, Van Rullen R (2001) Spike-based strategies for rapid processing. Neural Netw 14:715–725PubMedCrossRefGoogle Scholar
  8. 8.
    Gautrais J, Thorpe S (1998) Rate coding versus temporal order coding: a theoretical approach. BioSystems 48:57–65PubMedCrossRefGoogle Scholar
  9. 9.
    Tsodyks M, Pawelzik K, Markram H (1998) Neural networks with dynamic synapses. Neural Comput 10:821–835CrossRefPubMedGoogle Scholar
  10. 10.
    Fuhrmann G, Segev I, Markram H, Tsodyks M (2002) Coding of temporal information by activity dependent synapses. J Neurophysiol 87:140–148PubMedGoogle Scholar
  11. 11.
    Markram H, Tsodyks M (1996) Redistribution of synaptic efficacy between pyramidal neurons. Nature 382:807–810PubMedCrossRefGoogle Scholar
  12. 12.
    Pantic L, Torres JJ, Kappen HJ (2003) Coincidence detection with dynamic synapses. Netw Comput Neural Syst 14(1):17–34CrossRefGoogle Scholar
  13. 13.
    Zador A, Dobrunz L (1997) Dynamic synapses in the cortex. Neuron 19:1–4CrossRefPubMedGoogle Scholar
  14. 14.
    George S, Dibazar A, Desai V, Berger TW (2003) Using dynamic synapse based neural networks with wavelet preprocessing for speech applications, Neural Networks. In: Proceedings of the international joint conference on volume 1, pp 666–669. 20–24 July 2003Google Scholar
  15. 15.
    Dibazar AA, Namarvar HH, Berger TW (2003) A new approach for isolated word recognition using dynamic synapse neural networks, Neural Networks. In: Proceedings of the international joint conference on volume 4, pp 3146–3150. July 20–24, 2003Google Scholar
  16. 16.
    Heittmann A, Ramacher U (2004) An architecture for feature detection utilizing dynamic synapses, Circuits and Systems, 2004. MWSCAS ’04. The 2004 47th Midwest symposium on volume 2, pp II 373–376. 25–28 July 2004Google Scholar
  17. 17.
    Song S, Miller KD et al (2000) Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3:919–926CrossRefPubMedGoogle Scholar
  18. 18.
    Bohte SM, La Poutré H, Kok JN (2000) SpikeProp: error- for networks of spiking neurons. ESANN’2000, pp 419–425Google Scholar
  19. 19.
    Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102CrossRefMATHGoogle Scholar
  20. 20.
    Rao RP, Sejnowski TJ (2001) Spike-timing-dependent plasticity as temporal difference learning. Neural Comput 13:2221–2237CrossRefPubMedMATHGoogle Scholar
  21. 21.
    Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544PubMedGoogle Scholar
  22. 22.
    Belatreche A, Maguire LP, McGinnity M (2004) Pattern recognition with spiking neural networks and dynamic synapses. In: Applied computational intelligence, world scientific, proceedings of the 6th international FLINS conference, pp 205–210Google Scholar
  23. 23.
    Bäck T, Schwefel H-P (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1–23Google Scholar
  24. 24.
    Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison Wesley, ReadingMATHGoogle Scholar
  25. 25.
    Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann ArborGoogle Scholar
  26. 26.
    Beyer H-G (1996) Toward a theory of evolution strategies: self-adaptation. Evol Comput 3(3):311 –347Google Scholar
  27. 27.
    Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7:179–188Google Scholar
  28. 28.
    Woldberg WH, Mangasarian OL (1990) Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proc Natl Acad Sci USA 87:9193–9196PubMedCrossRefGoogle Scholar
  29. 29.
    Mangasarian OL, Wolberg WH (1990) Cancer diagnosis via linear programming. SIAM News 23(5):1–18Google Scholar
  30. 30.
    Belatreche A, Maguire LP, McGinnity M, Wu QX (2003) An evolutionary strategy for supervised training of biologically plausible neural networks. In: The sixth international conference on computational intelligence and natural computing (CINC), proceedings of the 7th joint conference on information sciences, pp 1524–1527, North Carolina, USAGoogle Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • Ammar Belatreche
    • 1
  • Liam P. Maguire
    • 1
  • Martin McGinnity
    • 1
  1. 1.Intelligent Systems Engineering Laboratory, School of Computing and Intelligent Systems, Faculty of EngineeringUniversity of UlsterNorthland Road, DerryNorthern Ireland

Personalised recommendations