Evolving Systems

, Volume 4, Issue 2, pp 87–98 | Cite as

Evolving spiking neural network—a survey

Original Paper

Abstract

This paper provides a comprehensive literature survey on the evolving Spiking Neural Network (eSNN) architecture since its introduction in 2006 as a further extension of the ECoS paradigm introduced by Kasabov in 1998. We summarize the functioning of the method, discuss several of its extensions and present a number of applications in which the eSNN method was employed. We focus especially on some proposed extensions that allow the processing of spatio-temporal data and for feature and parameter optimisation of eSNN models to achieve better accuracy on classification/prediction problems and to facilitate new knowledge discovery. Finally, some open problems are discussed and future directions highlighted.

Keywords

Evolving Spiking Neural Network Evolving Connectionist Systems spatio-temporal pattern recognition 

References

  1. Angelov P, Filev D, Kasabov N (2008) Guest editorial evolving fuzzy systems—preface to the special section. IEEE Trans Fuzzy Syst 16(6):1390–1392. doi:10.1109/TFUZZ.2008.2006743 CrossRefGoogle Scholar
  2. Angelov P, Filev D, Kasabov N (eds) (2010) Evolving intelligent systems: methodology and applications. Wiley, HobokenGoogle Scholar
  3. Arbib M (ed) (2003) The handbook of brain theory and neural networks, 2nd edn. MIT Press, CambridgeGoogle Scholar
  4. Benuskova L, Jain V, Wysoski SG, Kasabov N (2006) Computational neurogenetic modeling: a pathway to new discoveries in genetic neuroscience. Int J Neural Syst 16(3):215–227CrossRefGoogle Scholar
  5. Benuskova L, Kasabov N (2007) Computational neurogenetic modelling. Springer, New YorkGoogle Scholar
  6. Bohte SM, Kok JN, Poutré JAL (2002) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4):17–37MATHCrossRefGoogle Scholar
  7. Carpenter G, Grossberg S, Markuzon N, Reynolds J, Rosen D (1991) Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analogue multi-dimensional maps. IEEE Trans Neural Netw 3(5):698–713CrossRefGoogle Scholar
  8. de Sousa HC, Riul Jr A (2002) Using MLP networks to classify red wines and water readings of an electronic tongue. Braz Symp Neural Netw 0, 13. doi:10.1109/SBRN.2002.1181428
  9. Defoin-Platel M, Schliebs S, Kasabov N (2007) A versatile quantum-inspired evolutionary algorithm. In: IEEE Congress on Evolutionary Computation, CEC’07, pp 423–430. IEEE Press, SingaporeGoogle Scholar
  10. Defoin-Platel M, Schliebs S, Kasabov N (2009) Quantum-inspired evolutionary algorithm: A multimodel EDA. IEEE Trans Evol Comput 13(6):1218–1232. doi:10.1109/TEVC.2008.2003010 CrossRefGoogle Scholar
  11. Delorme A, Gautrais J, VanRullen R, Thorpe S (1999) SpikeNET: a simulator for modeling large networks of integrate and fire neurons. http://citeseer.ist.psu.edu/delorme99spikenet.html
  12. Delorme A, Thorpe SJ (2001) Face identification using one spike per neuron: resistance to image degradations. Neural Netw 14(6–7):795–803CrossRefGoogle Scholar
  13. Delorme A, Perrinet L, Thorpe SJ (2001) Networks of integrate-and-fire neurons using rank order coding B: Spike timing dependent plasticity and emergence of orientation selectivity. Neurocomputing 38(40):539–545CrossRefGoogle Scholar
  14. Delorme A, Thorpe SJ (2003) SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons. Netw Comput Neural Syst 14:613–627. doi:10.1088/0954-898X/14/4/301 CrossRefGoogle Scholar
  15. Dias D, Madeo R, Rocha T, Biscaro H, Peres S (2009) Hand movement recognition for brazilian sign language: a study using distance-based neural networks. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009. pp 697–704. doi:10.1109/IJCNN.2009.5178917
  16. Fritzke B (1995) A growing neural gas network learns topologies. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems, vol 7. MIT Press, Cambridge, pp 625–632Google Scholar
  17. Fusi S, Annunziato M, Badoni D, Salamon A, Amit DJ (2000) Spike-driven synaptic plasticity: theory, simulation, vlsi implementation. Neural Comput Appl 12(10):2227–2258CrossRefGoogle Scholar
  18. Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, CambridgeGoogle Scholar
  19. Goodman D, Brette R (2008) Brian: a simulator for spiking neural networks in python. BMC Neuroscience 9(Suppl 1):P92 doi:10.1186/1471-2202-9-S1-P92
  20. Grossberg S (1982) Studies of the mind and brain. Reidel, Boston,Google Scholar
  21. Hamed H, Kasabov N, Shamsuddin S (2009) Integrated feature selection and parameter optimisation for evolving spiking neural networks using quantum inspired particle swarm optimisation. In: International Conference on Soft Computing and Pattern Recognition, pp 695–698. IEEE PressGoogle Scholar
  22. Hamed H, Kasabov N, Shamsuddin S (2010) Probabilistic evolving spiking neural network optimization using dynamic quantum-inspired particle swarm optimization. Aust J Intel Inf Process Syst 11(1):23–28Google Scholar
  23. Hamed HNA, Shamsuddin SM, Kasabov N (2011) Quantum-inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks. In: Numerical Analysis and Scientific Computing, pp 133–148. InTech, AUTGoogle Scholar
  24. Hisada M, Ozawa S, Zhang K, Kasabov N (2010) Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems. Evol Syst 1:17–27. doi:10.1007/s12530-010-9000-3 CrossRefGoogle Scholar
  25. Huang L, Song Q, Kasabov N (2008) Evolving connectionist system based role allocation for robotic soccer. Int J Adv Rob Syst 5:59–62Google Scholar
  26. Indiveri G, Linares-Barranco B, Julia Hamilton T, Van Schaik A, Etienne-Cummings R, Delbruck T, Liu SC, Dudek P, Häfliger P, Renaud S, Schemmel J, Cauwenberghs G, Arthur J, Hynna K, Folowosele F, Saighi S, Serrano-Gotarredona T, Wijekoon J, Wang Y, Boahen K (2011) Neuromorphic silicon neuron circuits. Front Neurosci :1–23. doi:10.3389/fnins.2011.00073
  27. Johnston SP, Prasad G, Maguire L, McGinnity TM (2010) An fpga hardware/software co-design towards evolvable spiking neural networks for robotics applications. Int J Neural Syst 20(6):447–461CrossRefGoogle Scholar
  28. Kasabov N (1998a) ECOS: Evolving connectionist systems and the ECO learning paradigm. In: Usui S, Omori T (eds) The Fifth international conference on neural information processing, ICONIP’98, pp 1232–1235. IOA Press, Kitakyushu, JapanGoogle Scholar
  29. Kasabov N (1998b) Evolving fuzzy neural networks-algorithms, applications and biological motivation. In: Yamakawa T, Matsumoto G (eds) Methodologies for the conception design and application of soft computing, pp 271–274. World Scientific, SingaporeGoogle Scholar
  30. Kasabov N (2001a) Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Trans Syst Man Cybern B Cybern 31(6):902–918. doi:10.1109/3477.969494 CrossRefGoogle Scholar
  31. Kasabov N (2001b) On-line learning, reasoning, rule extraction and aggregation in locally optimized evolving fuzzy neural networks. Neurocomputing 41(14):25–45. doi:10.1016/S0925-2312(00)00346-5 MATHCrossRefGoogle Scholar
  32. Kasabov N (2002) Evolving connectionist systems. Methods and applications in bioinformatics, brain study and intelligent machines. perspectives in neural computing. Springer, LondonGoogle Scholar
  33. Kasabov N, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154. doi:10.1109/91.995117 CrossRefGoogle Scholar
  34. Kasabov N, Benuskova L, Wysoski S (2005) A computational neurogenetic model of a spiking neuron. In: IEEE International Joint Conference on Neural Networks, 2005 IJCNN ’05. Proceedings 2005, vol 1, pp 446–451. doi:10.1109/IJCNN.2005.1555872
  35. Kasabov N (2006) Adaptation and interaction in dynamical systems: Modelling and rule discovery through evolving connectionist systems. Appl Soft Comput 6(3):307–322. doi:10.1016/j.asoc.2005.01.006 CrossRefGoogle Scholar
  36. Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach, second edn. Springer, New York, Secaucus, NJ, USAGoogle Scholar
  37. Kasabov N (2008) Adaptive modeling and discovery in bioinformatics: the evolving connectionist approach. Int J Intell Syst 23(5):545–555. doi:10.1002/int.20282 CrossRefGoogle Scholar
  38. Kasabov N (2010) To spike or not to spike: a probabilistic spiking neuron model. Neural Netw 23(1):16–19. doi:10.1016/j.neunet.2009.08.010
  39. Kasabov N, Hu Y (2010) Integrated optimisation method for personalised modelling and case studies for medical decision support. Int J Funct Inf Personal Med 3(3):236–256. doi:10.1504/IJFIPM.2010.039123 Google Scholar
  40. Kasabov K, Schliebs R, Kojima H (2011) Probabilistic computational neurogenetic modeling: From cognitive systems to alzheimer’s disease. IEEE Trans Auton Ment Dev 3(4):300–311. doi:10.1109/TAMD.2011.2159839 CrossRefGoogle Scholar
  41. Kasabov N (2012a) Evolving, probabilistic spiking neural networks and neurogenetic systems for spatio-and spectro-temporal data modelling and pattern recognition. Nat Intel INNS Mag 1(2):23–37Google Scholar
  42. Kasabov N (2012b) Neucube evospike architecture for spatio-temporal modelling and pattern recognition of brain signals. In: Mana, Schwenker, Trentin (eds) ANNPR, pp 225–243. Springer LNAI, Heidelberg, GermanyGoogle Scholar
  43. Kasabov N, Dhoble K, Nuntalid N, Indiveri G (2013) Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw. In printGoogle Scholar
  44. Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1-2):273–324. doi:10.1016/S0004-3702(97)00043-X MATHCrossRefGoogle Scholar
  45. Kohonen T (1997) Self-organizing maps, second edn. Springer, BerlinGoogle Scholar
  46. Komijani M, Lucas C, Araabi B, Kalhor A (2012) Introducing evolving takagisugeno method based on local least squares support vector machine models. Evol Syst 3:81–93. doi:10.1007/s12530-011-9043-0 CrossRefGoogle Scholar
  47. Lichtsteiner P, Delbruck T (2005) A 64x64 aer logarithmic temporal derivative silicon retina. Res Microelectron Electron 2:202–205. doi:10.1109/RME.2005.1542972 Google Scholar
  48. Lin CT, Lee CSG (1996) Neuro fuzzy systems. Prentice HallGoogle Scholar
  49. Loiselle S, Rouat J, Pressnitzer D, Thorpe S (2005) Exploration of rank order coding with spiking neural networks for speech recognition. In: IEEE International Joint Conference on Neural Networks, IJCNN ’ 05, vol 4, pp 2076–2080. doi:10.1109/IJCNN.2005.1556220
  50. Maass W, Natschläger T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14(11):2531–2560. doi:10.1162/089976602760407955 MATHCrossRefGoogle Scholar
  51. Meng Y, Jin Y, Yin J, Conforth M (2010) Human activity detection using spiking neural networks regulated by a gene regulatory network. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp 1–6. doi:10.1109/IJCNN.2010.5596340
  52. Ozawa S, Toh SL, Abe S, Pang S, Kasabov N (2005) Incremental learning of feature space and classifier for face recognition. Neural Netw 18(56):575–584. doi:10.1016/j.neunet.2005.06.016 CrossRefGoogle Scholar
  53. Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for online pattern classification systems. IEEE Trans Neural Netw 19(6):1061–1074. doi:10.1109/TNN.2007.2000059 CrossRefGoogle Scholar
  54. Perrinet L, Delorme A, Samuelides M, Thorpe SJ (2001) Networks of integrate-and-fire neuron using rank order coding A: how to implement spike time dependent Hebbian plasticity. Neurocomputing 38(40):817–822CrossRefGoogle Scholar
  55. Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3(2):213–225. doi:10.1162/neco.1991.3.2.213 MathSciNetCrossRefGoogle Scholar
  56. Rabiner L, Juang BH (1993) Fundamentals of speech recognition. Prentice-Hall, Inc., Upper Saddle RiverGoogle Scholar
  57. Riul A, de Sousa HC, Malmegrim RR, dos Santos DS, Carvalho ACPLF, Fonseca FJ, Oliveira ON, Mattoso LHC (2004) Wine classification by taste sensors made from ultra-thin films and using neural networks. Sens Actuators B Chem 98(1):77–82. doi:10.1016/j.snb.2003.09.025 Google Scholar
  58. Schaffer JD, Eshelman L, Offutt D (1991) Foundations of Genetic algorithms, chap. Spurious correlations and premature convergence in genetic algorithms. Morgan Kaufmann, San Mateo, pp 102–112Google Scholar
  59. Schliebs S, Defoin-Platel M, Kasabov N (2009a) Integrated feature and parameter optimization for an evolving spiking neural network. In: Köppen M, Kasabov NK, Coghill GG (eds) Advances in neuro-information processing, 15th international conference, Lecture Notes in Computer Science, vol 5506, pp 1229–1236. Springer, Heidelberg. doi:10.1007/978-3-642-02490-0_149
  60. Schliebs S, Defoin-Platel M, Worner S, Kasabov N (2009b) Integrated feature and parameter optimization for an evolving spiking neural network: exploring heterogeneous probabilistic models. Neural Netw 22(5–6):623–632. doi:10.1016/j.neunet.2009.06.038 CrossRefGoogle Scholar
  61. Schliebs S, Defoin-Platel M, Worner S, Kasabov N (2009c) Quantum-inspired feature and parameter optimisation of evolving spiking neural networks with a case study from ecological modeling. In: International joint conference on neural networks, IEEE—INNS - ENNS, vol 0, pp 2833–2840. IEEE Computer Society, Los Alamitos. doi:10.1109/IJCNN.2009.5179049
  62. Schliebs S (2010) Heterogeneous probabilistic models for optimisation and modelling of evolving spiking neural networks. Ph.D. thesis, Auckland University of Technology. http://hdl.handle.net/10292/963
  63. Schliebs S, Defoin-Platel M, Kasabov N (2010a) Analyzing the dynamics of the simultaneous feature and parameter optimization of an evolving spiking neural network. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp 1–8. doi:10.1109/IJCNN.2010.5596548
  64. Schliebs S, Defoin-Platel M, Kasabov N (2010b) On the probabilistic optimization of spiking neural networks. Int J Neural Syst 20(6):481–500CrossRefGoogle Scholar
  65. Schliebs S, Nuntalid N, Kasabov N (2010c) Towards spatio-temporal pattern recognition using evolving spiking neural networks. In: Wong K, Mendis B, Bouzerdoum A (eds) Neural information processing, theory and algorithms, Lecture Notes in Computer Science, vol 6443, pp 163–170. Springer Berlin. doi:10.1007/978-3-642-17537-4_21
  66. Schliebs S, Hamed HNA, Kasabov N (2011) A reservoir-based evolving spiking neural network for on-line spatio-temporal pattern learning and recognition. In: 18th international conference on neural information processing, no 7063 in LNCS, pp 160–168. Springer, Heidelberg/ShanghaiGoogle Scholar
  67. Schliebs S, Fiasché M, Kasabov N (2012) Constructing robust liquid state machines to process highly variable data streams. In: International Conference on Neural Networks (ICANN’12), LNCS, pp 604–611. Springer, Heidelberg/Lausanne, SwitzerlandGoogle Scholar
  68. Soltic S, Wysoski S, Kasabov N (2008) Evolving spiking neural networks for taste recognition. In: IEEE World congress on computational intelligence (WCCI), Hong Kong, pp 2091–2097. doi:10.1109/IJCNN.2008.4634085
  69. Soltic S, Kasabov N (2010) Knowledge extraction from evolving spiking neural networks with rank order population coding. Int J Neural Syst 20(6):437–445. doi:10.1142/S012906571000253X CrossRefGoogle Scholar
  70. Song Q, Kasabov N (2006) TWNFI—a transductive neuro-fuzzy inference system with weighted data normalization for personalized modeling. Neural Netw 19(10):1591–1596. doi:10.1016/j.neunet.2006.05.028 MATHCrossRefGoogle Scholar
  71. Swope J (2012) ARTdECOS, adaptive evolving connectionist model and application to heart rate variability. Evol Syst 3:95–109. doi:10.1007/s12530-012-9049-2 CrossRefGoogle Scholar
  72. Thorpe SJ (1990) Spike arrival times: a highly efficient coding scheme for neural networks. In: Eckmiller R, Hartmann G, Hauske G (eds) International Conference on Parallel Processing in Neural Systems, pp 91–94. Elsevier, North-HollandGoogle Scholar
  73. Thorpe SJ, Gautrais J (1996) Rapid visual processing using spike asynchrony. In: Advances in Neural Information Processing Systems 9, NIPS, pp 901–907. MIT Press, DenverGoogle Scholar
  74. Thorpe SJ, Fize D, Marlot C (1996) Speed of processing in the human visual system. Nature 381:520–522. doi:10.1038/381520a0 CrossRefGoogle Scholar
  75. Thorpe SJ (1997) How can the human visual system process a natural scene in under 150 ms? On the role of asynchronous spike propagation. In: ESANN. D-Facto publicGoogle Scholar
  76. Thorpe SJ, Gautrais J (1998) Rank order coding. In: CNS ’97: Proceedings of the 6th annual conference on Computational neuroscience: trends in research, 1998, pp 113–118. Plenum Press, New YorkGoogle Scholar
  77. Thorpe SJ, Delorme A, van Rullen R (2001) Spike-based strategies for rapid processing. Neural Netw 14(6–7):715–725CrossRefGoogle Scholar
  78. Thorpe SJ, Guyonneau R, Guilbaud N, Allegraud JM, VanRullen R (2004) SpikeNet: real-time visual processing with one spike per neuron. Neurocomputing 58(60):857–864. doi:10.1016/j.neucom.2004.01.138 CrossRefGoogle Scholar
  79. Van Rullen R, Gautrais J, Delorme A, Thorpe S (1998) Face processing using one spike per neurone. Biosyst Eng 48(1–3):229–239CrossRefGoogle Scholar
  80. Van Rullen R, Thorpe SJ (2001) Rate coding versus temporal order coding: What the retinal ganglion cells tell the visual cortex. Neural Comput 13(6):1255–1283. doi:10.1162/08997660152002852 MATHCrossRefGoogle Scholar
  81. Watts M (2009) A decade of Kasabov’s evolving connectionist systems: a review. IEEE Trans Syst Man Cybern C Appl Rev 39(3):253–269. doi:10.1109/TSMCC.2008.2012254 CrossRefGoogle Scholar
  82. Wysoski SG, Benuskova L, Kasabov N (2006a) On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition. In: Artificial Neural Networks ICANN 2006, pp 61–70. Springer, Berlin. doi:10.1007/11840817_7
  83. Wysoski SG, Benuskova L, Kasabov NK (2006b) Adaptive learning procedure for a network of spiking neurons and visual pattern recognition. In: Advanced Concepts for Intelligent Vision Systems, pp 1133–1142. Springer, Berlin. doi:10.1007/11864349_103
  84. Wysoski SG, Benuskova L, Kasabov N (2007) Text-independent speaker authentication with spiking neural networks. In: ICANN (2), vol 4669/2007, pp 758–767. Springer, Berlin. doi:10.1007/978-3-540-74695-9_78
  85. Wysoski SG (2008) Evolving spiking neural networks for adaptive audiovisual pattern recognition. Ph.D. thesis, Auckland University of Technology. http://hdl.handle.net/10292/390
  86. Wysoski SG, Benuskova L, Kasabov N (2008a) Adaptive spiking neural networks for audiovisual pattern recognition. In: Neural Information Processing: 14th International Conference, ICONIP 2007, pp 406–415. Springer, Berlin. doi:10.1007/978-3-540-69162-4_42
  87. Wysoski SG, Benuskova L, Kasabov N (2008b) Fast and adaptive network of spiking neurons for multi-view visual pattern recognition. Neurocomputing 71(13–15):2563–2575. doi:10.1016/j.neucom.2007.12.038 CrossRefGoogle Scholar
  88. Wysoski S, Benuskova L, Kasabov N (2010a) Brain-like evolving spiking neural networks for multimodal information processing. In: Hanazawa A, Miki T, Horio K (eds) Brain-inspired information technology, studies in computational intelligence, vol 266, pp 15–27. Springer, Berlin. doi:10.1007/978-3-642-04025-2_3
  89. Wysoski SG, Benuskova L, Kasabov N (2010b) Evolving spiking neural networks for audiovisual information processing. Neural Netw 23(7):819–835. doi:10.1016/j.neunet.2010.04.009 CrossRefGoogle Scholar
  90. Zuppicich A, Soltic S (2009) FPGA implementation of an evolving spiking neural network. In: Köppen M, Kasabov NK, Coghill GG (eds) Advances in Neuro-Information Processing, 15th international conference, Lecture Notes in Computer Science, vol 5506, pp 1129–1136. Springer, Heidelberg. doi:10.1007/978-3-642-02490-0_137

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.KEDRIAuckland University of TechnologyAucklandNew Zealand
  2. 2.Institute for Neuroinformatics, ETH/UZHZurichSwitzerland

Personalised recommendations