Skip to main content

Advertisement

Log in

A hierarchical taxonomic survey of spiking neural networks

  • Regular paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

Artificial Neural Network (ANN) has served as an important pillar of machine learning which played a crucial role in fueling the robust artificial intelligence (AI) revival experienced in the last few years. Inspired by the biological brain architecture of living things, ANN has shown widespread success in pattern recognition, data analysis and classification tasks. Among the many models of neural networks conceptualized and developed over the years, the Spiking Neural Network (SNN) which was initiated in 1996 has shown great promise in the current push towards compact embedded AI. By combining both spatial and temporal information as features in the training and testing process, many inherent shortcomings of traditional ANNs can be overcome. With temporal features and event-driven updating of the network, SNNs hold the potential of improving computational and energy efficiency. In SNN, the most basic signal carrier element is the spike, bringing about a revolution in neural network weights updating compared to traditional methods that are widely applied in ANNs. In literature, there have been numerous SNN weights updating algorithms developed in recent years. With the active and dynamic research work on SNN, a consolidation of the state-of-the-art SNN research is beneficial. This paper is aimed at reviewing and surveying the current status of research pertaining to SNN, in particular highlighting the various novel SNN training techniques coupled with an objective comparison of the techniques. SNN applications and associated neuromorphic hardware systems are also covered in this survey, with some thoughts on the challenges in developing new SNN training algorithms and discussion on potential future research trends are presented in this survey.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. All the data in this figure was exported from Scopus by keyword (artificial neural network/spiking neural network) search on the scope of article title, abstract and keywords.

References

  1. Andreeva N, Ryndin E, Gerasimova M (2020) Memristive logic design of multifunctional spiking neural network with unsupervised learning. BioNanoScience 10(4):824–833

    Article  Google Scholar 

  2. Anwani N, Rajendran B (2015) NormAD-normalized approximate descent based supervised learning rule for spiking neurons. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 IEEE

  3. Auge D, Hille J, Mueller E, Knoll A (2021) A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters, 1–18

  4. Azghadi MR, Iannella N, Al-Sarawi SF, Indiveri G, Abbott D (2014) Spike-based synaptic plasticity in silicon: Design, implementation, application, and challenges. Proc IEEE 102(5):717–737

    Article  Google Scholar 

  5. Balaji A, Corradi F, Das A, Pande S, Schaafsma S, Catthoor F (2018) Power-accuracy trade-offs for heartbeat classification on neural networks hardware. J Low Power Electr 14(4):508–519

    Article  Google Scholar 

  6. Basu A, Acharya J, Karnik T, Liu H, Li H, Seo J-S, Song C (2018) Low-power, adaptive neuromorphic systems: Recent progress and future directions. IEEE J Emerg Sel Top Circ Syst 8(1):6–27

    Article  Google Scholar 

  7. Benjamin BV, Gao P, McQuinn E, Choudhary S, Chandrasekaran AR, Bussat J-M, Alvarez-Icaza R, Arthur JV, Merolla PA, Boahen K (2014) Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proc IEEE 102(5):699–716

    Article  Google Scholar 

  8. Bohte SM, Kok JN, La Poutre H (2002) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4):17–37

    Article  MATH  Google Scholar 

  9. Bouvier M, Valentian A, Mesquida T, Rummens F, Reyboz M, Vianello E, Beigne E (2019) Spiking neural networks hardware implementations and challenges: A survey. ACM J Emerg Technol Comput Syst (JETC) 15(2):1–35

    Article  Google Scholar 

  10. Bush D, Burgess N (2019) Neural oscillations: Phase coding in the absence of rhythmicity. Curr Biol 29(2):55–57

    Article  Google Scholar 

  11. Cao Y, Chen Y, Khosla D (2015) Spiking deep convolutional neural networks for energy-efficient object recognition. Int J Comput Vision 113(1):54–66

    Article  MathSciNet  Google Scholar 

  12. Cawley S, Morgan F, McGinley B, Pande S, McDaid L, Carrillo S, Harkin J (2011) Hardware spiking neural network prototyping and application. Genet Program Evolvable Mach 12(3):257–280

    Article  Google Scholar 

  13. Cheng X, Hao Y, Xu J, Xu B (2020) LISNN: Improving spiking neural networks with lateral interactions for robust object recognition. In: IJCAI, pp 1519–1525

  14. Cheng L, Liu Y, Hou Z-G, Tan M, Du D, Fei M (2019) A rapid spiking neural network approach with an application on hand gesture recognition. IEEE Transactions on Cognitive and Developmental Systems

  15. Davidson S, Furber SB (2021) Comparison of artificial and spiking neural networks on digital hardware. Front Neurosci 15:345

    Article  Google Scholar 

  16. Davies M, Srinivasa N, Lin T-H, Chinya G, Cao Y, Choday SH, Dimou G, Joshi P, Imam N, Jain S et al (2018) Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82–99

    Article  Google Scholar 

  17. Delorme A, Thorpe SJ (2001) Face identification using one spike per neuron: Resistance to image degradations. Neural Netw 14(6–7):795–803

    Article  Google Scholar 

  18. Diehl PU, Zarrella G, Cassidy A, Pedroni BU, Neftci E (2016) Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. In: 2016 IEEE International Conference on Rebooting Computing (ICRC), pp. 1–8 IEEE

  19. Esser SK, Merolla PA, Arthur JV, Cassidy AS, Appuswamy R, Andreopoulos A, Berg DJ, McKinstry JL, Melano T, Barch DR et al (2016) Convolutional networks for fast, energy-efficient neuromorphic computing. Proc Natl Acad Sci 113(41):11441–11446

    Article  Google Scholar 

  20. Fang W, Yu Z, Chen Y, Huang T, Masquelier T, Tian Y (2021) Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems 34

  21. Fang W, Yu Z, Chen Y, Masquelier T, Huang T, Tian Y (2021) Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2661–2671

  22. Farries MA, Fairhall AL (2007) Reinforcement learning with modulated spike timing-dependent synaptic plasticity. J Neurophysiol 98(6):3648–3665

    Article  Google Scholar 

  23. Florian RV (2012) The chronotron: A neuron that learns to fire temporally precise spike patterns. PLoS ONE 7(8):40233

    Article  Google Scholar 

  24. Furber S (2016) Large-scale neuromorphic computing systems. J Neural Eng 13(5):051001

    Article  Google Scholar 

  25. Furber SB, Lester DR, Plana LA, Garside JD, Painkras E, Temple S, Brown AD (2012) Overview of the spinnaker system architecture. IEEE Trans Comput 62(12):2454–2467

    Article  MathSciNet  Google Scholar 

  26. Ghosh-Dastidar S, Adeli H (2007) Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Int Comput-Aided Eng 14(3):187–212

    Article  Google Scholar 

  27. Ghosh-Dastidar S, Adeli H (2009) Spiking neural networks. Int J Neural Syst 19(04):295–308

    Article  Google Scholar 

  28. Guo W, Fouda ME, Eltawil AM, Salama KN (2021) Neural coding in spiking neural networks: A comparative study for robust neuromorphic systems. Front Neurosci 15:212

    Google Scholar 

  29. Huh D, Sejnowski TJ (2018) Gradient descent for spiking neural networks. In: Advances in Neural Information Processing Systems, pp. 1433–1443

  30. Hunsberger E, Eliasmith C (2015) Spiking deep networks with LIF neurons. arXiv preprint arXiv:1510.08829

  31. Jang H, Simeone O, Gardner B, Gruning A (2019) An introduction to probabilistic spiking neural networks: Probabilistic models, learning rules, and applications. IEEE Signal Process Mag 36(6):64–77

    Article  Google Scholar 

  32. Kasabov NK (2014) NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw 52:62–76

    Article  Google Scholar 

  33. Kasabov N, Dhoble K, Nuntalid N, Indiveri G (2013) Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition. Neural Netw 41:188–201

    Article  Google Scholar 

  34. Kim Y, Panda P (2021) Optimizing deeper spiking neural networks for dynamic vision sensing. Neural Netw 144:686–698

    Article  Google Scholar 

  35. Kulkarni SR, Rajendran B (2018) Spiking neural networks for handwritten digit recognition-supervised learning and network optimization. Neural Netw 103:118–127

    Article  Google Scholar 

  36. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  37. Lee JH, Delbruck T, Pfeiffer M (2016) Training deep spiking neural networks using backpropagation. Front Neurosci 10:508

    Google Scholar 

  38. Lee WW, Kukreja SL, Thakor NV (2016) CONE: Convex-optimized-synaptic efficacies for temporally precise spike mapping. IEEE Trans Neural Networks Learning Syst 28(4):849–861

    Article  Google Scholar 

  39. Lee C, Sarwar SS, Panda P, Srinivasan G, Roy K (2020) Enabling spike-based backpropagation for training deep neural network architectures. Frontiers in neuroscience, 119

  40. Lobo JL, Del Ser J, Bifet A, Kasabov N (2020) Spiking neural networks and online learning: An overview and perspectives. Neural Netw 121:88–100

    Article  Google Scholar 

  41. Maass W (1997) Networks of spiking neurons: The third generation of neural network models. Neural Netw 10(9):1659–1671

    Article  Google Scholar 

  42. Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78(10):1629–1636

    Article  Google Scholar 

  43. Meftah B, Lezoray O, Benyettou A (2010) Segmentation and edge detection based on spiking neural network model. Neural Process Lett 32(2):131–146

    Article  Google Scholar 

  44. Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, Jackson BL, Imam N, Guo C, Nakamura Y et al (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Sci 345(6197):668–673

    Article  Google Scholar 

  45. Meuth R, Lim M-H, Ong Y-S, Wunsch DC (2009) A proposition on memes and meta-memes in computing for higher-order learning. Memetic Computing 1(2):85–100

    Article  Google Scholar 

  46. Mohemmed A, Schliebs S, Matsuda S, Kasabov N (2012) SPAN: Spike pattern association neuron for learning spatio-temporal spike patterns. Int J Neural Syst 22(04):1250012

    Article  Google Scholar 

  47. Moradi S, Qiao N, Stefanini F, Indiveri G (2017) A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Trans Biomed Circuits Syst 12(1):106–122

    Article  Google Scholar 

  48. Mozafari M, Kheradpisheh SR, Masquelier T, Nowzari-Dalini A, Ganjtabesh M (2018) First-spike-based visual categorization using reward-modulated STDP. IEEE Trans Neural Networks Learning Syst 29(12):6178–6190

    Article  Google Scholar 

  49. Mozafari M, Ganjtabesh M, Nowzari-Dalini A, Thorpe SJ, Masquelier T (2018) Combining STDP and reward-modulated STDP in deep convolutional spiking neural networks for digit recognition. arXiv preprint 1 (2018) arXiv:1804.00227

  50. Neftci EO, Mostafa H, Zenke F (2019) Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Process Mag 36(6):51–63

    Article  Google Scholar 

  51. Neil D, Liu S-C (2016) Effective sensor fusion with event-based sensors and deep network architectures. In: 2016 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2282–2285. IEEE

  52. Nichols E, McDaid L, Siddique N (2010) Case study on a self-organizing spiking neural network for robot navigation. Int J Neural Syst 20(06):501–508

    Article  Google Scholar 

  53. Obo T, Kubota N, Taniguchi K, Sawayama T (2011) Human localization based on spiking neural network in intelligent sensor networks. In: 2011 IEEE Workshop on Robotic Intelligence In Informationally Structured Space, pp 125–130. IEEE

  54. O’Connor P, Gavves E, Welling M (2019) Training a spiking neural network with equilibrium propagation. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1516–1523

  55. O’Connor P, Welling M (2016) Deep spiking networks. arXiv preprint arXiv:1602.08323

  56. Orhan AE, Ma WJ (2015) Neural population coding of multiple stimuli. J Neurosci 35(9):3825–3841

    Article  Google Scholar 

  57. Pei J, Deng L, Song S, Zhao M, Zhang Y, Wu S, Wang G, Zou Z, Wu Z, He W et al (2019) Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572(7767):106–111

    Article  Google Scholar 

  58. Perez-Nieves N, Goodman D (2021) Sparse spiking gradient descent. Adv Neural Inf Process Syst 34:11795–11808

    Google Scholar 

  59. Pfister J-P, Toyoizumi T, Barber D, Gerstner W (2006) Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Comput 18(6):1318–1348

    Article  MathSciNet  MATH  Google Scholar 

  60. Pollack JB (1990) Recursive distributed representations. Artif Intell 46(1–2):77–105

    Article  Google Scholar 

  61. Ponulak F, Kasiński A (2010) Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classification, and spike shifting. Neural Comput 22(2):467–510

    Article  MathSciNet  MATH  Google Scholar 

  62. Rathi N, Roy K (2020) DIET-SNN: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008.03658

  63. Reid D, Hussain AJ, Tawfik H (2014) Financial time series prediction using spiking neural networks. PLoS ONE 9(8):103656

    Article  Google Scholar 

  64. Roggen D, Hofmann S, Thoma Y, Floreano D (2003) Hardware spiking neural network with run-time reconfigurable connectivity in an autonomous robot. In: NASA/DoD Conference on Evolvable Hardware, 2003. Proceedings., pp 189–198. IEEE

  65. Roy K, Jaiswal A, Panda P (2019) Towards spike-based machine intelligence with neuromorphic computing. Nature 575(7784):607–617

    Article  Google Scholar 

  66. Rueckauer B, Lungu I-A, Hu Y, Pfeiffer M, Liu S-C (2017) Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front Neurosci 11:682

    Article  Google Scholar 

  67. Rumelhart DE, Hinton GE, Williams RJ (1985) Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science

  68. Scellier B, Bengio Y (2017) Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Front Comput Neurosci 11:24

    Article  Google Scholar 

  69. Schemmel J, Billaudelle S, Dauer P, Weis J (2020) Accelerated analog neuromorphic computing. arXiv preprint arXiv:2003.11996

  70. Schemmel J, Grubl A, Meier K, Mueller E (2006) Implementing synaptic plasticity in a VLSI spiking neural network model. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings, pp 1–6. IEEE

  71. Schrauwen B, Van Campenhout J (2003) BSA, a fast and accurate spike train encoding scheme. In: Proceedings of the International Joint Conference on Neural Networks, 4, 2825–2830 (2003). IEEE

  72. Schrauwen B, Van Campenhout J (2004) Extending SpikeProp. In: 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), 1: 471–475. IEEE

  73. Schuman CD, Potok TE, Patton RM, Birdwell JD, Dean ME, Rose GS, Plank JS (2017) A survey of neuromorphic computing and neural networks in hardware. arXiv preprint arXiv:1705.06963

  74. Schuman CD, Kulkarni SR, Parsa M, Mitchell JP, Kay B et al (2022) Opportunities for neuromorphic computing algorithms and applications. Nat Comput Sci 2(1):10–19

    Article  Google Scholar 

  75. Sengupta A, Ye Y, Wang R, Liu C, Roy K (2019) Going deeper in spiking neural networks: VGG and residual architectures. Front Neurosci 13:95

    Article  Google Scholar 

  76. Soltic S, Wysoski SG, Kasabov NK (2008) Evolving spiking neural networks for taste recognition. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp 2091–2097. IEEE

  77. Stewart K, Neftci E (2022) Meta-learning spiking neural networks with surrogate gradient descent. arXiv preprint arXiv:2201.10777

  78. Taherkhani A, Belatreche A, Li Y, Maguire LP (2015) DL-ReSuMe: A delay learning-based remote supervised method for spiking neurons. IEEE Trans Neural Networks Learning Syst 26(12):3137–3149

    Article  MathSciNet  Google Scholar 

  79. Taherkhani A, Belatreche A, Li Y, Cosma G, Maguire LP, McGinnity TM (2020) A review of learning in biologically plausible spiking neural networks. Neural Netw 122:253–272

    Article  Google Scholar 

  80. Tapson J, van Schaik A (2013) Learning the pseudoinverse solution to network weights. Neural Netw 45:94–100

    Article  MATH  Google Scholar 

  81. Tapson JC, Cohen GK, Afshar S, Stiefel KM, Buskila Y, Hamilton TJ, van Schaik A (2013) Synthesis of neural networks for spatio-temporal spike pattern recognition and processing. Front Neurosci 7:153

    Article  Google Scholar 

  82. Tavanaei A, Ghodrati M, Kheradpisheh SR, Masquelier T, Maida A (2019) Deep learning in spiking neural networks. Neural Netw 111:47–63

    Article  Google Scholar 

  83. Thiele JC, Bichler O, Dupret A (2019) SpikeGrad: An ANN-equivalent computation model for implementing backpropagation with spikes. arXiv preprint arXiv:1906.00851

  84. Wang X, Hou Z-G, Zou A, Tan M, Cheng L (2008) A behavior controller based on spiking neural networks for mobile robots. Neurocomputing 71(4–6):655–666

    Article  Google Scholar 

  85. Wang J, Belatreche A, Maguire LP, McGinnity TM (2015) SpikeTemp: An enhanced rank-order-based learning approach for spiking neural networks with adaptive structure. IEEE Trans Neural Networks Learning Syst 28(1):30–43

    Article  Google Scholar 

  86. Wang X, Lin X, Dang X (2020) Supervised learning in spiking neural networks: A review of algorithms and evaluations. Neural Netw 125:258–280

    Article  Google Scholar 

  87. Wang J, Belatreche A, Maguire LP, McGinnity TM (2015) SpikeComp: An evolving spiking neural network with adaptive compact structure for pattern classification. In: International Conference on Neural Information Processing, pp. 259–267. Springer

  88. Wu Q, McGinnity TM, Maguire LP, Belatreche A, Glackin B (2008) Processing visual stimuli using hierarchical spiking neural networks. Neurocomputing 71(10–12):2055–2068

    Article  Google Scholar 

  89. Wu Y, Deng L, Li G, Zhu J, Shi L (2018) Spatio-temporal backpropagation for training high-performance spiking neural networks. Front Neurosci 12:331

    Article  Google Scholar 

  90. Wu Y, Deng L, Li G, Zhu J, Xie Y, Shi L (2019) Direct training for spiking neural networks: Faster, larger, better. In: Proceedings of the AAAI Conference on Artificial Intelligence 33: 1311–1318

  91. Wysoski SG, Benuskova L, Kasabov N (2006) Adaptive learning procedure for a network of spiking neurons and visual pattern recognition. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp 1133–1142. Springer

  92. Wysoski SG, Benuskova L, Kasabov N (2007) Adaptive spiking neural networks for audiovisual pattern recognition. In: International Conference on Neural Information Processing, pp. 406–415. Springer

  93. Wysoski SG, Benuskova L, Kasabov N (2007) Text-independent speaker authentication with spiking neural networks. In: International Conference on Artificial Neural Networks, pp 758–767. Springer

  94. Wysoski SG, Benuskova L, Kasabov N (2010) Evolving spiking neural networks for audiovisual information processing. Neural Netw 23(7):819–835

    Article  Google Scholar 

  95. Xie X, Qu H, Yi Z, Kurths J (2016) Efficient training of supervised spiking neural network via accurate synaptic-efficiency adjustment method. IEEE Trans Neural Networks Learning Syst 28(6):1411–1424

    Article  Google Scholar 

  96. Yin S, Venkataramanaiah SK, Chen GK, Krishnamurthy R, Cao Y, Chakrabarti C, Seo J-s (2017) Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations. In: 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp 1–5. IEEE

  97. Young AR, Dean ME, Plank JS, Rose GS (2019) A review of spiking neuromorphic hardware communication systems. IEEE Access 7:135606–135620

    Article  Google Scholar 

  98. Yu Q, Tang H, Tan KC, Li H (2013) Precise-spike-driven synaptic plasticity: Learning hetero-association of spatiotemporal spike patterns. PLoS ONE 8(11):78318

    Article  Google Scholar 

  99. Yu Q, Ma C, Song S, Zhang G, Dang J, Tan KC (2021) Constructing accurate and efficient deep spiking neural networks with double-threshold and augmented schemes. IEEE Transactions on Neural Networks and Learning Systems

  100. Zenke F, Ganguli S (2018) SuperSpike: Supervised learning in multilayer spiking neural networks. Neural Comput 30(6):1514–1541

    Article  MathSciNet  MATH  Google Scholar 

  101. Zhao F, Zeng Y, Xu B (2018) A brain-inspired decision-making spiking neural network and its application in unmanned aerial vehicle. Front Neurorobot 12:56

    Article  Google Scholar 

  102. Zheng H, Wu Y, Deng L, Hu Y, Li G (2020) Going deeper with directly-trained larger spiking neural networks. arXiv preprint arXiv:2011.05280

  103. Zimmer R, Pellegrini T, Singh SF, Masquelier T (2019) Technical report: Supervised training of convolutional spiking neural networks with PyTorch. arXiv preprint arXiv:1911.10124

  104. Zou X, Hwu T, Krichmar J, Neftci E (2020) Terrain classification with a reservoir-based network of spiking neurons. In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS), pp 1–5. IEEE

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Siqi Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Cheng, T.H. & Lim, M.H. A hierarchical taxonomic survey of spiking neural networks. Memetic Comp. 14, 335–354 (2022). https://doi.org/10.1007/s12293-022-00373-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-022-00373-w

Keywords

Navigation