Skip to main content
SpringerLink
Log in

The Spiking Rates Inspired Encoder and Decoder for Spiking Neural Networks: An Illustration of Hand Gesture Recognition

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The spiking neural network (SNN) is the third generation of artificial neural networks. The transmission and expression of information in SNN are performed by spike trains, making the SNN have the advantages of high calculation speed and low power consumption. Recently, researchers have employed the SNN to recognize surface electromyography (sEMG) signals, but problems are still left. The sEMG encoders may cause information loss, and the network decoders may cause poor training performance. The strength of the neuron stimulated can be expressed by the frequency of the input or output spikes (namely firing rate). Inspired by the firing rate principle, we proposed the smoothed frequency-domain decomposition encoder, which converts the sEMG to spike trains. Furthermore, we also proposed the network efferent energy decoder, which converts the network output to recognizing results. The employed SNN is a three-layer fully-connected network trained by the grey wolf optimizer. The proposed methods are verified by a hand gestures recognition task. A total of 11 subjects participated in the experiment, and sEMG signals were acquired from five commonly used hand gestures by three sEMG sensors. The results indicate that the loss function can be reduced to below 0.4, and the average gesture recognizing accuracy is 91.21%. These results show the potential of using the proposed methods for the actual prosthesis. In the future, we will optimize the SNN training method to improve the training speed and stability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Source Code

The code and acquired sEMG data of this word can be found at here.

References

  1. Yahya U, Senanayake SMNA, Naim AG. A database-driven neural computing framework for classification of vertical jump patterns of healthy female netballers using 3D kinematics-EMG features. Neural Comput Applic. 2020;32(5):1481–500.

    Article  Google Scholar 

  2. Lu Z, Stampas A, Francisco GE, Zhou P. Offline and online myoelectric pattern recognition analysis and real-time control of a robotic hand after spinal cord injury. J Neural Eng. 2019;16(3):1–10.

    Article  Google Scholar 

  3. Chen C, Ma S, Sheng X, Farina D, Zhu X. Adaptive real-time identification of motor unit discharges from non-stationary high-density surface electromyographic signals. IEEE Trans Biomed Eng. 2020;67(12):3501–9.

    Article  Google Scholar 

  4. Xu H, Xiong A. Advances and disturbances in sEMG-based intentions and movements recognition: a review. IEEE Sens J. 2021;21(12):13019–28.

    Article  Google Scholar 

  5. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958;65(6):386–408.

    Article  Google Scholar 

  6. Rumelhart DE, McClelland JL. Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations. MIT Press; 1986.

  7. Maass W. Networks of spiking neurons: The third generation of neural network models. Neural Netw. 1997;10(9):1659–71.

    Article  Google Scholar 

  8. Roy K, Jaiswal A, Panda P. Towards spike-based machine intelligence with neuromorphic computing. Nature. 2019;575(7784):607–17.

    Article  Google Scholar 

  9. Pei J, Deng L, Song S, Zhao M, Zhang Y, Wu S, et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature. 2019;572(7767):106–11.

    Article  Google Scholar 

  10. Karnam NK, Turlapaty AC, Dubey SR, Gokaraju B. Classification of sEMG signals of hand gestures based on energy features. Biomed Signal Process Control. 2021;70:1–8.

    Article  Google Scholar 

  11. Tam S, Boukadoum M, Campeau-Lecours A, Gosselin B. A fully embedded adaptive real-time hand gesture classifier leveraging HD-sEMG and deep learning. IEEE Trans Biomed Circuits Syst. 2020;14(2):232–43.

    Article  Google Scholar 

  12. Cheng L, Liu Y, Hou ZG, Tan M, Du D, Fei M. A rapid spiking neural network approach with an application on hand gesture recognition. IEEE Transactions on Cognitive and Developmental Systems. 2021;13(1):151–61.

    Article  Google Scholar 

  13. Donati E, Payvand M, Risi N, Krause R, Indiveri G. Discrimination of EMG signals using a neuromorphic implementation of a spiking neural network. IEEE Trans Biomed Circuits Syst. 2019;13(5):795–803.

    Article  Google Scholar 

  14. Ceolini E, Frenkel C, Shrestha SB, Taverni G, Khacef L, Payvand M, et al. Hand-gesture recognition based on EMG and event-based camera sensor fusion: a benchmark in neuromorphic computing. Front Neurosci. 2020;14,:1–15.

  15. Behrenbeck J, Tayeb Z, Bhiri C, Richter C, Rhodes O, Kasabov N, et al. Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware. J Neural Eng. 2019;16(2):1–13.

    Article  Google Scholar 

  16. Ma Y, Chen B, Ren P, Zheng N, Indiveri G, Donati E. EMG-based gestures classification using a mixed-signal neuromorphic processing system. IEEE J Emerging Sel Top Circuits Syst. 2020;10(4):578–87.

    Article  Google Scholar 

  17. Lobov SA, Chernyshov AV, Krilova NP, Shamshin MO, Kazantsev VB. Competitive learning in a spiking neural network: towards an intelligent pattern classifier. Sensors. 2020;20(2):1–14.

    Article  Google Scholar 

  18. Lin ZT, Ma D, Meng JY, Chen LN. Relative ordering learning in spiking neural network for pattern recognition. Neurocomputing. 2018;275:94–106.

    Article  Google Scholar 

  19. Adrian ED, Zotterman Y. The impulses produced by sensory nerve-endings: Part II. The response of a single end-organ. J Physiol. 1926;61(2):151–71.

  20. Orchard G, Meyer C, Etienne-Cummings R, Posch C, Thakor N, Benosman R. HFirst: A temporal approach to object recognition. IEEE Trans Pattern Anal Mach Intell. 2015;37(10):2028–40.

    Article  Google Scholar 

  21. Liu Y, Cheng L. Spiking-neural-network based Fugl-Meyer hand gesture recognition for wearable hand rehabilitation robot. In: 2018 International Joint Conference on Neural Networks. IEEE International Joint Conference on Neural Networks (IJCNN); 2018. p. 1–6.

  22. Menon R, DCaterina G, Lakany H, Petropoulakis L, Conway BA, Soraghan JJ. Study on interaction between temporal and spatial information in classification of EMG signals for myoelectric prostheses. IEEE Trans Neural Syst Rehabil Eng. 2017;25(10):1832–1842.

  23. Hodgkin AL, Huxley AF. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol. 1952;117(4):500–44.

    Article  Google Scholar 

  24. Izhikevich EM. Simple model of spiking neurons. IEEE Trans Neural Netw. 2003;14(6):1569–72.

    Article  MathSciNet  Google Scholar 

  25. Delorme A, Gautrais J, van Rullen R, Thorpe S. SpikeNET: A simulator for modeling large networks of integrate and fire neurons. Neurocomputing. 1999;26–27:989–96.

    Article  Google Scholar 

  26. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 1991;4:251–7.

    Article  Google Scholar 

  27. Albu F, Mateescu A, Dumitriu N. Architecture selection for a multilayer feedforward network. In: International Conference on Microelectronics and Computer Science. 1997. p. 131–134.

  28. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R. Improving neural networks by preventing co-adaptation of feature detectors. ArXiv, 2012.

  29. Lee C, Sarwar SS, Panda P, Srinivasan G, Roy K. Enabling spike-based backpropagation for training deep neural network architectures. Front Neurosci. 2020;14,:1–22.

  30. Mirjalili S, Mirjalili SM, Lewis A. Grey wolf optimizer. Adv Eng Softw. 2014;69:46–61.

    Article  Google Scholar 

  31. Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci. 1997;55(1):119–39.

    Article  MathSciNet  MATH  Google Scholar 

  32. Christianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods. Press: Cambridge Univ; 2000.

    Book  Google Scholar 

  33. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature. 1986;323(6088):533–6.

    Article  MATH  Google Scholar 

  34. Lemaire E, Miramond B, Bilavarn S, Saoud H, Abderrahmane N. Synaptic activity and hardware footprint of spiking neural networks in digital neuromorphic systems. ACM Trans Embed Comput Syst. 2022.

Download references

Acknowledgements

The author would like to thank H. Cheng, Y. Lv, J. Xue and Y. Wang for their help in the method designing. The authors would also like to thank all of the subjects for their cooperation during the experiments.

Funding

This work was supported in part by the National Natural Science Foundation of China (Key Program) under Grant 11932013, in part by Tianjin Natural Science Foundation for Distinguished Young Scholars under Grant 18JCJQJC46100, in part by Tianjin Research Innovation Project for Postgraduate Students under Grant 2020YJSB003 and 2021YJSB018.

Author information

Authors and Affiliations

  1. College of Artificial Intelligence, Nankai University, No.38 Tongyan Road, Tianjin, 300350, China

    Yikang Yang, Jia Ren & Feng Duan

Authors
  1. Yikang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Duan.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Human and Animal Rights

The sEMG acquired experiments involved in this work are not harmful to the subjects. All subjects participated in the experiments voluntarily, and they could withdraw at any time during the experiments.

Conflicts of Interest

All of the authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Ren, J. & Duan, F. The Spiking Rates Inspired Encoder and Decoder for Spiking Neural Networks: An Illustration of Hand Gesture Recognition. Cogn Comput 15, 1257–1272 (2023). https://doi.org/10.1007/s12559-022-10027-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-022-10027-1

Keywords

Mathematics Subject Classification

Search

Navigation