Skip to main content
SpringerLink
Log in

STGNN-LMR: A Spatial–Temporal Graph Neural Network Approach Based on sEMG Lower Limb Motion Recognition

  • Research Article
  • Published:
Journal of Bionic Engineering Aims and scope Submit manuscript

Abstract

Lower limb motion recognition techniques commonly employ Surface Electromyographic Signal (sEMG) as input and apply a machine learning classifier or Back Propagation Neural Network (BPNN) for classification. However, this artificial feature engineering technique is not generalizable to similar tasks and is heavily reliant on the researcher’s subject expertise. In contrast, neural networks such as Convolutional Neural Network (CNN) and Long Short-term Memory Neural Network (LSTM) can automatically extract features, providing a more generalized and adaptable approach to lower limb motion recognition. Although this approach overcomes the limitations of human feature engineering, it may ignore the potential correlation among the sEMG channels. This paper proposes a spatial–temporal graph neural network model, STGNN-LMR, designed to address the problem of recognizing lower limb motion from multi-channel sEMG. STGNN-LMR transforms multi-channel sEMG into a graph structure and uses graph learning to model spatial–temporal features. An 8-channel sEMG dataset is constructed for the experimental stage, and the results show that the STGNN-LMR model achieves a recognition accuracy of 99.71%. Moreover, this paper simulates two unexpected scenarios, including sEMG sensors affected by sweat noise and sudden failure, and evaluates the testing results using hypothesis testing. According to the experimental results, the STGNN-LMR model exhibits a significant advantage over the control models in noise scenarios and failure scenarios. These experimental results confirm the effectiveness of the STGNN-LMR model for addressing the challenges associated with sEMG-based lower limb motion recognition in practical scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data are available from the corresponding author on reasonable request.

References

  1. Ajiboye, A. B., & Weir, R. F. (2009). Muscle synergies as a predictive framework for the EMG patterns of new hand postures. Journal of Neural Engineering, 6(3), 036004. https://doi.org/10.1088/1741-2560/6/3/036004

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  2. Chen, B., Ma, H., Qin, L. Y., Gao, F., Chan, K. M., Law, S. W., Qin, L., & Liao, W. H. (2016). Recent developments and challenges of lower extremity exoskeletons. Journal of Orthopaedic Translation, 5, 26–37. https://doi.org/10.1016/j.jot.2015.09.007

    Article  PubMed  Google Scholar 

  3. Kyeong, S., Kim, W. D., Feng, J. R., & Kim, J. (2018). Implementation issues of EMG-based motion intention detection for exoskeletal robots. 2018 27th IEEE International symposium on robot and human interactive communication (RO-MAN), Nanjing, China, 2018, 915-920. https://doi.org/10.1109/ROMAN.2018.8525649

  4. Chowdhury, A., Ramadas, R., & Karmakar, S. (2013). Muscle computer interface: A review. 2013 4th international conference on research into design, Chennai, India, 2013, 411–421. https://doi.org/10.1007/978-81-322-1050-4_33

  5. Dhindas, I. S., Agarwal, R., & Ryait, H. S. (2019). Performance evaluation of various classifiers for predicting knee angle from electromyography signals. Expert Systems, 36(3), e12381. https://doi.org/10.1111/exsy.12381

    Article  Google Scholar 

  6. Zhu, M., Guan, X. R., Li, Z., Gao, Y. L., Zou, K. F., Gao, X. A., Wang, Z., Li, H. B., & Cai, K. S. (2022). Prediction of knee trajectory based on surface electromyogram with independent component analysis combined with support vector regression. International Journal of Advanced Robotic Systems, 19(4), 172988062211196. https://doi.org/10.1177/17298806221119668

    Article  Google Scholar 

  7. Shima, K., & Tsuji, T. (2010). Classification of combined motions in human joints through learning of individual motions based on muscle synergy theory. 2010 IEEE/SICE international symposium on system integration, Sendai, Japan, 2010, 323-328. https://doi.org/10.1109/SII.2010.5708346

  8. Khiabani, H. & Ahmadi, M. (2021). A classical machine learning approach for emg-based lower limb intention detection for human-robot interaction systems. 2021 IEEE international conference on autonomous systems (ICAS), Montréal, Canada, 2021, 1–5. https://doi.org/10.1109/ICAS49788.2021.9551190

  9. Chen, X., Zeng, Y., & Yin, Y. H. (2017). Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(6), 577–588. https://doi.org/10.1109/TNSRE.2016.2582321

    Article  PubMed  Google Scholar 

  10. Liu, G. Y., Zhang, X. D., Sun, Q. Y., & Dong, R. L. (2020). Multi-joint motor intention recognition of lower limbs based on muscle synergies. 2020 10th institute of electrical and electronics engineers international conference on cyber technology in automation, control, and intelligent systems (CYBER), Xi’an, China, 2020, 1–5. https://doi.org/10.1109/CYBER50695.2020.9279175

  11. Akhundov, R., Saxby, D. J., Edwards, S., Snodgrass, S., Clausen, P., & Diamond, L. E. (2019). Development of a deep neural network for automated electromyographic pattern classification. Journal of Experimental Biology, 222(5), jeb198101. https://doi.org/10.1242/jeb.198101

    Article  PubMed  Google Scholar 

  12. Tao, Y. F., Huang, Y. P., Zheng, J. G., Chen, J., Zhang, Z. J., Guo, Y. J., & Li, P. F. (2019). Multi-channel sEMG based human lower limb motion intention recognition method. 2019 IEEE/ASME international conference on advanced intelligent mechatronics (AIM), Hong Kong, China, 2019, 1037–1042. https://doi.org/10.1109/AIM.2019.8868529

  13. Yuan, Y., Guo, Z. M., Wang, C., Duan, S. C., Zhang, L. F., & Wu, X. Y. (2020). Gait phase classification based on semg signals using long short-term memory for lower limb exoskeleton robot. IOP Conference Series: Materials Science and Engineering, 853(1), 012041. https://doi.org/10.1088/1757-899X/853/1/012041

    Article  Google Scholar 

  14. Bittibssi, T. M., Zekry, A. H., Genedy, M. A., & Maged, S. A. (2021). sEMG pattern recognition based on recurrent neural network. Biomedical Signal Processing and Control, 70, 103048. https://doi.org/10.1016/j.bspc.2021.103048

    Article  Google Scholar 

  15. Gautam, A., Panwar, M., Biswas, D., & Acharyya, A. (2020). MyoNet: A transfer-learning-based LRCN for lower limb movement recognition and knee joint angle prediction for remote monitoring of rehabilitation progress from sEMG. IEEE Journal of Translational Engineering in Health and Medicine, 8, 1–10. https://doi.org/10.1109/JTEHM.2020.2972523

    Article  Google Scholar 

  16. Wu, Y. H., Zheng, B., & Zhao, Y. T. (2018). Dynamic gesture recognition based on LSTM-CNN. 2018 Chinese automation congress (CAC), Xi’an, China, 2018, 2446-2450. https://doi.org/10.1109/CAC.2018.8623035

  17. Song, G., Wang, M. L., Wang, Z. J., & Ye, X. D. (2019). A motion intent recognition method for lower limbs based on CNN-RF combined model. 2019 IEEE 5th International Conference on Mechatronics System and Robots (ICMSR), Singapore, Singapore, 2019, 49–53. https://doi.org/10.1109/ICMSR.2019.8835469

  18. Zhu, M., Guan, X. R., Li, Z., He, L., Wang, Z., & Cai, K. S. (2023). sEMG-based lower limb motion prediction using CNN-LSTM with improved PCA optimization algorithm. Journal of Bionic Engineering, 20, 612–627. https://doi.org/10.1007/s42235-022-00280-3

    Article  Google Scholar 

  19. Xuan, Q., Zhou, J. C., Qiu, K. F., Chen, Z. Z., Xu, D. W., Zheng, S. L., & Yang, X. N. (2022). AvgNet: Adaptive visibility graph neural network and its application in modulation classification. IEEE Transactions on Network Science and Engineering, 9(3), 1516–1526. https://doi.org/10.1109/TNSE.2022.3146836

    Article  Google Scholar 

  20. Wu, Z. H., Pan, S. R., Long, G. D., Jiang, J., Chang, X. J., & Zhang, C. Q. (2020). Connecting the dots: Multivariate time series forecasting with graph neural networks. Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, New York, USA, 2020, 753–763. https://doi.org/10.1145/3394486.3403118

  21. Duan, Z. H., Xu, H. Y., Wang, Y. Y., Huang, Y. D., Ren, A. N., Xu, Z. B., Sun, Y. Z., & Wang, W. (2022). Multivariate time-series classification with hierarchical variational graph pooling. Neural Networks, 154, 481–490. https://doi.org/10.1016/j.neunet.2022.07.032

    Article  PubMed  Google Scholar 

  22. Lai, Z. P., Kang, X. Y., Wang, H. B., Zhang, W. Q., Zhang, X. Z., Gong, P. X., Niu, L., & Huang, H. J. (2021). STCN-GR: Spatial-temporal convolutional networks for surface-electromyography-based gesture recognition. 2021 28th international conference on neural information processing, Sanur Bali, Indonesia, 2021, 27–39. https://doi.org/10.1007/978-3-030-92238-2_3

  23. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2008). The graph neural network model. IEEE Transactions on Neural Networks, 20(1), 61–80. https://doi.org/10.1109/TNN.2008.2005605

    Article  PubMed  Google Scholar 

  24. Kipf, T. N., & Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv preprint https://doi.org/10.48550/arXiv.1609.02907

  25. Xu, K., Hu, W. H., Leskovec, J. & Jegelka, S. (2018). How powerful are graph neural networks?. arXiv preprint https://doi.org/10.48550/arXiv.1810.00826

  26. Ying, Z. T., You, J. X., Morris, C., Ren, X., Hamilton, W., & Leskovec, J. (2018). Hierarchical graph representation learning with differentiable pooling. Advances in Neural Information Processing Systems, Montréal, Canada, 2018, 4800–4810. https://proceedings.neurips.cc/paper/2018/file/e77dbaf6759253c7c6d0efc5690369c7-Paper.pdf

  27. Shervashidze, N., Schweitzer, P., Van Leeuwen, E. J., Mehlhorn, K., & Borgwardt, K. M. (2011). Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9), 2539–2561.

    MathSciNet  Google Scholar 

  28. Shi, X., Zhu, J. Q., Qin, P. J., Zhai, M. Q., & Tian, W. B. (2020). Feature extraction method of lower limb surface EMG signal based on improved energy nucleus. Chinese Journal of Scientific Instrument, 41(1), 121–128. https://doi.org/10.19650/j.cnki.cjsi.j1905438inChinese

    Article  Google Scholar 

  29. Xu, L. K., Zhang, K. Q., Xu, Z. H., & Yang, G. K. (2021). Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel. Journal of Biomedical Engineering, 38(4), 621–629. https://doi.org/10.7507/1001-5515.202010080inChinese

    Article  PubMed  Google Scholar 

  30. Cheng, H. R., Cao, G. Z., Li, C. H., Zhu, A. B., & Zhang, X. D. (2020). CNN-LSTM hybrid model for ankle joint motion recognition method based on sEMG. 2020 17th International Conference on Ubiquitous Robots (UR), Kyoto, Japan, 2020, 339–344. https://doi.org/10.1109/UR49135.2020.9144698

  31. Wu, H. F., Huang, Q., Wang, D. Q., & Gao, L. F. (2018). A CNN-SVM combined model for pattern recognition of knee motion using mechanomyography signals. Journal of Electromyography and Kinesiology, 42, 136–142. https://doi.org/10.1016/j.jelekin.2018.07.005

    Article  PubMed  Google Scholar 

  32. Hu, H. Y., Shan, W. F., Chen, J., Xing, L. L., Heidari, A. A., Chen, H. L., He, X. X., & Wang, M. F. (2023). Dynamic individual selection and crossover boosted forensic-based investigation algorithm for global optimization and feature selection. Journal of Bionic Engineering, 20, 2416–2442. https://doi.org/10.1007/s42235-023-00367-5

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, 201210, China

    Weifan Mao, Bin Ma, Jianxing Zhang & Yizhou Lu

  2. University of Chinese Academy of Sciences, Beijing, 101408, China

    Weifan Mao, Bin Ma & Yizhou Lu

  3. University of Science and Technology of China, Hefei, 230026, China

    Zhao Li

  4. Shanghai Zhongyan Jiuyi Technology Co., Ltd., Shanghai, 201821, China

    Zhuting Yu

  5. Academy of Military Science, Beijing, 100091, China

    Feng Zhang

Authors
  1. Weifan Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianxing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yizhou Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhuting Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Feng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Ma.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 8

Table 8 The list of abbreviation
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, W., Ma, B., Li, Z. et al. STGNN-LMR: A Spatial–Temporal Graph Neural Network Approach Based on sEMG Lower Limb Motion Recognition. J Bionic Eng 21, 256–269 (2024). https://doi.org/10.1007/s42235-023-00448-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42235-023-00448-5

Keywords

Search

Navigation