Skip to main content

Detections of Steady-State Visual Evoked Potential and Simultaneous Jaw Clench Action from Identical Occipital Electrodes: A Hybrid Brain-Computer Interface Study

Abstract

Purpose

For users with severe motor impairments such as high paraplegia, constructing a simplified, practical, and effective brain-computer interface (BCI) system is critical to improve the quality of life and reduce nursing requirements. This is challenging for users who retain only muscle function above the neck.

Method

In this experiment, both able-bodied and motor impairment subjects attended to a flickering stimulus and performed jaw clenches simultaneously. This paper focused on the feasibility of sharing the same collection sites for steady-state visual evoked potentials (SSVEPs) with electromyograms (EMGs) and the potential of building a parallel hybrid BCI system.

Results

The results reveal that when the visual stimulation frequency was lower than 20 Hz, there was no serious crosstalk between SSVEP and EMG from jaw clench actions. The EMG signal slightly affects the recognition of SSVEP, while the recognition rate of jaw clench movements based on the mixed signal exceeded 95%.

Conclusions

For patients with severe disabilities, the rare applicable EMG signal is facial muscle electrical activity. The proposed study made full use of the combination of jaw clench-related EMG and SSVEP to solve this problem. Only using the same occipital electrodes to simultaneously collect SSVEP with jaw clench-related EMG and classify them could further promote the development and practical application of hybrid BCIs.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Vaughan, T. M., Heetderks, W. J., Trejo, L. J., Rymer, W. Z., Weinrich, M., Noore, M. M., Kubler, A., Dobkin, B. H., Birbaumer, N., Douchin, E., Wolpaw, E. W., & Wolpaw, J. R. (2003). Brain-computer interface technology: A review of the second international meeting. IEEE Transactions on Neural Systems & Rehabilitation Engineering, 11(2), 94–109. https://doi.org/10.1109/TNSRE.2003.814799

    Article  Google Scholar 

  2. 2.

    Wolpaw, J. R., & Wolpaw, E. W. (2012). Brain-computer interfaces: Something new under the sun. Oxford University Press.

    Google Scholar 

  3. 3.

    Wickelgren, I. (2004). Neuroprosthetics. Brain-computer interface adds a new dimension. Science, 306(53), 1878–1879. https://doi.org/10.1126/science.306.5703.1878a

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Bin, G., Gao, X., Yan, Z., Hong, B., & Gao, S. (2009). An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. Journal of Neural Engineering, 6(4), 046002. https://doi.org/10.1088/1741-2560/6/4/046002

    Article  PubMed  Google Scholar 

  5. 5.

    Aloise, F., Schettini, F., Aricò, P., Leotta, F., Salinari, S., Mattia, D., Babiloni, F., & Cincotti, F. (2011). P300-based brain-computer interface for environmental control: An asynchronous approach. Journal of Neural Engineering, 8(2), 025025. https://doi.org/10.1088/1741-2560/8/2/025025

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    Hwang, H., Kwon, K., & Im, C. (2009). Neurofeedback-based motor imagery training for brain-computer interface (BCI). Journal of Neuroscience Methods, 179(1), 150–156. https://doi.org/10.1016/j.jneumeth.2009.01.015

    Article  PubMed  Google Scholar 

  7. 7.

    Rebsamen, B., Guan, C., Zhang, H., Wang, C., Teo, C., Ang, M. H., & Burdet, E. (2010). A brain controlled wheelchair to navigate in familiar environments. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(6), 590–598. https://doi.org/10.1109/TNSRE.2010.2049862

    Article  PubMed  Google Scholar 

  8. 8.

    Shyu, K., Chiu, Y., Lee, P., Lee, M., Sie, J., Wu, C., Wu, Y., & Tung, P. (2013). Total design of an FPGA-based brain-computer interface control hospital bed nursing system. IEEE Transactions on Industrial Electronics, 60(7), 2731–2739. https://doi.org/10.1109/TIE.2012.2196897

    Article  Google Scholar 

  9. 9.

    Pfurtscheller, G., Solis-Escalante, T., Ortner, R., Linortner, P., & Muller-Putz, G. R. (2010). Self-Paced operation of an SSVEP-based orthosis with and without an imagery-based “Brain Switch:” A feasibility study towards a hybrid BCI. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(4), 409–414. https://doi.org/10.1109/TNSRE.2010.2040837

    Article  PubMed  Google Scholar 

  10. 10.

    Allison, B. Z., Brunner, C., Altstätter, C., Wagner, I. C., Grissmann, S., & Neuper, C. (2012). A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control. Journal of Neuroscience Methods, 209(2), 299–307. https://doi.org/10.1016/j.jneumeth.2012.06.022

    Article  PubMed  Google Scholar 

  11. 11.

    Scherer, R., Müller-Putz, G. R., & Pfurtscheller, G. (2007). Self-initiation of EEG-Based brain-computer communication using the heart rate response. Journal of Neural Engineering, 4(4), L23-29. https://doi.org/10.1088/1741-2560/4/4/L01

    CAS  Article  PubMed  Google Scholar 

  12. 12.

    Shinde, N. & George, K. (2016). Brain-controlled driving aid for electric wheelchairs. In 2016 IEEE 13th international conference on wearable and implantable body sensor networks (BSN), pp. 115–118. https://doi.org/10.1109/BSN.2016.7516243.

  13. 13.

    Chai, X., Zhang, Z., Guan, K., Lu, Y., Liu, G., Zhang, T., & Niu, H. (2020). A hybrid BCI-controlled smart home system combining SSVEP and EMG for individuals with paralysis. Biomedical Signal Processing and Control, 56(2), 101687. https://doi.org/10.1016/j.bspc.2019.101687

    Article  Google Scholar 

  14. 14.

    Lin, K., Cinetto, A., Wang, Y., Chen, X., Gao, S., & Gao, X. (2016). An online hybrid BCI system based on SSVEP and EMG. Journal of Neural Engineering, 13(2), 026020. https://doi.org/10.1088/1741-2560/13/2/026020

    Article  PubMed  Google Scholar 

  15. 15.

    Chang, B. C., & Seo, B. H. (2009). Development of new brain computer interface based on EEG and EMG. In: 2008 IEEE international conference on robotics and biomimetics, pp. 1665–1670. https://doi.org/10.1109/ROBIO.2009.4913251.

  16. 16.

    Li, Z., Lei, S., Su, C., & Li, G. (2013). Hybrid brain/muscle-actuated control of an intelligent wheelchair. IEEE International Conference on Robotics and Biomimetics (ROBIO), 2013, 19–25. https://doi.org/10.1109/ROBIO.2013.6739429

    Article  Google Scholar 

  17. 17.

    Shah, M. A., Sheikh, A. A., Sajjad, A. M., & Uppal, M. (2015). A hybrid training-less brain-machine interface using SSVEP and EMG signal. In: 2015 13th international conference on frontiers of information technology (FIT), pp. 93–97. https://doi.org/10.1109/FIT.2015.26

  18. 18.

    Gao, Q., Dou, L., Belkacem, A. N., & Chen, C. (2017). Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system. BioMed Research International, 2017, 8316485. https://doi.org/10.1155/2017/8316485

    Article  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Costa, Á., Hortal, E., Iáñez, E., & Azorín, J. M. (2014). A supplementary system for a brain-machine interface based on jaw artifacts for the bidimensional control of a robotic arm. PLoS ONE, 9(11), e112352. https://doi.org/10.1371/journal.pone.0112352

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Foldes, S. T., & Taylor, D. M. (2010). Discreet discrete commands for assistive and neuroprosthetic devices. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(3), 236–244. https://doi.org/10.1109/TNSRE.2009.2033428

    Article  PubMed  Google Scholar 

  21. 21.

    Chen, X., Chen, Z., Gao, S., & Gao, X. (2014). A high-ITR SSVEP-based BCI speller. Brain-computer interfaces, 1(3–4), 181–191. https://doi.org/10.1080/2326263X.2014.944469

    Article  Google Scholar 

  22. 22.

    Ma, K., Wang, S., Zhang, S., Sun, Y., & Zheng, D. Z. (2019). Electrode channel optimisation method for steady-state visual evoked potentials. The Journal of Engineering, 2019(23), 8632–8636. https://doi.org/10.1049/joe.2018.9071

    Article  Google Scholar 

  23. 23.

    Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4), 433–436. https://doi.org/10.1163/156856897X00357

    CAS  Article  PubMed  Google Scholar 

  24. 24.

    Chai, X., Zhang, Z., Guan, K., Liu, G., & Niu, H. (2019). A radial zoom motion-based paradigm for steady state motion visual evoked potentials. Frontiers in Human Neuroscience, 13, 127. https://doi.org/10.3389/fnhum.2019.00127

    Article  PubMed  PubMed Central  Google Scholar 

  25. 25.

    Englehart, K., & Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering, 50(7), 848–854. https://doi.org/10.1109/TBME.2003.813539

    Article  PubMed  Google Scholar 

  26. 26.

    Goncharova, I. I., McFarland, D. J., Vaughan, T. M., & Wolpaw, J. R. (2003). EMG contamination of EEG: Spectral and topographical characteristics. Clinical Neurophysiology, 114(9), 1580–1593. https://doi.org/10.1016/S1388-2457(03)00093-2

    CAS  Article  PubMed  Google Scholar 

  27. 27.

    Leeb, R., Sagha, H., Chavarriaga, R., & Millán, J. R. (2011). A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. Journal of Neural Engineering, 8(2), 025011. https://doi.org/10.1088/1741-2560/8/2/025011

    Article  PubMed  Google Scholar 

  28. 28.

    Hong, J. (2017). Multimodal brain-computer interface combining synchronously electroencephalography and electromyography. Journal of Intelligent & Fuzzy Systems, 33(6), 3355–3362. https://doi.org/10.3233/JIFS-162104

    Article  Google Scholar 

  29. 29.

    Chai, X., Zhang, Z., Lu, Y., Liu, G., Zhang, T., & Niu, H. (2019). A hybrid BCI-based environmental control system using SSVEP and EMG signals. World Congress on Medical Physics and Biomedical Engineering, 68(3), 59–63. https://doi.org/10.1007/978-981-10-9023-3_11

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 11772037) and the Key Research and Development Project of Shanxi Province (No. 201903D321167). The authors would also like to thank all of the participants who generously volunteered their time to participate in this study.

Author information

Affiliations

Authors

Contributions

ZZ, YF, and HN conceived and designed the study. ZZ established the protocol. ZZ, XC, and KG performed the experiments. KG, JX, TL helped with data processing. ZZ and HN wrote the manuscript, and all other authors reviewed and commented on the draft. All authors read and approved the manuscript.

Corresponding author

Correspondence to Haijun Niu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

The authors have read and abided by the statement of ethical standards for manuscripts submitted to the Journal of Medical and Biological Engineering. Ethical approval and Informed consent are shown in the Method section.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Chai, X., Guan, K. et al. Detections of Steady-State Visual Evoked Potential and Simultaneous Jaw Clench Action from Identical Occipital Electrodes: A Hybrid Brain-Computer Interface Study. J. Med. Biol. Eng. (2021). https://doi.org/10.1007/s40846-021-00662-8

Download citation

Keywords

  • Hybrid BCI
  • Electrode multiplexing
  • SSVEP
  • Masseter EMG
  • Jaw clenches