Skip to main content
SpringerLink
Log in

The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback

  • Published:
Human Physiology Aims and scope Submit manuscript

Abstract

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wolpaw, J.R., Birbaumer, N., Mcfarland, D.J., et al., Brain–computer interfaces for communication and control, Clin. Neurophysiol., 2002, vol. 113, no. 6, p. 767.

    Article  PubMed  Google Scholar 

  2. Kaplan, A.Ya., Zhigul’skaya, D.D., and Kir’yanov, D.A., The possible control of individual fingers of the phantom of the human hand in the contour of the brain-computer interface on the P300 wave, Vestn. Ross. Gos. Med. Univ., 2016, no. 2, p. 26.

    Google Scholar 

  3. Birbaumer, N. and Cohen, L.G., Brain–computer interfaces: communication and restoration of movement in paralysis, J. Physiol., 2007, vol. 579, no. 3, p. 621.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  4. Frolov, A.A., Húsek, D., Biryukova, E.V., et al., Principles of motor recovery in post-stroke patients using hand exoskeleton controlled by the brain-computer interface based on motor imagery, Neural Network World, 2017, vol. 107, p. 137.

    Google Scholar 

  5. Kotov, S.V., Turbina, L.G., Bobrov, P.D., et al., Rehabilitation of stroke patients with a bioengineered “brain–computer interface with exoskeleton” system, Neurosci. Behav. Physiol., 2016, vol. 46, no. 5, p. 518.

    Article  Google Scholar 

  6. Kaplan, A., Vasilyev, A., Liburkina, S., and Yakovlev, L., Poor BCI performers still could benefit from motor imagery training, Proc. 10th Int. Conf. “AC Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience,” Toronto, ON, Canada, July 17–22, 2016, New York: Springer-Verlag, 2016, part 1, p. 46.

    Google Scholar 

  7. Vasilyev, A., Liburkina, S., Yakovlev, L., et al., Assessing motor imagery in brain-computer interface training: psychological and neurophysiological correlates, Neuropsychologia, 2017, vol. 97, p. 56.

    Article  PubMed  Google Scholar 

  8. Mokienko, O.A., Chervyakov, A.V., Kulikova, S.N., et al., Increased motor cortex excitability during motor imagery in brain-computer interface trained subjects, Front. Comput. Neurosci., 2013, vol. 7, p. 168. doi 10.3389/fncom.2013.00168

    Article  PubMed  PubMed Central  Google Scholar 

  9. Mulder, T., Motor imagery and action observation: cognitive tools for rehabilitation, J. Neural Transm., 2007, vol. 114, no. 10, p. 1265.

    Article  PubMed  PubMed Central  Google Scholar 

  10. De Vries, S., Tepper, M., Feenstra, W., et al., Motor imagery ability in stroke patients: the relationship between implicit and explicit motor imagery measures, Front. Hum. Neurosci., 2013, vol. 7, p. 790.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Simmons, L., Sharma, N., Baron, J.C., and Pomeroy, V.M., Motor imagery to enhance recovery after subcortical stroke: who might benefit, daily dose, and potential effects, Neurorehabil. Neural Repair, 2008, vol. 22, no. 5, p. 458.

    Article  PubMed  Google Scholar 

  12. Vuckovic, A. and Osuagwu, B.A., Using a motor imagery questionnaire to estimate the performance of a brain–computer interface based on object oriented motor imagery, Clin. Neurophysiol., 2013, vol. 124, no. 8, p. 1586.

    Article  PubMed  Google Scholar 

  13. Alimardani, M., Nishio, S., and Ishiguro, H., The importance of visual feedback design in BCIs; from embodiment to motor imagery learning, PloS One, 2016, vol. 11, no. 9, p. e0161945.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Sollfrank, T., Ramsay, A., Perdikis, S., et al., The effect of multimodal and enriched feedback on SMR-BCI performance, Clin. Neurophysiol., 2016, vol. 127, no. 1, p. 490.

    Article  PubMed  CAS  Google Scholar 

  15. Kaufmann, T., Holz, E.M., and Kübler, A., Comparison of tactile, auditory, and visual modality for brain–computer interface use: a case study with a patient in the locked-in state, Front. Neurosci., 2013, vol. 7, p. 129.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Kauhanen, L., Palomäki, T., Jylänki, P., et al., Haptic feedback compared with visual feedback for BCI, Proc. 3rd Int. Brain-Computer Interface Workshop & Training Course 2006, Graz, 2006.

    Google Scholar 

  17. Angulo-Sherman, I.N. and Gutierrez, D., Effect of different feedback modalities in the performance of braincomputer interfaces, 24th Int. Conf. on Electronics, Communications and Computers (CONIELECOMP 2014), México: Univ. Am. Puebla, 2014, p. 14.

    Chapter  Google Scholar 

  18. Godlove, J.M., Whaite, E.O., and Batista, A.P., Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control, J. Neural Eng., 2014, vol. 11, no. 4, p. 046025.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Halder, S., Rea, M., Andreoni, R., et al., An auditory oddball brain–computer interface for binary choices, Clin. Neurophysiol., 2010, vol. 121, no. 4, p. 516.

    Article  PubMed  CAS  Google Scholar 

  20. Schreuder, M., Rost, T., and Tangermann, M., Listen, you are writing! Speeding up online spelling with a dynamic auditory BCI, Front. Neurosci., 2011, vol. 5, p. 112.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hinterberger, T., Neumann, N., Pham, M., et al., A multimodal brain-based feedback and communication system, Exp. Brain Res., 2004, vol. 154, no. 4, p. 521.

    Article  PubMed  CAS  Google Scholar 

  22. Pham, M., Hinterberger, T., Neumann, N., et al., An auditory brain-computer interface based on the selfregulation of slow cortical potentials, Neurorehabil. Neural Repair, 2005, vol. 19, no. 3, p. 206.

    Article  Google Scholar 

  23. Nijboer, F., Furdea, A., Gunst, I., et al., An auditory brain–computer interface (BCI), J. Neurosci. Methods, 2008, vol. 167, no. 1, p. 43.

    Article  PubMed  Google Scholar 

  24. Furdea, A., Halder, S., Krusienski, D.J., et al., An auditory oddball (P300) spelling system for brain computer interfaces, Psychophysiology, 2009, vol. 46, no. 3, p. 617.

    Article  PubMed  CAS  Google Scholar 

  25. Schreuder, M., Blankertz, B., and Tangermann, M., A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue, PloS One, 2010, vol. 5, no. 4, p. e9813.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. Changa, M., Morib, K., Makinoa, S., and Rutkowsk, T.M., Spatial auditory two-step input Japanese syllabary brain-computer interface speller, Procedia Technol., 2014, vol. 18, p. 25.

    Article  Google Scholar 

  27. Leeb, R., Gwak, K., Kim, D.-S., and del Millán, J.R., Freeing the visual channel by exploiting vibrotactile BCI feedback, 35th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Piscataway, NJ: IEEE Eng. Med. Biol. Soc., 2013, p. 3093.

    Google Scholar 

  28. Cincotti, F., Kauhanen, L., Aloise, F., et al., Vibrotactile feedback for brain-computer interface operation, Comput. Intell. Neurosci., 2007, vol. 2007, art. ID 48937. doi 10.1155/2007/48937

  29. Gwak, K., Leeb, R., del Millán, J.R., and Kim, D.-S., Quantification and reduction of visual load during BCI operation, IEEE Int. Conf. on Systems, Man, and Cybernetics, San Diego, 2014, p. 2795.

    Google Scholar 

  30. Jeunet, C., Vi, C., Spelmezan, D., et al., Continuous tactile feedback for motor-imagery based brain-computer interaction in a multitasking context, Int. Conf. on Human-Computer Interaction, INTERACT 2015, New York: Springer-Verlag, 2015, p. 488.

    Chapter  Google Scholar 

  31. Gomez-Rodriguez, M., Peters, J., Hill, J., et al., Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery, J. Neural Eng., 2011, vol. 8, no. 3, p. 036005.

    Article  PubMed  CAS  Google Scholar 

  32. Vourvopoulos, A., Ferreira, A., and Badia, S.B., Neu-Row: an immersive VRenvironment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback, Proc. 3rd Int. Conf. on Physiological Computing Systems (PhyCS 2016), Lisbon, 2016, p. 43. doi 10.5220/0005939400430053

    Google Scholar 

  33. Oldfield, R.C., The assessment and analysis of handedness: the Edinburgh inventory, Neuropsychologia, 1971, vol. 9, no. 1, p. 97.

    Article  PubMed  CAS  Google Scholar 

  34. Doskin, V.A., Lavrent’eva, N.A., Miroshnikova, M.P., and Sharai, V.B., Test of differential self-evaluation of a functional state, Vopr. Psikhol., 1973, vol. 6, p. 141.

    Google Scholar 

  35. Koles, Z.J., Lazar, M.S., and Zhou, S.Z., Spatial patterns underlying population differences in the background EEG, Brain Topogr., 1990, vol. 2, no. 4, p. 275.

    Article  PubMed  CAS  Google Scholar 

  36. Trefethen, L.N. and Bau, III D., Numerical Linear Algebra, Philadelphia: Soc. Ind. Appl. Math., 1997. ISBN 978-0-89871-361-9.

    Book  Google Scholar 

  37. http://martinos.org/mne/stable/generated/mne.decoding. CSP.html.

  38. Hjorth, B., An on-line transformation of EEG scalp potentials into orthogonal source derivations, Electroencephalogr. Clin. Neurophysiol., 1975, vol. 39, no. 5, p. 526.

    Article  PubMed  CAS  Google Scholar 

  39. R.C. Team, R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, 2016. http://www.R-project.org.

    Google Scholar 

  40. Lawrence, M.A., ez: Easy analysis and visualization of factorial experiments, R Package version 4.4-0, 2016. https://CRAN.R-project.org/package=ez.

    Google Scholar 

  41. Thurlings, M.E., van Erp, J.B.F., Brouwer, A.M., et al., Control-display mapping in brain–computer interfaces, Ergonomics, 2012, vol. 55, no. 5, p. 564.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lobachevskii Nizhny Novgorod State University, Nizhny Novgorod, Russia

    M. V. Lukoyanov, S. Yu. Gordleeva, A. S. Pimashkin, N. A. Grigor’ev, A. V. Savosenkov, A. Motailo, V. B. Kazantsev & A. Ya. Kaplan

  2. Moscow State University, Moscow, Russia

    A. Ya. Kaplan

  3. Nizhny Novgorod State Medical Academy, Nizhny Novgorod, Russia

    M. V. Lukoyanov

Authors
  1. M. V. Lukoyanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Yu. Gordleeva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. S. Pimashkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. N. A. Grigor’ev
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. V. Savosenkov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Motailo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. V. B. Kazantsev
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. A. Ya. Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Yu. Gordleeva.

Additional information

Original Russian Text © M.V. Lukoyanov, S.Yu. Gordleeva, A.S. Pimashkin, N.A. Grigor’ev, A.V. Savosenkov, A. Motailo, V.B. Kazantsev, A.Ya. Kaplan, 2018, published in Fiziologiya Cheloveka, 2018, Vol. 44, No. 3, pp. 53–61.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lukoyanov, M.V., Gordleeva, S.Y., Pimashkin, A.S. et al. The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback. Hum Physiol 44, 280–288 (2018). https://doi.org/10.1134/S0362119718030088

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0362119718030088

Keywords

Search

Navigation