Skip to main content
SpringerLink
Log in

A Software System for Training Motor Imagery in Virtual Reality

  • Conference paper
  • First Online:
Advances in Neural Computation, Machine Learning, and Cognitive Research VI (NEUROINFORMATICS 2022)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1064))

Included in the following conference series:

  • 525 Accesses

Abstract

This paper describes the hardware and software system to connect the brain-computer interface system and the virtual reality environment. The software package includes a virtual reality game application, which control is paired with the performance of mental equivalents of real movements. The motivational component of the technique is based on a game interface in a virtual reality environment. The combination of software package and virtual reality contributes to deep immersion of the user in the process of ideomotor training. A neural network classifier, which was trained and tested using publicly available data, was implemented to solve the problem of recognition and classification of mental ideomotor commands.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pfurtscheller, G., et al.: Graz-BCI: state of the art and clinical applications. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 1–4 (2003)

    Article  Google Scholar 

  2. McFarland, D.J., Wolpaw, J.R.: Brain-computer interfaces for communication and control. Commun. ACM 54(5), 60–66 (2011)

    Article  Google Scholar 

  3. Wolpaw, J.R., et al.: Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. 8(2), 164–173 (2000)

    Article  Google Scholar 

  4. Lotte, F., Larrue, F., Mühl, C.: Flaws in current human training protocols for spontaneous brain-computer interfaces: lessons learned from instructional design. Front. Hum. Neurosci. 7, 568 (2013)

    Article  Google Scholar 

  5. Skola, F., Tinková, S., Liarokapis, F.: Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Front. Hum. Neurosci. 13, 329 (2019)

    Article  Google Scholar 

  6. Leeb, R., Lancelle, M., Kaiser, V., Fellner, D.W., Pfurtscheller, G.: Thinking penguin: multimodal brain-computer interface control of a VR game. IEEE Trans. Comput. Intell. AI Games 5(2), 117–128 (2013)

    Article  Google Scholar 

  7. Abbasi-Asl, R., Keshavarzi, M., Chan, D.Y.: Brain-computer interface in virtual reality. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE (2019)

    Google Scholar 

  8. Pfurtscheller, G., Leeb, R., Faller, J., Neuper, C.: Brain-computer interface systems used for virtual reality control in Virtual Reality, pp.4–20 (2011)

    Google Scholar 

  9. Athanasios, V., Carolina, J., Rodolfo, A., Patrícia, F., Jean-Claude, F., Sergi, B.B.: Efficacy and brain imaging correlates of an immersive motor imagery BCI-driven VR system for upper limb motor rehabilitation: a clinical case report. Front. Hum. Neurosci. 13, 244 (2019)

    Google Scholar 

  10. Athanasios, V., et al.: Effects of a brain-computer interface with virtual reality (VR) neurofeedback: a pilot study in chronic stroke patients. Front. Hum. Neurosci. 13, 210 (2019)

    Article  Google Scholar 

  11. Bohil, C., Owen, C.B., Jeong, E.J., Alicea, B., Biocca, F.: Virtual Reality and Presence. 21st Century Communication: A Reference Handbook, pp. 534–544 (2009)

    Google Scholar 

  12. Kaya, M., Binli, M.K., Ozbay, E., Yanar, H., Mishchenko, Y.: A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces. In: Scientific Data, vol. 5, no. 1. Springer Science and Business Media LLC (2018)

    Google Scholar 

  13. Mishchenko, Y., Kaya, M., Ozbay, E., Yanar, H.: Developing a three- to six-state EEG-based brain-computer interface for a virtual robotic manipulator control. IEEE Trans. Biomed. Eng. 66(4), 977–987 (2019)

    Article  Google Scholar 

  14. Hjorth, B.: EEG analysis based on time domain properties. Electroencephalogr. Clin. Neurophysiol. 29, 306–310 (1970)

    Article  Google Scholar 

  15. Welch, P.: The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans. Audio Electroacoust. 15(2), 70–73 (1967)

    Article  Google Scholar 

  16. Kiroi, V.N.: Brain-Computer Interface (History, Current State, Prospects): Monograph, p. 240. Southern Federal University Press, Rostov-on-Don (2011)

    Google Scholar 

  17. Shepelev, I.E., Vladimirskiy, B.M.: Building a neural network classifier for brain-computer interface. Neurocomput. Des. Appl. (2), 4–10

    Google Scholar 

  18. Khan, J., Bhatti, M.H., Khan, U.G., Iqbal, R.: Multiclass EEG motor-imagery classification with sub-band common spatial patterns. EURASIP J. Wirel. Commun. Netw. 2019(1), 1–9 (2019). https://doi.org/10.1186/s13638-019-1497-y

    Article  Google Scholar 

  19. Jochumsen, M., Knoche, H., Kjaer, T.W., Dinesen, B., Kidmose, P.: EEG headset evaluation for detection of single-trial movement intention for brain-computer interfaces. Sensors 20(10), 2804 (2020)

    Article  Google Scholar 

  20. Nam, C.S., Jeon, Y., Kim, Y.J., Lee, I., Park, K.: Movement imagery-related lateralization of event-related (de) synchronization (ERD/ERS): motor-imagery duration effects. Clin. Neurophysiol. 122(3), 567–577 (2011)

    Google Scholar 

  21. Pfurtscheller, G.: S5: Movement-related desynchronization and resting state sensorimotor networks. Clin. Neurophysiol. 125, S2 (2014)

    Article  Google Scholar 

  22. Kaiser, V., et al.: Cortical effects of user training in a motor imagery-based brain-computer interface measured by fNIRS and EEG. Neuroimage 85, 432–444 (2014)

    Google Scholar 

  23. McFarland, D.J., Sarnacki, W.A., Wolpaw, J.R.: Effects of training pre-movement sensorimotor rhythms on behavioral performance. J. Neural Eng. 12(6), 066021 (2015)

    Article  Google Scholar 

  24. Lorey, B., et al.: How equivalent are the action execution, imagery, and observation of intransitive movements? Revisiting the concept of somatotopy during action simulation. Brain Cogn. 81(1), 139–150 (2013)

    Google Scholar 

  25. Velu, P.D., de Sa, V.R.: Single-trial classification of gait and point movement preparation from human EEG. Front Neurosci. 7 (2013)

    Google Scholar 

  26. Gonzalez, S.L., de Peralta, R.G., Thut, G., Millán, J.D.R., Morier, P., Landis, T.: Very high frequency oscillations (VHFO) as a predictor of movement intentions. Neuroimage 32(1), 170–179 (2006)

    Article  Google Scholar 

  27. Tang, Z., Li, C., Sun, S.: Single-trial EEG classification of motor imagery using deep convolutional neural networks. Optik 130, 11–18 (2017)

    Article  Google Scholar 

  28. Kiroy, V.N., Vladimirskiy, B.M., Aslanian, E.V., Bakhtin, O.M., Miniaeva, N.R.: Electrographic correlates of real and imaginary movements: spectral analysis. Zhurnal vysshei nervnoi deiatelnosti imeni IP Pavlova 60(5), 525–533 (2010)

    Google Scholar 

  29. Lazurenko, D.M., Kiroy, V.N., Aslanyan, E.V., Shepelev, I.E., Bakhtin, O.M., Minyaeva, N.R.: Electrographic properties of movement-related potentials. Neurosci. Behav. Physiol. 48(9), 1078–1087 (2018)

    Article  Google Scholar 

  30. Smith, M.M., et al.: Non-invasive detection of high gamma band activity during motor imagery Front. Hum. Neurosci. 8 (2014)

    Google Scholar 

  31. Mashat, M.E.M., Lin, C.T., Zhang, D.: Effects of task complexity on motor imagery-based brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 27(10), 2178–2185 (2019)

    Article  Google Scholar 

  32. Kiroy, V.N., et al.: Spoken and inner speech-related EEG connectivity in different spatial direction. Biomed. Signal Process. Control 71, 103224 (2022)

    Article  Google Scholar 

  33. Amin, S.U., Alsulaiman, M., Muhammad, G., Mekhtiche, M.A., Hossain, M.S.: Deep learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Futur. Gener. Comput. Syst. 101, 542–554 (2019)

    Article  Google Scholar 

  34. Jahangiri, A., Sepulveda, F.: The relative contribution of high-gamma linguistic processing stages of word production, and motor imagery of articulation in class separability of covert speech tasks in EEG data. J. Med. Syst. 43(2), 20 (2019)

    Article  Google Scholar 

  35. Amo, U.C., Boquete, L., de Santiago, L., Barea, N.R., Cavaliere, C.: Induced gamma-band activity during actual and imaginary movements: EEG analysis. Sensors 20(6), 1545 (2020)

    Article  Google Scholar 

  36. Mohseni, M., Shalchyan, V., Jochumsen, M., Niazi, I.K.: Upper limb complex movements decoding from pre-movement EEG signals using wavelet common spatial patterns. Comput. Meth. Prog. Biomed. 183, 105076 (2020)

    Article  Google Scholar 

Download references

Funding

This work was supported financially by the Russian Science Foundation Grant No. 20-19-00627: “Development of a stimulus-independent Brain-Computer Interface Model for Rehabilitation of People with Disabilities”.

Author information

Authors and Affiliations

  1. Research Center for Neurotechnology, Southern Federal University, Rostov-on-Don, 344090, Russian Federation

    Danil I. Shepelev, Anton I. Saevsky, Igor E. Shepelev, Dmitry G. Shaposhnikov & Dmitry M. Lazurenko

Authors
  1. Danil I. Shepelev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anton I. Saevsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Igor E. Shepelev
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dmitry G. Shaposhnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dmitry M. Lazurenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Danil I. Shepelev .

Editor information

Editors and Affiliations

  1. Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, Russia

    Boris Kryzhanovsky

  2. Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, Russia

    Witali Dunin-Barkowski

  3. Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, Russia

    Vladimir Redko

  4. Moscow Aviation Institute (National Research University), Moscow, Russia

    Yury Tiumentsev

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shepelev, D.I., Saevsky, A.I., Shepelev, I.E., Shaposhnikov, D.G., Lazurenko, D.M. (2023). A Software System for Training Motor Imagery in Virtual Reality. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research VI. NEUROINFORMATICS 2022. Studies in Computational Intelligence, vol 1064. Springer, Cham. https://doi.org/10.1007/978-3-031-19032-2_9

Download citation

Publish with us

Policies and ethics

Search

Navigation