Skip to main content
Log in

Virtual Reality Embodiment in Motor Imagery Brain–Computer Interface Training

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

This study investigates how the avatar embodiment in virtual reality (VR) influences training for operation of motor imagery brain–computer interfaces (MI-BCIs). In previous studies, we demonstrated the possibility to induce a BCI-based VR avatar embodiment (using mental commands instead of motion tracking) for purposes of training facilitation. This paper examines the relationship between BCI performance and subjective levels of embodiment, as well as differences in BCI performance achieved with training in two different VR environments. MI-BCI training variants with the following forms of feedback were studied: a) with feedback exploiting avatar embodiment in immersive VR (avatar movements indicating the user performance), b) with feedback exploiting avatar embodiment in gamified and progressively paced VR training, and c) with symbolic feedback in the standard training with Graz protocol. On-line performance from the BCI experiments and questionnaires on the sense of ownership and sense of agency toward the virtual avatar were studied. Questionnaire analysis showed that a robust sense of embodiment arose in the VR training environments, with a strong correlation between the reported ownership and agency toward the avatar. Interestingly, the achieved BCI performance was uncorrelated with both the ownership and the agency. Using gamification further increased the performance (but not the reported sense of ownership) in the training session. Embodiment in VR mediated by synchrony between mental commands and visual stimulation in VR arose under different conditions than embodiment based on visuo-motor synchrony. Consistency between the perceived sense of ownership and agency plays a more important role than the ability to issue MI-BCI commands correctly. These findings help to elucidate the positive effect of embodiment to initial steps in BCI training and can be leveraged in future MI-BCI training designs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Slater M. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc B Biol Sci. 2009;364(1535):3549–57.

    Article  Google Scholar 

  2. Graimann B, Allison B, Pfurtscheller G. (eds.): Brain-computer Interfaces: Revolutionizing Human-computer Interaction. Frontiers collection. Springer, Heidelberg.2010; OCLC: ocn707710772.

  3. Mühl C, Allison B, Nijholt A, Chanel G. A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges. Brain-Comput Interfaces. 2014;1(2):66–84.

    Article  Google Scholar 

  4. Kerous B, Skola F, Liarokapis F. EEG-based BCI and video games: a progress report. Virtual Reality;2017.

  5. Kilteni K, Groten R, Slater M. The Sense of Embodiment in Virtual Reality. Presence. 2012;21(4):373–87.

    Article  Google Scholar 

  6. Škola F, Liarokapis F. Embodied VR environment facilitates motor imagery brain-computer interface training. Comput Graph. 2018;75:59–71.

    Article  Google Scholar 

  7. Škola F, Tinková S, Liarokapis F. Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment. Front Hum Neurosci. 2019;13:329.

    Article  Google Scholar 

  8. Škola F. Reducing the motor imagery brain-computer interface training time with embodiment in virtual reality; 2020.

  9. Niedermeyer E, Lopes da Silva F.H. (eds.): Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, 5th ed edn. Lippincott Williams & Wilkins, Philadelphia; 2005.

  10. von Ellenrieder N, Dan J, Frauscher B, Gotman J. Sparse asynchronous cortical generators can produce measurable scalp EEG signals. Neuroimage. 2016;138:123–33.

    Article  Google Scholar 

  11. Nunez PL, Srinivasan R, Fields RD. EEG functional connectivity, axon delays and white matter disease. Clin Neurophysiol. 2015;126(1):110–20.

    Article  Google Scholar 

  12. Song J, Davey C, Poulsen C, Luu P, Turovets S, Anderson E, Li K, Tucker D. EEG source localization: Sensor density and head surface coverage. J Neurosci Methods. 2015;256:9–21.

    Article  Google Scholar 

  13. Gevins A, Leong H, Smith ME, Le J, Du R. Mapping cognitive brain function with modern high-resolution electroencephalography. Trends Neurosci. 1995;18(10):429–36.

    Article  Google Scholar 

  14. Lotte F, Bougrain L, Clerc M. Electroencephalography (EEG)-Based Brain-Computer Interfaces. Wiley Encyclopedia of Electrical and Electronics Engineering; 2015.

  15. Mathewson KE, Harrison TJL, Kizuk SAD. High and dry? Comparing active dry EEG electrodes to active and passive wet electrodes. Psychophysiology. 2017;54(1):74–82.

    Article  Google Scholar 

  16. Brodu N, Lotte F, Lécuyer A. Comparative study of band-power extraction techniques for motor imagery classification. In: Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), 2011 IEEE Symposium on, pp. 1–6. IEEE, ???; 2011.

  17. Pfurtscheller G, Neuper C. Motor imagery and direct brain-computer communication. Proc IEEE. 2001;89(7):1123–34.

    Article  Google Scholar 

  18. Pfurtscheller G, Aranibar A. Event-related cortical desynchronization detected by power measurements of scalp EEG. Electroencephalogr Clin Neurophysiol. 1977;42(6):817–26.

    Article  Google Scholar 

  19. Jeannerod M. Mental imagery in the motor context. Neuropsychologia. 1995;33(11):1419–32.

    Article  Google Scholar 

  20. Kandel ER, Schwartz JH, Jessell TM, Biochemistry Do, Jessell MBT, Siegelbaum S, Hudspeth AJ. Principles of Neural Science vol. 4. McGraw-hill New York, ???; 2000.

  21. Kalcher J, Pfurtscheller G. Discrimination between phase-locked and non-phase-locked event-related EEG activity. Electroencephalogr Clin Neurophysiol. 1995;94(5):381–4.

    Article  Google Scholar 

  22. Dickhaus T, Sannelli C, Müller K-R, Curio G, Blankertz B. Predicting BCI performance to study BCI illiteracy. BMC Neurosci. 2009;10(Suppl 1):84.

    Article  Google Scholar 

  23. Kaiser V, Bauernfeind G, Kreilinger A, Kaufmann T, Kübler A, Neuper C, Müller-Putz GR. Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG. Neuroimage. 2014;85:432–44.

    Article  Google Scholar 

  24. Sweller J, van Merrienboer JJG, Paas FGWC. Cognitive Architecture and Instructional Design. Educ Psychol Rev. 1998;10(3):251–96.

    Article  Google Scholar 

  25. Lotte F, Larrue F, Muehl C. Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front Hum Neurosci. 2013;7:568.

    Article  Google Scholar 

  26. Jeunet C, Jahanpour E, Lotte F. Why standard brain-computer interface (BCI) training protocols should be changed: an experimental study. J Neural Eng. 2016;13(3):036024.

    Article  Google Scholar 

  27. Jeunet C, N’Kaoua B, Lotte F. Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates. Prog Brain Res. 2016;228:3–35.

    Article  Google Scholar 

  28. Kosmyna N, Lécuyer A. Designing Guiding Systems for Brain-Computer Interfaces. Front Hum Neurosci. 2017;11.

  29. Sollfrank T, Ramsay A, Perdikis S, Williamson J, Murray-Smith R, Leeb R, Millán JdR, Kübler A. The effect of multimodal and enriched feedback on SMR-BCI performance. Clin Neurophysiol. 2016;127(1), 490–498.

  30. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining” gamification”. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 2011; pp 9–15.

  31. de Freitas S. Technology: Game for change. Nature. 2011;470:330–1.

    Article  Google Scholar 

  32. Yee N, Bailenson J. The Proteus Effect: The Effect of Transformed Self-Representation on Behavior. Hum Commun Res. 2007;33(3):271–90.

    Article  Google Scholar 

  33. Osimo SA, Pizarro R, Spanlang B, Slater M. Conversations between self and self as Sigmund Freud-A virtual body ownership paradigm for self counselling. Sci Rep. 2015;5:13899.

    Article  Google Scholar 

  34. Falconer CJ, Rovira A, King JA, Gilbert P, Antley A, Fearon P, Ralph N, Slater M, Brewin CR. Embodying self-compassion within virtual reality and its effects on patients with depression. Br J Psychiatry Open. 2016;2(1):74–80.

    Article  Google Scholar 

  35. Seinfeld S, Arroyo-Palacios J, Iruretagoyena G, Hortensius R, Zapata LE, Borland D, Gelder Bd, Slater M, Sanchez-Vives MV. Offenders become the victim in virtual reality: impact of changing perspective in domestic violence. Sci Rep. 2018;8(1), 2692.

  36. Petkova VI, Ehrsson HH. If I were you: perceptual illusion of body swapping. PLoS ONE. 2008;3(12):3832.

    Article  Google Scholar 

  37. Slater M, Perez-Marcos D, Ehrsson HH, Sanchez-Vives MV. Inducing Illusory Ownership of a Virtual Body. Front Neurosci. 2009;3(2):214–20.

    Article  Google Scholar 

  38. Jeannerod M. The mechanism of self-recognition in humans. Behav Brain Res. 2003;142(1):1–15.

    Article  Google Scholar 

  39. Meehan M, Insko B, Whitton M, Brooks FP, Jr. Physiological Measures of Presence in Stressful Virtual Environments. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’02, 2002;pp. 645–652. ACM, New York, NY, USA.

  40. Gallagher S. Philosophical conceptions of the self: implications for cognitive science. Trends Cogn Sci. 2000;4(1):14–21.

    Article  Google Scholar 

  41. Blanke O, Metzinger T. Full-body illusions and minimal phenomenal selfhood. Trends Cogn Sci. 2009;13(1):7–13.

    Article  Google Scholar 

  42. Gallagher S. The Natural Philosophy of Agency. Philos Compass. 2007;2(2):347–57.

    Article  Google Scholar 

  43. Perez-Marcos D, Slater M, Sanchez-Vives MV. Inducing a virtual hand ownership illusion through a brain-computer interface. NeuroReport. 2009;20(6):589–94.

    Article  Google Scholar 

  44. Bell CJ, Shenoy P, Chalodhorn R, Rao RPN. Control of a humanoid robot by a noninvasive brain-computer interface in humans. J Neural Eng. 2008;5(2):214–20.

    Article  Google Scholar 

  45. Alimardani M, Nishio S, Ishiguro H. Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators. Sci Rep. 2013;3.

  46. Alimardani M, Nishio S, Ishiguro H. Removal of proprioception by BCI raises a stronger body ownership illusion in control of a humanlike robot. Sci Rep. 2016;6:33514.

    Article  Google Scholar 

  47. Alimardani M, Nishio S, Ishiguro H. The Importance of Visual Feedback Design in BCIs; from Embodiment to Motor Imagery Learning. PLoS ONE. 2016;11(9):0161945.

    Article  Google Scholar 

  48. Evans N, Gale S, Schurger A, Blanke O. Visual feedback dominates the sense of agency for brain-machine actions. PLoS ONE. 2015;10(6):0130019.

    Article  Google Scholar 

  49. Vourvopoulos A, Ferreira A, Badia SBi. NeuRow: An Immersive VR Environment for Motor-Imagery Training with the Use of Brain-Computer Interfaces and Vibrotactile Feedback:. In: 3rd International Conference on Physiological Computing Systems, 2016;pp. 43–53. SCITEPRESS - Science and Technology Publications, ???.

  50. Braun N, Emkes R, Thorne JD, Debener S. Embodied neurofeedback with an anthropomorphic robotic hand. Sci Rep. 2016;6:37696.

    Article  Google Scholar 

  51. Salisbury DB, Dahdah M, Driver S, Parsons TD, Richter KM. Virtual reality and brain computer interface in neurorehabilitation. Proceedings (Baylor University. Medical Center). 2016;29(2), 124–127.

  52. Vourvopoulos A, Bermúdez i Badia S. Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J NeuroEng Rehabil. 2016;13:69.

  53. Vourvopoulos A, Pardo OM, Lefebvre S, Neureither M, Saldana D, Jahng E, Liew S-L. Effects of a Brain-Computer Interface With Virtual Reality (VR) Neurofeedback: A Pilot Study in Chronic Stroke Patients. Front Hum Neurosci. 2019;13.

  54. Botvinick M, Cohen J. Rubber hands’ feel’touch that eyes see. Nature. 1998;391(6669):756.

    Article  Google Scholar 

  55. Longo MR, Schüür F, Kammers MPM, Tsakiris M, Haggard P. What is embodiment? A psychometric approach. Cognition. 2008;107(3):978–98.

    Article  Google Scholar 

  56. Shannon CE. A Mathematical Theory of Communication. In: Bell System Technical Journal. 3, vol. 27, 1949;pp. 379–423. American Telephone and Telegraph Company, ???.

  57. Krausz G, Scherer R, Korisek G, Pfurtscheller G. Critical decision-speed and information transfer in the “Graz Brain-Computer Interface.’’. Appl Psychophysiol Biofeedback. 2003;28(3):233–40.

    Article  Google Scholar 

  58. Kondo T, Saeki M, Hayashi Y, Nakayashiki K, Takata Y. Effect of instructive visual stimuli on neurofeedback training for motor imagery-based brain-computer interface. Hum Mov Sci. 2015;43:239–49.

    Article  Google Scholar 

  59. Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, Bertrand O, Lécuyer A. Openvibe: An open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence Teleoper Virt Environ. 2010; 19(1), 35–53.

  60. Barbero, Grosse-Wentrup M. Biased feedback in brain-computer interfaces. J NeuroEng Rehabil. 2010;7(1), 34.

  61. Faller J, Vidaurre C, Solis-Escalante T, Neuper C, Scherer R. Autocalibration and Recurrent Adaptation: Towards a Plug and Play Online ERD-BCI. IEEE Trans Neural Syst Rehabil Eng. 2012;20(3):313–9.

    Article  Google Scholar 

Download references

Acknowledgements

This research was partially supported by the project that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Filip Škola.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

“This article is part of the topical collection “Advances on Computer Vision, Imaging and Computer Graphics Theory and Applications” guest edited by Kadi Bouatouch, Augusto Sousa, Mounia Ziat and Helen Purchase”.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Škola, F., Liarokapis, F. Virtual Reality Embodiment in Motor Imagery Brain–Computer Interface Training. SN COMPUT. SCI. 4, 17 (2023). https://doi.org/10.1007/s42979-022-01402-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-022-01402-8

Keywords

Navigation