Skip to main content
SpringerLink
Log in
Book cover

Handbook of Digital Games and Entertainment Technologies pp 151–171Cite as

Brain-Computer Interfacing and Virtual Reality

  • Reference work entry
  • First Online:

Abstract

Brain-computer interface (BCI) and virtual reality (VR) are natural companions. BCI provides a new interaction technique for controlling VR, and VR provides a rich feedback environment for BCI while retaining a controlled and safe environment. The combination of VR and BCI allows for providing participants with novel experiences that are impossible otherwise. Both fields still pose many technological challenges to scientists and engineers, but both are making rapid progress.

VR and BCI have been combined in multiple ways: BCI can be used for navigation in VR, for controlling a virtual body, and for controlling the virtual world directly. More recent directions explore the possibilities of using BCI for purposes other than control in VR, such as designing and implementing VR systems that adapt to the participant’s cognitive and emotional state.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Recommended Reading

  • H. Alkadhi, P. Brugger, S.H. Boendermaker, G. Crelier, A. Curt, M.C. Hepp-Reymond, S.S. Kollias, What disconnection tells about motor imagery: evidence from paraplegic patients. Cereb. Cortex 15, 131–140 (2005). doi:10.1093/cercor/bhh116

    Article  Google Scholar 

  • J.D. Bayliss, Use of the evoked potential P3 component for control in a virtual apartment. IEEE Trans. Rehabil. Eng. 11(2), 113–116 (2003)

    Google Scholar 

  • J.D. Bayliss, D.H. Ballard, A virtual reality testbed for brain computer interface research. IEEE Trans. Rehabil. Eng. 8(2), 188–190 (2000)

    Google Scholar 

  • C. Berka, D.J. Levendowski, M.M. Cvetinovic, M.M. Petrovic, G. Davis, M.N. Lumicao, … R. Olmstead, Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int. J. Hum. Comput. Interact. (2004) doi:10.1207/s15327590ijhc1702_3

    Google Scholar 

  • I. Bermúdez, S. Badia, A. García Morgade, H. Samaha, P.F.M.J. Verschure, Using a hybrid brain computer interface and virtual reality system to monitor and promote cortical reorganization through motor activity and motor imagery training. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 174–181 (2013). doi:10.1109/TNSRE.2012.2229295

    Article  Google Scholar 

  • M. Botvinick, J. Cohen, Rubber hands “feel” touch that eyes see. Nature 391(6669), 756 (1998)

    Article  Google Scholar 

  • P. Brunner, S. Joshi, S. Briskin, J.R. Wolpaw, H. Bischof, G. Schalk, Does the “P300” speller depend on eye gaze? J. Neural Eng. 7, 056013 (2010). doi:10.1088/1741-2560/7/5/056013

    Article  Google Scholar 

  • M. Cheng, X. Gao, S. Gao, D. Xu, Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 49, 1181–1186 (2002). doi:10.1109/TBME.2002.803536

    Article  Google Scholar 

  • J.A. Coan, J.J.B. Allen, Frontal EEG asymmetry as a moderator and mediator of emotion. Biol. Psychol. (2004). doi:10.1016/j.biopsycho.2004.03.002

    Google Scholar 

  • O. Cohen, S. Druon, S. Lengagne, A. Mendelsohn, A. Kheddar, R. Malach, D. Friedman, fMRI-based robotic embodiment: a pilot study, in IEEE International Conference on Biomedical Robotics and Biomechatronics (Rome, 2012), pp. 314–319

    Google Scholar 

  • O. Cohen, M. Koppel, R. Malach, D. Friedman, A generic machine learning tool for whole brain classification from fMRI. in 6th International BCI Conference, Graz, Austria, (2014a)

    Google Scholar 

  • O. Cohen, M. Koppel, R. Malach, D. Friedman, Controlling an avatar by thought using real-time fMRI. J. Neural Eng. 11(3), 35006 (2014b)

    Article  Google Scholar 

  • R.J. Davidson, P. Ekman, C.D. Saron, J.A. Senulis, W.V. Friesen, Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I. J. Pers. Soc. Psychol. 58, 330–341 (1990). doi:10.1037/0022-3514.58.2.330

    Article  Google Scholar 

  • C. Davies, Virtual spaces, in Space: In Science, Art, and Society, ed. by F. Penz, G. Radick, R. Howell (Cambridge University Press, Cambridge, UK, 2004), pp. 69–104

    Google Scholar 

  • C. Davies, J. Harrison, Osmose: towards broadening the aesthetics of virtual reality. ACM Comput. Graph. [special Issue on Virtual Reality] 30(4), 25–28 (1996)

    Google Scholar 

  • E. Donchin, K.M. Spencer, R. Wijesinghe, The mental prosthesis: assessing the speed of a P300-based brain- computer interface. IEEE Trans. Rehabil. Eng. 8, 174–179 (2000). doi:10.1109/86.847808

    Article  Google Scholar 

  • M. Donnerer, A. Steed, Using a P300 brain–computer interface in an immersive virtual environment. Presence Teleop. Virt. Environ. 19(1), 12–24 (2010). doi:10.1162/pres.19.1.12

    Article  Google Scholar 

  • G. Edlinger, C. Holzner, C. Groenegress, C. Guger, M. Slater, Goal-oriented control with brain-computer interface, in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNAI, vol 5638 (2009), pp. 732–740. doi:10.1007/978-3-642-02812-0_83

    Google Scholar 

  • G. Edlinger, C. Holzner, C. Guger, A hybrid brain-computer interface for smart home control, in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNCS, vol 6762 (2011), pp. 417–426. doi:10.1007/978-3-642-21605-3_46

    Google Scholar 

  • H.H. Ehrsson, The experimental induction of out-of-body experiences. Science 317(5841), 1048 (2007)

    Article  Google Scholar 

  • N. Evans, S. Gale, A. Schurger, O. Blanke, Visual feedback dominates the sense of agency for brain-machine actions. PLoS One 10(6), e0130019 (2015). doi:10.1371/journal.pone.0130019

    Article  Google Scholar 

  • J. Faller, R. Leeb, Avatar navigation in virtual and augmented reality environments using an SSVEP BCI ICABB-2010. … and Virtual Reality. (2010) Retrieved from http://brainable.org/Documents/FallerJ-ICABB.pdf

  • J. Faller, G. Müller-Putz, D. Schmalstieg, G. Pfurtscheller, An application framework for controlling an avatar in a desktop-based virtual environment via a software SSVEP brain–computer interface. Presence Teleop. Virt. Environ. (2010). doi:10.1162/pres.19.1.25

    Google Scholar 

  • D. Friedman, R. Leeb, A. Antley, M. Garau, C. Guger, C. Keinrath, … M. Slater, Navigating virtual reality by thought: what is it like? Presence Teleop. Virt. Environ. 16(1), 100–110 (2007a)

    Google Scholar 

  • D. Friedman, R. Leeb, L. Dikovsky, M. Reiner, G. Pfurtscheller, M. Slater, Controlling a virtual body by thought in a highly-immersive virtual environment. Proc. Graph. Appl. Barcelona, Spain, 83–90 (2007b)

    Google Scholar 

  • D. Friedman, A. Donenfeld, E. Zafran, Neurophysiology-based art in immersive virtual reality. Int. J. Arts Technol. 2(4), 331 (2009)

    Article  Google Scholar 

  • D. Friedman, R. Leeb, G. Pfurtscheller, M. Slater, Human–computer interface issues in controlling virtual reality with brain–computer interface. Hum. Comput. Interact. 25(1), 67–94 (2010)

    Article  Google Scholar 

  • S.W. Gilroy, J. Porteous, F. Charles, M. Cavazza, E. Soreq, G. Raz, … T. Hendler, A brain-computer interface to a plan-based narrative, 1997–2005. (2013). Retrieved from http://dl.acm.org/citation.cfm?id=2540128.2540415

  • J. Giron, D. Friedman, Eureka: realizing that an application is responding to your brainwaves, in Universal Access in Human-Computer Interaction. Design and Development Methods for Universal Access (2014), Springer International Publishing, Crete, Greece, pp. 495–502

    Google Scholar 

  • J. Giron, M. Segal, D. Friedman, Implicit learning of SSVEP based brain computer interface, in The 6th International Brain-Computer Interface Conference, (Graz, 2014)

    Google Scholar 

  • C. Groenegress, C. Holzner, C. Guger, M. Slater, Effects of P300-based BCI use on reported presence in a virtual environment. Presence Teleop. Virt. Environ. 19(1), 1–11 (2010). doi:10.1162/pres.19.1.1

    Article  Google Scholar 

  • C. Guger, G. Edlinger, W. Harkam, I. Niedermayer, G. Pfurtscheller, How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans. Neural Syst. Rehabil. Eng. 11, 145–147 (2003)

    Google Scholar 

  • T. Harmelech, D. Friedman, R. Malach, Differential magnetic resonance neurofeedback modulations across extrinsic (visual) and intrinsic (default-mode) nodes of the human cortex. J. Neurosci. 35(6), 2588–2595 (2015)

    Google Scholar 

  • C. Heeter, Being there: the subjective experience of presence. Presence Teleop. Virt. Environ. 1(2), 262–271 (1992)

    Article  Google Scholar 

  • L.R. Hochberg, D. Bacher, B. Jarosiewicz, N.Y. Masse, J.D. Simeral, J. Vogel, … van der P. Smagt, Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398), 372–375 (2012)

    Google Scholar 

  • J.E. Huggins, J.R. Wolpaw, Papers from the fifth international brain-computer interface meeting. Preface. J. Neural Eng. 11, 030301 (2014). doi:10.1088/1741-2560/11/3/030301

    Article  Google Scholar 

  • I.E.E.S. Sutherland, The ultimate display, in Proceedings of the {IFIPS} Congress 1965 New York: IFIP, 65(2), 506–508 (1965)

    Google Scholar 

  • D.J. Krusienski, M. Grosse-Wentrup, F. Galán, D. Coyle, K.J. Miller, E. Forney, C.W. Anderson, Critical issues in state-of-the-art brain-computer interface signal processing. J. Neural Eng. 8, 025002 (2011). doi:10.1088/1741-2560/8/2/025002

    Article  Google Scholar 

  • E. Lalor, S.P. Kelly, C. Finucane, R. Burke, R. Smith, R.B. Reilly, G. McDarby, Steady-state VEP-based brain computer interface control in an immersive 3-D gaming environment. EURASIP JASP 19, 3156–3164 (2005)

    MATH  Google Scholar 

  • F. Larrue, H. Sauzéon, L. Aguilova, F. Lotte, M. Hachet, B.N. Kaoua, Brain computer interface vs walking interface in VR: the impact of motor activity on spatial transfer, in Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST (2012), pp. 113–120. doi:10.1145/2407336.2407359

    Google Scholar 

  • R. Leeb, C. Keinrath, D. Friedman, C. Guger, C. Neuper, M. Garau, … G. Pfurtscheller, Walking from thoughts: not the muscles are crucial, but the brain waves! Presence Teleop. Virt. Environ. (2006). doi:10.1155/2007/79642

    Google Scholar 

  • R. Leeb, D. Friedman, G.R. Müller-Putz, R. Scherer, M. Slater, G. Pfurtscheller, Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic. Comput. Intell. Neurosci. (2007a)

    Google Scholar 

  • R. Leeb, F. Lee, C. Keinrath, R. Scherer, H. Bischof, G. Pfurtscheller, Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment. IEEE Trans. Neural Syst. Rehabil. Eng. 15(4), 473–482 (2007b). doi:10.1109/TNSRE.2007.906956

    Article  Google Scholar 

  • R. Leeb, V. Settgast, D. Fellner, G. Pfurtscheller, Self-paced exploration of the Austrian National Library through thought. Int. J. Bioelectromagn. 9, 237–244 (2007c)

    Google Scholar 

  • R. Leeb, M. Lancelle, V. Kaiser, D. Fellner, G. Pfurtscheller, Thinking penguin: multimodal brain-computer interface control of a VR game. IEEE Trans. Comput. Intell. AI Games 5, 117–128 (2013). doi:10.1109/TCIAIG.2013.2242072

    Article  Google Scholar 

  • J. Legény, R.V. Abad, A. Lécuyer, Navigating in virtual worlds using a self-paced SSVEP-based brain–computer interface with integrated stimulation and real-time feedback. Presence Teleop. Virt. Environ. 20(6), 529–544 (2011). doi:10.1162/PRES_a_00075

    Article  Google Scholar 

  • Y. Liu, O. Sourina, M.K. Nguyen, Real-time EEG-based emotion recognition and its applications, in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNCS, vol 6670 (2011), pp. 256–277. doi:10.1007/978-3-642-22336-5_13

    Google Scholar 

  • Y. Liu, X. Jiang, T. Cao, F. Wan, P.U. Mak, P.I. Mak, M.I. Vai, Implementation of SSVEP based BCI with Emotiv EPOC, in Proceedings of IEEE International Conference on Virtual Environments, Human-Computer Interfaces, and Measurement Systems, VECIMS (2012), pp 34–37. doi:10.1109/VECIMS.2012.6273184

    Google Scholar 

  • M. Lombard, T.B. Ditton, At the heart of it all: the concept of presence. J. Comput. Mediat. Commun. 3(2), 20 pp (1997). doi:10.1093/cid/cir583

    Google Scholar 

  • F. Lotte, A. van Langhenhove, F. Lamarche, T. Ernest, Y. Renard, B. Arnaldi, A. Lécuyer, Exploring large virtual environments by thoughts using a brain–computer interface based on motor imagery and high-level commands. Presence Teleop. Virt. Environ. 19(1), 54–70 (2010). doi:10.1162/pres.19.1.54

    Article  Google Scholar 

  • D.P. Marcos, H. Ehrsson, M.V. Sanchez, Inducing illusory ownership of a virtual body. Front. Neurosci. 3(2), 214–220 (2009)

    Article  Google Scholar 

  • A. Maselli, M. Slater, The building blocks of the full body ownership illusion. Front. Hum. Neurosci. 7, 83 (2013). doi:10.3389/fnhum.2013.00083

    Article  Google Scholar 

  • C.C. Neira, D.J. Sandin, T.A. DeFanti, R.V. Kenyon, J.C. Hart, The CAVE: audio visual experience automatic virtual environment. Comm. ACM 35(6), 65–72 (1992).

    Google Scholar 

  • W. Nelson, L. Hettinger, J. Cunningham, M.R. Nelson, Navigating through virtual flight environments using brain-body-actuated control, in Proceedings of IEEE Virtual Reality Annual International Symposium (1997), pp. 30–37

    Google Scholar 

  • G. Pfurtscheller, R. Leeb, C. Keinrath, D. Friedman, C. Neuper, C. Guger, M. Slater, Walking from thought. Brain Res. 1071(1), 145–152 (2006)

    Article  Google Scholar 

  • G. Pfurtscheller, R. Leeb, D. Friedman, M. Slater, Centrally controlled heart rate changes during mental practice in immersive virtual environment: a case study with a tetraplegic. Int. J. Psychophysiol. 68(1), 1–5 (2008). doi:10.1016/j.ijpsycho.2007.11.003

    Article  Google Scholar 

  • R.W. Picard, Affective computing. Pattern Recogn. 73. (1997) doi:10.1007/BF01238028

    Google Scholar 

  • D. Plass-Oude Bos, B. Reuderink, B. Laar, H. Gürkök, C. Mühl, M. Poel, … D. Heylen, Brain-computer interfacing and games, in Brain-Computer Interfaces (2010), pp. 149–178. doi:10.1007/978-1-84996-272-8

    Google Scholar 

  • H. Putnam, Brain in a vat, in Reason, Truth, and History (1982), Cambridge University Press, pp. 5–8

    Google Scholar 

  • J.R. Quinlan, Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)

    Google Scholar 

  • Y. Renard, F. Lotte, G. Gibert, M. Congedo, E. Maby, V. Delannoy, … A. Lécuyer, OpenViBE: an open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. Presence Teleop. Virt. Environ. (2010). doi:10.1162/pres.19.1.35

    Google Scholar 

  • M.V. Sanchez-Vives, M. Slater, From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6(4), 332–339 (2005)

    Google Scholar 

  • R. Scherer, F. Lee, A. Schlogl, R. Leeb, H. Bischof, G. Pfurtscheller, Toward self-paced brain–computer communication: navigation through virtual worlds. IEEE Trans. Biomed. Eng. 55(2), 675–682 (2008)

    Article  Google Scholar 

  • M. Slater, Presence in immersive virtual environments, in Proceedings of the IEEE Virtual Reality 1993 (Seattle, 1993)

    Google Scholar 

  • M. Slater, Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. 364(1535), 3549–3557 (2009)

    Article  Google Scholar 

  • M. Slater, D. Perez-Marcos, H.H. Ehrsson, M.V. Sanchez-Vives, Towards a digital body: the virtual arm illusion. Front. Hum. Neurosci. 2, 6 (2008). doi:10.3389/neuro.09.006.2008

    Article  Google Scholar 

  • M. Slater, D. Perez-Marcos, H.H. Ehrsson, M.V. Sanchez-Vives, Inducing illusory ownership of a virtual body. Front. Neurosci. 3(2), 214 (2009)

    Article  Google Scholar 

  • Y. Su, Y. Qi, J. Luo, B. Wu, F. Yang, Y. Li, … W. Chen, A hybrid brain-computer interface control strategy in a virtual environment. J. Zhejiang Univ. Sci. C (2011). doi:10.1631/jzus.C1000208

    Google Scholar 

  • M. Van Gerven, J. Farquhar, R. Schaefer, R. Vlek, J. Geuze, A. Nijholt, … P. Desain, The brain-computer interface cycle. J. Neural Eng. 6, 041001 (2009). doi:10.1088/1741-2560/6/4/041001

    Google Scholar 

  • B.G. Witmer, M.J. Singer, Measuring presence in virtual environments: a presence questionnaire. Telepresence Teleop. Virt. Environ. 7(3), 225–240 (1998)

    Article  Google Scholar 

  • T.O. Zander, C. Kothe, Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J. Neural Eng. 8(2), 25005 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Advanced Reality Lab, The Interdisciplinary Center, Herzliya, Israel

    Doron Friedman

Authors
  1. Doron Friedman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Doron Friedman .

Editor information

Editors and Affiliations

  1. Design School, Kyoto University, Kyoto, Japan

    Ryohei Nakatsu

  2. Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands

    Matthias Rauterberg

  3. Dipto. Scienze dell'Informazione, Università di Bologna , Bologna, Italy

    Paolo Ciancarini

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Singapore

About this entry

Cite this entry

Friedman, D. (2017). Brain-Computer Interfacing and Virtual Reality. In: Nakatsu, R., Rauterberg, M., Ciancarini, P. (eds) Handbook of Digital Games and Entertainment Technologies. Springer, Singapore. https://doi.org/10.1007/978-981-4560-50-4_2

Download citation

Publish with us

Policies and ethics

Search

Navigation