Skip to main content
SpringerLink
Log in

Designing Touch: Intracortical Neurohaptic Feedback in Virtual Reality

  • Chapter
  • First Online:
Brain-Computer Interface Research

Part of the book series: SpringerBriefs in Electrical and Computer Engineering ((BRIEFSELECTRIC))

  • 388 Accesses

Abstract

Touch sensation offers complex and innate biological feedback that underlies dexterous interactions with our environments, making the restoration of touch an important component of functional brain computer interface (BCI). To advance synthetic touch research, I developed a virtual reality (VR) platform for use with human patients receiving intracranial neural implants for epilepsy evaluation (Fig. 1) [Paschall et al. (An immersive virtual reality platform integrating human ECOG & sEEG: implementation & noise analysis, 2022]. Following validation, noise, and user studies with this VR research platform, I designed a BCI experiment in which tactile information about contact with a virtual object was delivered to the brain through direct cortical stimulation (DCS). I recoined the term neurohaptics to describe this use of DCS for haptic feedback [Paschall et al. (2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2022)]. I also developed a new way to deliver neurohaptic feedback using amplitude modulated DCS to evoke distinct and discriminable sensory percepts that better mimicked natural touch sensation [Paschall et al. (2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2022)]. This work demonstrates the intrinsic interpretability of dynamic temporal structure in DCS and its utility in the restoration of touch sensation. It paves the way for further first-person VR-BCI neurohaptic design and brings a novel bidirectional BCI and cognitive neuroscience platform into the clinic for work with implanted neural devices in patient populations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cronin JA et al (2016) Task-specific somatosensory feedback via cortical stimulation in humans. IEEE Trans Haptics 9:515–522

    Article  Google Scholar 

  2. Flesher SN et al (2021) A brain-computer interface that evokes tactile sensations improves robotic arm control. Science 372:831–836

    Article  Google Scholar 

  3. Paschall C, Rao RPN, Hauptman J, Ojemann G, Herron J (2022) An immersive virtual reality platform integrating human ECOG & sEEG: implementation & noise analysis

    Google Scholar 

  4. Paschall CJ, Hauptman JS, Rao RPN, Ojemann JG, Herron J (2022) Touching the void: intracranial stimulation for NeuroHaptic feedback in virtual reality. In: 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE. https://doi.org/10.1109/smc53654.2022.9945366

    Chapter  Google Scholar 

  5. Collinger JL et al (2013) High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381:557–564

    Article  Google Scholar 

  6. Ajiboye AB et al (2017) Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. Lancet 389:1821–1830

    Article  Google Scholar 

  7. Ethier C, Oby ER, Bauman MJ, Miller LE (2012) Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485:368–371

    Article  Google Scholar 

  8. Collinger JL, Gaunt RA, Schwartz AB (2018) Progress towards restoring upper limb movement and sensation through intracortical brain-computer interfaces. Curr Opin Biomed Eng 8:84–92

    Article  Google Scholar 

  9. Paralysis statistics. Reeve Foundation. https://www.christopherreeve.org/living-with-paralysis/stats-about-paralysis.

  10. Bensmaia SJ, Miller LE (2014) Restoring sensorimotor function through intracortical interfaces: progress and looming challenges. Nat Rev Neurosci 15:313–325

    Article  Google Scholar 

  11. O’Doherty JE et al (2011) Active tactile exploration using a brain-machine-brain interface. Nature 479:228–231

    Article  Google Scholar 

  12. Armenta Salas M et al (2018) Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation. Elife 7

    Google Scholar 

  13. O’Doherty JE, Lebedev MA, Li Z, Nicolelis MAL (2012) Virtual active touch using randomly patterned intracortical microstimulation. IEEE Trans Neural Syst Rehabil Eng 20:85–93

    Article  Google Scholar 

  14. Caldwell DJ, Cronin JA, Levinson LH, Rao PN (2021) Chapter 14 – Touch restoration through electrical cortical stimulation in humans. In: Güçlü B (ed) Somatosensory feedback for neuroprosthetics. Academic Press, pp 443–478. https://doi.org/10.1016/B978-0-12-822828-9.00021-6

    Chapter  Google Scholar 

  15. Cronin, J. A. Human psychophysics of direct cortical stimulation of somatosensory cortex. (2018).

    Google Scholar 

  16. Libet B et al (1964) Production of threshold levels of conscious sensation by electrical stimulation of human somatosensory cortex. J Neurophysiol 27:546–578

    Article  Google Scholar 

  17. Johnson LA et al (2013) Direct electrical stimulation of the somatosensory cortex in humans using electrocorticography electrodes: a qualitative and quantitative report. J Neural Eng 10:036021

    Article  Google Scholar 

  18. Leuthardt EC, Miller KJ, Schalk G, Rao RPN, Ojemann JG (2006) Electrocorticography-based brain computer interface–the Seattle experience. IEEE Trans Neural Syst Rehabil Eng 14:194–198

    Article  Google Scholar 

  19. Libet B et al (1993) Production of threshold levels of conscious sensation by electrical stimulation of human somatosensory cortex. In: Libet B (ed) Neurophysiology of consciousness. Birkhäuser, Boston, pp 1–34. https://doi.org/10.1007/978-1-4612-0355-1_1

    Chapter  Google Scholar 

  20. Hiremath SV et al (2017) Human perception of electrical stimulation on the surface of somatosensory cortex. PLoS One 12:e0176020

    Article  Google Scholar 

  21. Caldwell DJ et al (2019) Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans. Sci Rep 9:3292

    Article  Google Scholar 

  22. Caldwell DJ et al (2019) Publisher Correction: Direct stimulation of somatosensory cortex results in slower reaction times compared to peripheral touch in humans. Sci Rep 9:20317

    Article  Google Scholar 

  23. Bjånes DA et al (2022) Multi-channel intra-cortical micro-stimulation yields quick reaction times and evokes natural somatosensations in a human participant. bioRxiv. https://doi.org/10.1101/2022.08.08.22278389

  24. O’Doherty JE, Shokur S, Medina LE, Lebedev MA, Nicolelis MAL (2019) Creating a neuroprosthesis for active tactile exploration of textures. Proc Natl Acad Sci U S A 116:21821–21827

    Article  Google Scholar 

  25. Tauscher J-P et al (2019) Immersive EEG: evaluating electroencephalography in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp 1794–1800. https://doi.org/10.1109/VR.2019.8797858

    Chapter  Google Scholar 

  26. Georgiev DD, Georgieva I, Gong Z, Nanjappan V, Georgiev GV (2021) Virtual reality for neurorehabilitation and cognitive enhancement. Brain Sci 11

    Google Scholar 

  27. Topalovic U et al (2020) Wireless programmable recording and stimulation of deep brain activity in freely moving humans. Neuron 108:322–334.e9

    Article  Google Scholar 

  28. Caldwell DJ, Ojemann JG, Rao RPN (2019) Direct electrical stimulation in electrocorticographic brain–computer interfaces: enabling technologies for input to cortex. Front Neurosci

    Google Scholar 

  29. Montag M, Paschall C, Ojemann J, Rao R, Herron J (2021) A platform for virtual reality task design with intracranial electrodes. Conf Proc IEEE Eng Med Biol Soc 2021:6659–6662

    Google Scholar 

  30. Cogan SF, Ludwig KA, Welle CG, Takmakov P (2016) Tissue damage thresholds during therapeutic electrical stimulation. J Neural Eng 13:021001

    Article  Google Scholar 

  31. Vogels IMLC (2004) Detection of temporal delays in visual-haptic interfaces. Hum Factors 46:118–134

    Article  Google Scholar 

Download references

Acknowledgement

We acknowledge and extend our gratitude to the patients who chose to work with us—in these and many other research efforts. Thank you!

Author information

Authors and Affiliations

  1. Department of Bioengineering, University of Washington, Washington, DC, USA

    Courtnie J. Paschall

  2. Seattle Children’s Hospital, Neurological Surgery, Seattle, WA, USA

    Jason S. Hauptman & Jeffrey G. Ojemann

  3. Paul G. Allen School of Computer Science, University of Washington, Washington, DC, USA

    Rajesh P. N. Rao

  4. Department of Neurological Surgery, University of Washington, Washington, DC, USA

    Jeffrey G. Ojemann & Jeffrey Herron

Authors
  1. Courtnie J. Paschall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jason S. Hauptman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajesh P. N. Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeffrey G. Ojemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeffrey Herron
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Courtnie J. Paschall .

Editor information

Editors and Affiliations

  1. g.tec medical engineering GmbH, Schiedlberg, Oberösterreich, Austria

    Christoph Guger

  2. Cognitive Science Department, UC San Diego, La Jolla, CA, USA

    Brendan Allison

  3. RIKEN AIP, Tokyo Chuo-ku, Japan

    Tomasz M. Rutkowski

  4. The Institute of Neuroapproaches, Winter Springs, FL, USA

    Milena Korostenskaja

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Paschall, C.J., Hauptman, J.S., Rao, R.P.N., Ojemann, J.G., Herron, J. (2024). Designing Touch: Intracortical Neurohaptic Feedback in Virtual Reality. In: Guger, C., Allison, B., Rutkowski, T.M., Korostenskaja, M. (eds) Brain-Computer Interface Research. SpringerBriefs in Electrical and Computer Engineering. Springer, Cham. https://doi.org/10.1007/978-3-031-49457-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49457-4_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49456-7

  • Online ISBN: 978-3-031-49457-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation