Skip to main content

Advertisement

SpringerLink
Log in

Sensory integration for neuroprostheses: from functional benefits to neural correlates

  • Review Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

In the field of sensory neuroprostheses, one ultimate goal is for individuals to perceive artificial somatosensory information and use the prosthesis with high complexity that resembles an intact system. To this end, research has shown that stimulation-elicited somatosensory information improves prosthesis perception and task performance. While studies strive to achieve sensory integration, a crucial phenomenon that entails naturalistic interaction with the environment, this topic has not been commensurately reviewed. Therefore, here we present a perspective for understanding sensory integration in neuroprostheses. First, we review the engineering aspects and functional outcomes in sensory neuroprosthesis studies. In this context, we summarize studies that have suggested sensory integration. We focus on how they have used stimulation-elicited percepts to maximize and improve the reliability of somatosensory information. Next, we review studies that have suggested multisensory integration. These works have demonstrated that congruent and simultaneous multisensory inputs provided cognitive benefits such that an individual experiences a greater sense of authority over prosthesis movements (i.e., agency) and perceives the prosthesis as part of their own (i.e., ownership). Thereafter, we present the theoretical and neuroscience framework of sensory integration. We investigate how behavioral models and neural recordings have been applied in the context of sensory integration. Sensory integration models developed from intact-limb individuals have led the way to sensory neuroprosthesis studies to demonstrate multisensory integration. Neural recordings have been used to show how multisensory inputs are processed across cortical areas. Lastly, we discuss some ongoing research and challenges in achieving and understanding sensory integration in sensory neuroprostheses. Resolving these challenges would help to develop future strategies to improve the sensory feedback of a neuroprosthetic system.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

PNS:

Peripheral nervous system

CNS:

Central nervous system

ICMS:

Intracortical microstimulation

MEA:

Microelectrode array

TENS:

Transcutaneous electrical nerve stimulation

AMI:

Agonist–antagonist myoneural interface

BBT:

Box and blocks test

ADL:

Activity of daily living

BCI:

Brain-computer interface

HMD:

Head mounted display

JND:

Just noticeable difference

2AFC:

Two-alternative forced-choice

MLE:

Maximum likelihood estimation

fMRI:

Functional magnetic resonance imaging

TMSR:

Targeted muscle and sensory reinnervation

EEG:

Electroencephalogram

M1:

Primary motor cortex

S1:

Primary somatosensory cortex

MS:

Multiple sclerosis

FEM:

Finite element modeling

References

  1. Abraira VE, Ginty DD (2013) The sensory neurons of touch. Neuron 79:618–639. https://doi.org/10.1016/j.neuron.2013.07.051

    Article  CAS  PubMed  Google Scholar 

  2. Armenta Salas M, Bashford L, Kellis S, Jafari M, Jo H, Kramer D, Shanfield K, Pejsa K, Lee B, Liu CY, Andersen RA (2018) Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation. eLife 7:e32904. https://doi.org/10.7554/eLife.32904

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bensmaia SJ, Tyler DJ, Micera S (2020) Restoration of sensory information via bionic hands. Nat Biomed Eng 7:1–13. https://doi.org/10.1038/s41551-020-00630-8

    Article  Google Scholar 

  4. Birznieks I, McIntyre S, Nilsson HM, Nagi SS, Macefield VG, Mahns DA, Vickery RM (2019) Tactile sensory channels over-ruled by frequency decoding system that utilizes spike pattern regardless of receptor type. eLife 8:e46510. https://doi.org/10.7554/eLife.46510

    Article  PubMed  PubMed Central  Google Scholar 

  5. Callier T, Suresh AK, Bensmaia SJ (2019) Neural coding of contact events in somatosensory cortex. Cereb Cortex 29:4613–4627. https://doi.org/10.1093/cercor/bhy337

    Article  PubMed  PubMed Central  Google Scholar 

  6. Caspar EA, Cleeremans A, Haggard P (2015) The relationship between human agency and embodiment. Conscious Cogn 33:226–236. https://doi.org/10.1016/j.concog.2015.01.007

    Article  PubMed  Google Scholar 

  7. Chan AWY, Baker CI (2015) Seeing is not feeling: posterior parietal but not somatosensory cortex engagement during touch observation. J Neurosci 35:1468–1480. https://doi.org/10.1523/JNEUROSCI.3621-14.2015

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Chandrasekaran C (2017) Computational principles and models of multisensory integration. Curr Opin Neurobiol 43:25–34. https://doi.org/10.1016/j.conb.2016.11.002

    Article  CAS  PubMed  Google Scholar 

  9. Chandrasekaran S, Nanivadekar AC, McKernan G, Helm ER, Boninger ML, Collinger JL, Gaunt RA, Fisher LE (2020) Sensory restoration by epidural stimulation of the lateral spinal cord in upper-limb amputees. eLife 9:e54349. https://doi.org/10.7554/eLife.54349

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Charkhkar H, Christie BP, Triolo RJ (2020) Sensory neuroprosthesis improves postural stability during sensory organization test in lower-limb amputees. Sci Rep 10:6984. https://doi.org/10.1038/s41598-020-63936-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Chee L, Valle G, Marazzi M, Preatoni G, Haufe FL, Xiloyannis M, Riener R, Raspopovic S (2022) Optimally-calibrated non-invasive feedback improves amputees’ metabolic consumption, balance and walking confidence. J Neural Eng 19:046049. https://doi.org/10.1088/1741-2552/ac883b

    Article  Google Scholar 

  12. Christie B, Osborn LE, McMullen DP, Pawar AS, Thomas TM, Bensmaia SJ, Celnik PA, Fifer MS, Tenore FV (2022) Perceived timing of cutaneous vibration and intracortical microstimulation of human somatosensory cortex. Brain Stimul 15:881–888. https://doi.org/10.1016/j.brs.2022.05.015

    Article  PubMed  Google Scholar 

  13. Christie BP, Charkhkar H, Shell CE, Burant CJ, Tyler DJ, Triolo RJ (2020) Ambulatory searching task reveals importance of somatosensation for lower-limb amputees. Sci Rep 10:10216. https://doi.org/10.1038/s41598-020-67032-3

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Christie BP, Charkhkar H, Shell CE, Marasco PD, Tyler DJ, Triolo RJ (2019) Visual inputs and postural manipulations affect the location of somatosensory percepts elicited by electrical stimulation. Sci Rep 9:11699. https://doi.org/10.1038/s41598-019-47867-1

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Christie BP, Graczyk EL, Charkhkar H, Tyler DJ, Triolo RJ (2019) Visuotactile synchrony of stimulation-induced sensation and natural somatosensation. J Neural Eng 16:036025. https://doi.org/10.1088/1741-2552/ab154c

    Article  PubMed  Google Scholar 

  16. Cimolato A, Ciotti F, Kljajić J, Valle G, Raspopovic S (2023) Symbiotic electroneural and musculoskeletal framework to encode proprioception via neurostimulation: ProprioStim. iScience 26:106248. https://doi.org/10.1016/j.isci.2023.106248

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Clites TR, Carty MJ, Ullauri JB, Carney ME, Mooney LM, Duval JF, Srinivasan SS, Herr HM (2018) Proprioception from a neurally controlled lower-extremity prosthesis. Sci Transl Med 10:eaap8373. https://doi.org/10.1126/scitranslmed.aap8373

    Article  PubMed  Google Scholar 

  18. Corniani G, Casal MA, Panzeri S, Saal HP (2022) Population coding strategies in human tactile afferents. PLOS Comput Biol 18:e1010763. https://doi.org/10.1371/journal.pcbi.1010763

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. D’Anna E, Petrini FM, Artoni F, Popovic I, Simanić I, Raspopovic S, Micera S (2017) A somatotopic bidirectional hand prosthesis with transcutaneous electrical nerve stimulation based sensory feedback. Sci Rep 7:10930. https://doi.org/10.1038/s41598-017-11306-w

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. D’Anna E, Valle G, Mazzoni A, Strauss I, Iberite F, Patton J, Petrini FM, Raspopovic S, Granata G, Iorio RD, Controzzi M, Cipriani C, Stieglitz T, Rossini PM, Micera S (2019) A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Sci Robot 4:eaau8892. https://doi.org/10.1126/scirobotics.aau8892

    Article  PubMed  Google Scholar 

  21. Davis TS, Wark HAC, Hutchinson DT, Warren DJ, O’Neill K, Scheinblum T, Clark GA, Normann RA, Greger B (2016) Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J Neural Eng 13:036001. https://doi.org/10.1088/1741-2560/13/3/036001

    Article  CAS  PubMed  Google Scholar 

  22. Delp SL, Anderson FC, Arnold AS, Loan P, Habib A, John CT, Guendelman E, Thelen DG (2007) OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Trans Biomed Eng 54:1940–1950. https://doi.org/10.1109/TBME.2007.901024

    Article  PubMed  Google Scholar 

  23. Ding K, Chen Y, Bose R, Osborn LE, Dragomir A, Thakor NV (2022) Sensory stimulation for upper limb amputations modulates adaptability of cortical large-scale systems and combination of somatosensory and visual inputs. Sci Rep 12:20467. https://doi.org/10.1038/s41598-022-24368-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Ding K, Dragomir A, Bose R, Osborn LE, Seet MS, Bezerianos A, Thakor NV (2020) Towards machine to brain interfaces: sensory stimulation enhances sensorimotor dynamic functional connectivity in upper limb amputees. J Neural Eng 17:035002. https://doi.org/10.1088/1741-2552/ab882d

    Article  PubMed  Google Scholar 

  25. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23. https://doi.org/10.1016/j.neuron.2007.12.013

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Ehrsson HH (2020) Chapter 8 - Multisensory processes in body ownership. In: Sathian K, Ramachandran VS (eds) Multisensory perception, pp 179–200. Academic Press. https://doi.org/10.1016/B978-0-12-812492-5.00008-5

  27. Emanuel AJ, Lehnert BP, Panzeri S, Harvey CD, Ginty DD (2021) Cortical responses to touch reflect subcortical integration of LTMR signals. Nature 600:680–685. https://doi.org/10.1038/s41586-021-04094-x

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433. https://doi.org/10.1038/415429a

    Article  CAS  PubMed  Google Scholar 

  29. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169. https://doi.org/10.1016/j.tics.2004.02.002

    Article  PubMed  Google Scholar 

  30. Fifer MS, McMullen DP, Osborn LE, Thomas TM, Christie B, Nickl RW, Candrea DN, Pohlmeyer EA, Thompson MC, Anaya MA, Schellekens W, Ramsey NF, Bensmaia SJ, Anderson WS, Wester BA, Crone NE, Celnik PA, Cantarero GL, Tenore FV (2022) Intracortical somatosensory stimulation to elicit fingertip sensations in an individual with spinal cord injury. Neurology 98:e679–e687. https://doi.org/10.1212/WNL.0000000000013173

    Article  PubMed  PubMed Central  Google Scholar 

  31. Flesher SN, Collinger JL, Foldes ST, Weiss JM, Downey JE, Tyler-Kabara EC, Bensmaia SJ, Schwartz AB, Boninger ML, Gaunt RA (2016) Intracortical microstimulation of human somatosensory cortex. Sci Transl Med 8:361ra141. https://doi.org/10.1126/scitranslmed.aaf8083

    Article  PubMed  Google Scholar 

  32. Gentile G, Petkova VI, Ehrsson HH (2010) Integration of visual and tactile signals from the hand in the human brain: an fMRI study. J Neurophysiol 105:910–922. https://doi.org/10.1152/jn.00840.2010

    Article  PubMed  PubMed Central  Google Scholar 

  33. George JA, Kluger DT, Davis TS, Wendelken SM, Okorokova EV, He Q, Duncan CC, Hutchinson DT, Thumser ZC, Beckler DT, Marasco PD, Bensmaia SJ, Clark GA (2019) Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand. Sci Robot 4:eaax2352. https://doi.org/10.1126/scirobotics.aax2352

    Article  PubMed  Google Scholar 

  34. Gonzalez M, Bismuth A, Lee C, Chestek CA, Gates DH (2022) Artificial referred sensation in upper and lower limb prosthesis users: a systematic review. J Neural Eng 19:051001. https://doi.org/10.1088/1741-2552/ac8c38

    Article  Google Scholar 

  35. Graczyk EL, Christie BP, He Q, Tyler DJ, Bensmaia SJ (2022) Frequency shapes the quality of tactile percepts evoked through electrical stimulation of the nerves. J Neurosci 42:2052–2064. https://doi.org/10.1523/JNEUROSCI.1494-21.2021

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Graczyk EL, Schiefer MA, Saal HP, Delhaye BP, Bensmaia SJ, Tyler DJ (2016) The neural basis of perceived intensity in natural and artificial touch. Sci Transl Med 8:362ra142-362ra. https://doi.org/10.1126/scitranslmed.aaf5187

    Article  PubMed  PubMed Central  Google Scholar 

  37. Greenspon CM, Valle G, Hobbs TG, Verbaarschot C, Callier T, Okorokova EV, Shelchkova ND, Sobinov AR, Jordan PM, Weiss JM, Fitzgerald EE, Prasad D, Driesche Av, Lee RC, Satzer D, Gonzalez-Martinez J, Warnke PC, Miller LE, Boninger ML, Collinger JL, Gaunt RA, Downey JE, Hatsopoulos NG, Bensmaia SJ (2023) Biomimetic multi-channel microstimulation of somatosensory cortex conveys high resolution force feedback for bionic hands. bioRxiv. https://doi.org/10.1101/2023.02.18.528972

  38. Gupta A, Vardalakis N, Wagner FB (2023) Neuroprosthetics: from sensorimotor to cognitive disorders. Commun Biol 6:1–17. https://doi.org/10.1038/s42003-022-04390-w

    Article  Google Scholar 

  39. Haggard P, Clark S, Kalogeras J (2002) Voluntary action and conscious awareness. Nat Neurosci 5:382–385. https://doi.org/10.1038/nn827

    Article  CAS  PubMed  Google Scholar 

  40. Hines ML, Carnevale NT (1997) The NEURON simulation environment. Neural Comput 9:1179–1209. https://doi.org/10.1162/neco.1997.9.6.1179

    Article  CAS  PubMed  Google Scholar 

  41. Horch K, Meek S, Taylor TG, Hutchinson DT (2011) Object discrimination with an artificial hand using electrical stimulation of peripheral tactile and proprioceptive pathways with intrafascicular electrodes. IEEE Trans Neural Syst Rehabil Eng 19:483–489. https://doi.org/10.1109/TNSRE.2011.2162635

    Article  PubMed  Google Scholar 

  42. Iberite F, Muheim J, Akouissi O, Gallo S, Rognini G, Morosato F, Clerc A, Kalff M, Gruppioni E, Micera S, Shokur S (2023) Restoration of natural thermal sensation in upper-limb amputees. Science 380:731–735. https://doi.org/10.1126/science.adf6121

    Article  CAS  PubMed  Google Scholar 

  43. Iskarous MM, Thakor NV (2019) E-skins: biomimetic sensing and encoding for upper limb prostheses. Proc IEEE 107:2052–2064. https://doi.org/10.1109/JPROC.2019.2939369

    Article  Google Scholar 

  44. Johnson KO (2001) The roles and functions of cutaneous mechanoreceptors. Curr Opin Neurobiol 11(4):455–461. https://doi.org/10.1016/S0959-4388(00)00234-8

    Article  CAS  PubMed  Google Scholar 

  45. Katic N, Siqueira RK, Cleland L, Strzalkowski N, Bent L, Raspopovic S, Saal H (2023) Modeling foot sole cutaneous afferents: FootSim. iScience 26:105874. https://doi.org/10.1016/j.isci.2022.105874

    Article  PubMed  Google Scholar 

  46. Kim D, Triolo R, Charkhkar H (2023) Plantar somatosensory restoration enhances gait, speed perception, and motor adaptation. Science Robotics 8:eadf8997. https://doi.org/10.1126/scirobotics.adf8997

    Article  PubMed  Google Scholar 

  47. Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA (2004) The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthet Orthot Int 28:245–253. https://doi.org/10.3109/03093640409167756

    Article  CAS  PubMed  Google Scholar 

  48. Kuiken TA, Marasco PD, Lock BA, Harden RN, Dewald JPA (2007) Redirection of cutaneous sensation from the hand to the chest skin of human amputees with targeted reinnervation. Proc Nat Acad Sci 104:20061–20066. https://doi.org/10.1073/pnas.0706525104

    Article  PubMed  PubMed Central  Google Scholar 

  49. Marasco PD, Hebert JS, Sensinger JW, Shell CE, Schofield JS, Thumser ZC, Nataraj R, Beckler DT, Dawson MR, Blustein DH, Gill S, Mensh BD, Granja-Vazquez R, Newcomb MD, Carey JP, Orzell BM (2018) Illusory movement perception improves motor control for prosthetic hands. Sci Transl Med 10:eaao6990. https://doi.org/10.1126/scitranslmed.aao6990

    Article  PubMed  PubMed Central  Google Scholar 

  50. Mathiowetz V, Volland G, Kashman N, Weber K (1985) Adult norms for the Box and Block Test of manual dexterity. Am J Occup Ther 39:386–391. https://doi.org/10.5014/ajot.39.6.386

    Article  CAS  PubMed  Google Scholar 

  51. Meijer D, Noppeney U (2020) Chapter 5 - Computational models of multisensory integration. In: Sathian K, Ramachandran VS (eds) Multisensory perception, pp 113–133. Academic Press. https://doi.org/10.1016/B978-0-12-812492-5.00005-X

  52. Nanivadekar AC, Bose R, Petersen BA, Okorokova EV, Sarma D, Madonna TJ, Barra B, Farooqui J, Dalrymple AN, Levy I, Helm ER, Miele VJ, Boninger ML, Capogrosso M, Bensmaia SJ, Weber DJ, Fisher LE (2023) Restoration of sensory feedback from the foot and reduction of phantom limb pain via closed-loop spinal cord stimulation. Nat Biomed Eng 1–12. https://doi.org/10.1038/s41551-023-01153-8

  53. Nanivadekar AC, Chandrasekaran S, Helm ER, Boninger ML, Collinger JL, Gaunt RA, Fisher LE (2022) Closed-loop stimulation of lateral cervical spinal cord in upper-limb amputees to enable sensory discrimination: a case study. Sci Rep 12:17002. https://doi.org/10.1038/s41598-022-21264-7

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Oddo, C.M., Raspopovic, S., Artoni, F., Mazzoni, A., Spigler, G., Petrini, F., Giambattistelli, F., Vecchio, F., Miraglia, F., Zollo, L., Di Pino, G., Camboni, D., Carrozza, M.C., Guglielmelli, E., Rossini, P.M., Faraguna, U., Micera, S.: Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. eLife 5, e09148 (2016). https://doi.org/10.7554/eLife.09148

  55. Okorokova EV, He Q, Bensmaia SJ (2018) Biomimetic encoding model for restoring touch in bionic hands through a nerve interface. J Neural Eng 15:066033. https://doi.org/10.1088/1741-2552/aae398

    Article  PubMed  PubMed Central  Google Scholar 

  56. Ortiz-Catalan M, Håkansson B, Brånemark R (2014) An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs. Sci Transl Med 6:257re6. https://doi.org/10.1126/scitranslmed.3008933

    Article  PubMed  Google Scholar 

  57. Ortiz-Catalan M, Mastinu E, Sassu P, Aszmann O, Brånemark R (2020) Self-contained neuromusculoskeletal arm prostheses. N Engl J Med 382:1732–1738. https://doi.org/10.1056/NEJMoa1917537

    Article  PubMed  Google Scholar 

  58. Osborn LE, Christie BP, McMullen DP, Nickl RW, Thompson MC, Pawar AS, Thomas TM, Alejandro Anaya M, Crone NE, Wester BA, Bensmaia SJ, Celnik PA, Cantarero GL, Tenore FV, Fifer MS (2021) Intracortical microstimulation of somatosensory cortex enables object identification through perceived sensations. In: 2021 43rd Annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 6259–6262. https://doi.org/10.1109/EMBC46164.2021.9630450

  59. Osborn LE, Ding K, Hays MA, Bose R, Iskarous MM, Dragomir A, Tayeb Z, Lévay GM, Hunt CL, Cheng G, Armiger RS, Bezerianos A, Fifer MS, Thakor NV (2020) Sensory stimulation enhances phantom limb perception and movement decoding. J Neural Eng 17:056006. https://doi.org/10.1088/1741-2552/abb861

    Article  PubMed  PubMed Central  Google Scholar 

  60. Osborn LE, Dragomir A, Betthauser JL, Hunt CL, Nguyen HH, Kaliki RR, Thakor NV (2018) Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain. Sci Robot 3:eaat3818. https://doi.org/10.1126/scirobotics.aat3818

    Article  Google Scholar 

  61. Osborn LE, Venkatasubramanian R, Himmtann M, Moran CW, Pierce JM, Gajendiran P, Wormley JM, Ung RJ, Nguyen HH, Crego ACG, Fifer MS, Armiger RS (2023) Evoking natural thermal perceptions using a thin-film thermoelectric device with high cooling power density and speed. Nat Biomed Eng 1–14. https://doi.org/10.1038/s41551-023-01070-w

  62. Page DM, George JA, Wendelken SM, Davis TS, Kluger DT, Hutchinson DT, Clark GA (2021) Discriminability of multiple cutaneous and proprioceptive hand percepts evoked by intraneural stimulation with Utah slanted electrode arrays in human amputees. J Neuroeng Rehabil 18:12. https://doi.org/10.1186/s12984-021-00808-4

    Article  PubMed  PubMed Central  Google Scholar 

  63. Pan L, Vargas L, Fleming A, Hu X, Zhu Y, Huang HH (2020) Evoking haptic sensations in the foot through high-density transcutaneous electrical nerve stimulations. J Neural Eng 17:036020. https://doi.org/10.1088/1741-2552/ab8e8d

    Article  PubMed  Google Scholar 

  64. Pandarinath C, Bensmaia SJ (2022) The science and engineering behind sensitized brain-controlled bionic hands. Physiol Rev 102:551–604. https://doi.org/10.1152/physrev.00034.2020

    Article  PubMed  Google Scholar 

  65. Petrini FM, Bumbasirevic M, Valle G, Ilic V, Mijović P, Čvančara P, Barberi F, Katic N, Bortolotti D, Andreu D, Lechler K, Lesic A, Mazic S, Mijović B, Guiraud D, Stieglitz T, Alexandersson A, Micera S, Raspopovic S (2019) Sensory feedback restoration in leg amputees improves walking speed, metabolic cost and phantom pain. Nat Med 25:1356–1363. https://doi.org/10.1038/s41591-019-0567-3

    Article  CAS  PubMed  Google Scholar 

  66. Petrini FM, Valle G, Bumbasirevic M, Barberi F, Bortolotti D, Cvancara P, Hiairrassary A, Mijovic P, Sverrisson AÖ, Pedrocchi A, Divoux JL, Popovic I, Lechler K, Mijovic B, Guiraud D, Stieglitz T, Alexandersson A, Micera S, Lesic A, Raspopovic S (2019) Enhancing functional abilities and cognitive integration of the lower limb prosthesis. Sci Transl Med 11:eaav8939. https://doi.org/10.1126/scitranslmed.aav8939

    Article  PubMed  Google Scholar 

  67. ...Petrini FM, Valle G, Strauss I, Granata G, Di Iorio R, D’Anna E, Čvančara P, Mueller M, Carpaneto J, Clemente F, Controzzi M, Bisoni L, Carboni C, Barbaro M, Iodice F, Andreu D, Hiairrassary A, Divoux JL, Cipriani C, Guiraud D, Raffo L, Fernandez E, Stieglitz T, Raspopovic S, Rossini PM, Micera S (2019) Six-month assessment of a hand prosthesis with intraneural tactile feedback. Ann Neurol 85:137–154. https://doi.org/10.1002/ana.25384

    Article  PubMed  Google Scholar 

  68. Pilger L, Berberich N, Paredes-Acuña N, Dendorfer A, Guadarrama-Olvera JR, Bergner F, Utpadel-Fischler D, Cheng G (2023) Human-centered design of a vibrotactile sensory substitution belt for feet somatosensation in a patient with multiple sclerosis. In: 2023 11th International IEEE/EMBS conference on neural engineering (NER), pp 1–4. https://doi.org/10.1109/NER52421.2023.10123871

  69. Ploumitsakou M, Muheim J, Felouzis A, Carbonell Muñoz NI, Iberite F, Akouissi O, Morosato F, Gruppioni E, Filingeri D, Micera S, Shokur S (2024) Remapping wetness perception in upper limb amputees. Adv Intell Syst 6:2300512. https://doi.org/10.1002/aisy.202300512

    Article  Google Scholar 

  70. Preatoni G, Valle G, Petrini FM, Raspopovic S (2021) Lightening the perceived prosthesis weight with neural embodiment promoted by sensory feedback. Curr Biol 31:1065-1071.e4. https://doi.org/10.1016/j.cub.2020.11.069

    Article  CAS  PubMed  Google Scholar 

  71. Raspopovic S, Capogrosso M, Micera S (2011) A computational model for the stimulation of rat sciatic nerve using a transverse intrafascicular multichannel electrode. IEEE Trans Neural Syst Rehabil Eng 19:333–344. https://doi.org/10.1109/TNSRE.2011.2151878

    Article  PubMed  Google Scholar 

  72. ...Raspopovic S, Capogrosso M, Petrini FM, Bonizzato M, Rigosa J, Pino GD, Carpaneto J, Controzzi M, Boretius T, Fernandez E, Granata G, Oddo CM, Citi L, Ciancio AL, Cipriani C, Carrozza MC, Jensen W, Guglielmelli E, Stieglitz T, Rossini PM, Micera S (2014) Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci Transl Med 6:222ra19-222ra19. https://doi.org/10.1126/scitranslmed.3006820

    Article  PubMed  Google Scholar 

  73. Raspopovic S, Valle G, Petrini FM (2021) Sensory feedback for limb prostheses in amputees. Nat Mater 20:925–939. https://doi.org/10.1038/s41563-021-00966-9

    Article  CAS  PubMed  Google Scholar 

  74. Risso G, Preatoni G, Valle G, Marazzi M, Bracher NM, Raspopovic S (2022) Multisensory stimulation decreases phantom limb distortions and is optimally integrated. iScience 25:104129. https://doi.org/10.1016/j.isci.2022.104129

    Article  PubMed  PubMed Central  Google Scholar 

  75. Risso G, Valle G (2022) Multisensory integration in bionics: relevance and perspectives. Curr Phys Med Rehabil Rep 10:123–130. https://doi.org/10.1007/s40141-022-00350-x

    Article  Google Scholar 

  76. Risso G, Valle G, Iberite F, Strauss I, Stieglitz T, Controzzi M, Clemente F, Granata G, Rossini PM, Micera S, Baud-Bovy G (2019) Optimal integration of intraneural somatosensory feedback with visual information: a single-case study. Sci Rep 9:7916. https://doi.org/10.1038/s41598-019-43815-1

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  77. Ro T, Wallace R, Hagedorn J, Farné A, Pienkos E (2004) Visual enhancing of tactile perception in the posterior parietal cortex. J Cogn Neurosci 16:24–30. https://doi.org/10.1162/089892904322755520

    Article  PubMed  Google Scholar 

  78. Roche AD, Bailey ZK, Gonzalez M, Vu PP, Chestek CA, Gates DH, Kemp SWP, Cederna PS, Ortiz-Catalan M, Aszmann OC (2023) Upper limb prostheses: bridging the sensory gap. J Hand Surg (Eur Vol) 48:182–190. https://doi.org/10.1177/17531934221131756

    Article  PubMed  Google Scholar 

  79. Rognini G, Petrini FM, Raspopovic S, Valle G, Granata G, Strauss I, Solcá M, Bello-Ruiz J, Herbelin B, Mange R, D’Anna E, Iorio RD, Pino GD, Andreu D, Guiraud D, Stieglitz T, Rossini PM, Serino A, Micera S, Blanke O (2019) Multisensory bionic limb to achieve prosthesis embodiment and reduce distorted phantom limb perceptions. J Neurol Neurosurg Psychiatry 90:833–836. https://doi.org/10.1136/jnnp-2018-318570

    Article  PubMed  Google Scholar 

  80. Rosenthal IA, Bashford L, Kellis S, Pejsa K, Lee B, Liu C, Andersen RA (2023) S1 represents multisensory contexts and somatotopic locations within and outside the bounds of the cortical homunculus. Cell Rep 42:112312. https://doi.org/10.1016/j.celrep.2023.112312

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Saal HP, Bensmaia SJ (2014) Touch is a team effort: interplay of submodalities in cutaneous sensibility. Trends Neurosci 37:689–697. https://doi.org/10.1016/j.tins.2014.08.012

    Article  CAS  PubMed  Google Scholar 

  82. Saal HP, Bensmaia SJ (2015) Biomimetic approaches to bionic touch through a peripheral nerve interface. Neuropsychologia 79:344–353. https://doi.org/10.1016/j.neuropsychologia.2015.06.010

    Article  PubMed  Google Scholar 

  83. Saal HP, Delhaye BP, Rayhaun BC, Bensmaia SJ (2017) Simulating tactile signals from the whole hand with millisecond precision. Proc Nat Acad Sci 114:E5693–E5702. https://doi.org/10.1073/pnas.1704856114

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  84. Schiefer M, Tan D, Sidek SM, Tyler DJ (2015) Sensory feedback by peripheral nerve stimulation improves task performance in individuals with upper limb loss using a myoelectric prosthesis. J Neural Eng 13:016001. https://doi.org/10.1088/1741-2560/13/1/016001

    Article  PubMed  PubMed Central  Google Scholar 

  85. Serino A, Akselrod M, Salomon R, Martuzzi R, Blefari ML, Canzoneri E, Rognini G, van der Zwaag W, Iakova M, Luthi F, Amoresano A, Kuiken T, Blanke O (2017) Upper limb cortical maps in amputees with targeted muscle and sensory reinnervation. Brain 140:2993–3011. https://doi.org/10.1093/brain/awx242

    Article  PubMed  Google Scholar 

  86. Serino A, Bockbrader M, Bertoni T, Colachis S IV, Solcà M, Dunlap C, Eipel K, Ganzer P, Annetta N, Sharma G, Orepic P, Friedenberg D, Sederberg P, Faivre N, Rezai A, Blanke O (2022) Sense of agency for intracortical brain–machine interfaces. Nat Hum Behav 6:565–578. https://doi.org/10.1038/s41562-021-01233-2

    Article  PubMed  Google Scholar 

  87. Shin H, Watkins Z, Huang HH, Zhu Y, Hu X (2018) Evoked haptic sensations in the hand via non-invasive proximal nerve stimulation. J Neural Eng 15:046005. https://doi.org/10.1088/1741-2552/aabd5d

    Article  PubMed  Google Scholar 

  88. Strauss I, Valle G, Artoni F, D’Anna E, Granata G, Di Iorio R, Guiraud D, Stieglitz T, Rossini PM, Raspopovic S, Petrini FM, Micera S (2019) Characterization of multi-channel intraneural stimulation in transradial amputees. Sci Rep 9:19258. https://doi.org/10.1038/s41598-019-55591-z

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  89. Tan DW, Schiefer MA, Keith MW, Anderson JR, Tyler J, Tyler DJ (2014) A neural interface provides long-term stable natural touch perception. Sci Transl Med 6:257ra138. https://doi.org/10.1126/scitranslmed.3008669

    Article  PubMed  PubMed Central  Google Scholar 

  90. Valle G (2022) Peripheral neurostimulation for encoding artificial somatosensations. Eur J Neurosci 56:5888–5901. https://doi.org/10.1111/ejn.15822

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Valle G, Katic Secerovic N, Eggemann D, Gorskii O, Pavlova N, Petrini FM, Cvancara P, Stieglitz T, Musienko P, Bumbasirevic M, Raspopovic S (2024) Biomimetic computer-to-brain communication enhancing naturalistic touch sensations via peripheral nerve stimulation. Nat Commun 15:1151. https://doi.org/10.1038/s41467-024-45190-6

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  92. Valle G, Mazzoni A, Iberite F, D’Anna E, Strauss I, Granata G, Controzzi M, Clemente F, Rognini G, Cipriani C, Stieglitz T, Petrini FM, Rossini PM, Micera S (2018) Biomimetic intraneural sensory feedback enhances sensation naturalness, tactile sensitivity, and manual dexterity in a bidirectional prosthesis. Neuron 100:37-45.e7. https://doi.org/10.1016/j.neuron.2018.08.033

    Article  CAS  PubMed  Google Scholar 

  93. Valle G, Petrini FM, Strauss I, Iberite F, D’Anna E, Granata G, Controzzi M, Cipriani C, Stieglitz T, Rossini PM, Mazzoni A, Raspopovic S, Micera S (2018) Comparison of linear frequency and amplitude modulation for intraneural sensory feedback in bidirectional hand prostheses. Sci Rep 8:16666. https://doi.org/10.1038/s41598-018-34910-w

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  94. Valle G, Saliji A, Fogle E, Cimolato A, Petrini FM, Raspopovic S (2021) Mechanisms of neuro-robotic prosthesis operation in leg amputees. Sci Adv 7:eabd8354. https://doi.org/10.1126/sciadv.abd8354

    Article  PubMed  PubMed Central  Google Scholar 

  95. Wendelken S, Page DM, Davis T, Wark HAC, Kluger DT, Duncan C, Warren DJ, Hutchinson DT, Clark GA (2017) Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves. J Neuroeng Rehabil 14:121. https://doi.org/10.1186/s12984-017-0320-4

    Article  PubMed  PubMed Central  Google Scholar 

  96. Zbinden J, Lendaro E, Ortiz-Catalan M (2022) A multi-dimensional framework for prosthetic embodiment: a perspective for translational research. J Neuroeng Rehabil 19:122. https://doi.org/10.1186/s12984-022-01102-7

    Article  PubMed  PubMed Central  Google Scholar 

  97. Zbinden J, Lendaro E, Ortiz-Catalan M (2022) Prosthetic embodiment: systematic review on definitions, measures, and experimental paradigms. J Neuroeng Rehabil 19:37. https://doi.org/10.1186/s12984-022-01006-6

    Article  PubMed  PubMed Central  Google Scholar 

  98. Zelechowski M, Valle G, Raspopovic S (2020) A computational model to design neural interfaces for lower-limb sensory neuroprostheses. J Neuroeng Rehabil 17:24. https://doi.org/10.1186/s12984-020-00657-7

    Article  PubMed  PubMed Central  Google Scholar 

  99. Zollo L, Pino GD, Ciancio AL, Ranieri F, Cordella F, Gentile C, Noce E, Romeo RA, Bellingegni AD, Vadalà G, Miccinilli S, Mioli A, Diaz-Balzani L, Bravi M, Hoffmann KP, Schneider A, Denaro L, Davalli A, Gruppioni E, Sacchetti R, Castellano S, Lazzaro VD, Sterzi S, Denaro V, Guglielmelli E (2019) Restoring tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands. Sci Robot 4:eaau9924. https://doi.org/10.1126/scirobotics.aau9924

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by the U.S. Department of Energy (DOE), Office of Science, Office of Advanced Scientific Computing Research through the Collaborative Research in Computational Neuroscience (CRCNS) program under award number DE-SC0022150, and by the Department of Defence (DOD) through the Orthotics and Prosthetics Outcome Research Program (OPORP) under award number W81XWH2010842. The content is solely the responsibility of the authors and does not necessarily represent the official views of the listed funding institutions. We thank R. Li and M. Cao for their help on the manuscript.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD, 21205, USA

    Keqin Ding & Nitish V. Thakor

  2. Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL, 32816, USA

    Mohsen Rakhshan

  3. Disability, Aging, and Technology Cluster, University of Central Florida, Orlando, FL, 32816, USA

    Mohsen Rakhshan

  4. Institute for Cognitive Systems, School of Computation, Information and Technology, Technical University of Munich, 80333, Munich, Germany

    Natalia Paredes-Acuña & Gordon Cheng

  5. Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, 21205, USA

    Nitish V. Thakor

Authors
  1. Keqin Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohsen Rakhshan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Natalia Paredes-Acuña
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gordon Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nitish V. Thakor
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to this work. Conceptualization: K.D., M.R.; literature search: K.D., M.R.; visualization: K.D., M.R., N.P.-A.; supervision: G.C., N.V.T.; writing — original draft: K.D., M.R., N.V.T.; writing — review and editing: K.D., M.R., N.P.-A., G.C., N.V.T.

Corresponding author

Correspondence to Keqin Ding.

Ethics declarations

Conflict of interest

G.C. is a shareholder and co-founder of intouch-robotics GmbH, Munich, Germany. N.V.T. is on the advisory board of Medical & Biological Engineering and Computing. N.V.T. is also a co-founder of Infinite Biomedical Technologies, Baltimore, MD, USA. The other authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, K., Rakhshan, M., Paredes-Acuña, N. et al. Sensory integration for neuroprostheses: from functional benefits to neural correlates. Med Biol Eng Comput (2024). https://doi.org/10.1007/s11517-024-03118-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11517-024-03118-8

Keywords

Search

Navigation