Brain-Machine Interface Development for Finger Movement Control

  • Tessy M. Lal
  • Guy Hotson
  • Matthew S. Fifer
  • David P. McMullen
  • Matthew S. Johannes
  • Kapil D. Katyal
  • Matthew P. Para
  • Robert Armiger
  • William S. Anderson
  • Nitish V. Thakor
  • Brock A. Wester
  • Nathan E. Crone
Chapter
First Online:
Part of the SpringerBriefs in Electrical and Computer Engineering book series (BRIEFSELECTRIC)

Abstract

There have been many developments in brain-machine interfaces (BMI) for controlling upper limb movements such as reaching and grasping. One way to expand the usefulness of BMIs in replacing motor functions for patients with spinal cord injuries and neuromuscular disorders would be to improve the dexterity of upper limb movements performed by including more control of individual finger movements. Many studies have been focusing on understanding the organization of movement control in the sensorimotor cortex of the human brain. Finding the specific mechanisms for neural control of different movements will help focus signal acquisition and processing so as to improve BMI control of complex actions. In a recently published study, we demonstrated, for the first time, online BMI control of individual finger movements using electrocorticography recordings from the hand area of sensorimotor cortex. This study expands the possibilities for combined control of arm movements and more dexterous hand and finger movements.

Keywords

Brain-machine interface (BMI) Brain-computer interface (BCI) Electrocorticography (ECoG) Finger movements 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    J. Kubánek, K.J. Miller, J.G. Ojemann, J.R. Wolpaw, G. Schalk, Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6(6), 66001 (2009)CrossRefGoogle Scholar
  2. 2.
    C.A. Chestek et al., Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas. J. Neural Eng. 10(2), 26002 (2013)CrossRefGoogle Scholar
  3. 3.
    G. Hotson et al., Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject. J. Neural Eng. 13(2), 26017 (2016)CrossRefGoogle Scholar
  4. 4.
    S.N. Flesher et al., Intracortical microstimulation of human somatosensory cortex. Sci. Transl. Med. aaf8083 (2016)Google Scholar
  5. 5.
    M.J. Morrell, Responsive cortical stimulation for the treatment of medically intractable partial epilepsy. Neurology 77(13), 1295–1304 (2011)CrossRefGoogle Scholar
  6. 6.
    T. Tsubokawa, Y. Katayama, T. Yamamoto, T. Hirayama, S. Koyama, Chronic motor cortex stimulation in patients with thalamic pain. J. Neurosurg. 78(3), 393–401 (1993)CrossRefGoogle Scholar
  7. 7.
    Y. Katayama, C. Fukaya, T. Yamamoto, Poststroke pain control by chronic motor cortex stimulation: neurological characteristics predicting a favorable response. J. Neurosurg. 89(4), 585–591 (1998)CrossRefGoogle Scholar
  8. 8.
    J.-P. Nguyen et al., Treatment of chronic neuropathic pain by motor cortex stimulation: results of a bicentric controlled crossover trial. Brain Stimul. 1(2), 89–96 (2008)CrossRefGoogle Scholar
  9. 9.
    Z.C. Chao, Y. Nagasaka, N. Fujii, Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front. Neuroeng. 3, 3 (2010)Google Scholar
  10. 10.
    N.E. Crone, D.L. Miglioretti, B. Gordon, R.P. Lesser, Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain 121(12), 2301–2315 (1998)CrossRefGoogle Scholar
  11. 11.
    K.J. Miller, M. den Nijs, P. Shenoy, J.W. Miller, R.P.N. Rao, J.G. Ojemann, Real-time functional brain mapping using electrocorticography. NeuroImage 37(2), 504–507 (2007)Google Scholar
  12. 12.
    E.C. Leuthardt et al., Electrocorticographic frequency alteration mapping: a clinical technique for mapping the motor cortex. Oper. Neurosurg. 60, 260–271 (2007)CrossRefGoogle Scholar
  13. 13.
    E.C. Leuthardt, G. Schalk, J.R. Wolpaw, J.G. Ojemann, D.W. Moran, A brain–computer interface using electrocorticographic signals in humans. J. Neural Eng. 1(2), 63 (2004)CrossRefGoogle Scholar
  14. 14.
    G. Schalk et al., Two-dimensional movement control using electrocorticographic signals in humans. J. Neural Eng. 5(1), 75 (2008)CrossRefGoogle Scholar
  15. 15.
    E.C. Leuthardt, K.J. Miller, G. Schalk, R.P.N. Rao, J.G. Ojemann, Electrocorticography-based brain computer Interface-the seattle experience. IEEE Trans. Neural Syst. Rehabil. Eng. 14(2), 194–198 (2006)CrossRefGoogle Scholar
  16. 16.
    K.J. Miller, G. Schalk, E.E. Fetz, M. den Nijs, J.G. Ojemann, R.P.N. Rao, Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Natl. Acad. Sci. 107(9), 4430–4435 (2010)CrossRefGoogle Scholar
  17. 17.
    W. Wang et al., An electrocorticographic brain interface in an individual with tetraplegia. PLoS ONE 8(2), e55344 (2013)CrossRefGoogle Scholar
  18. 18.
    M.G. Bleichner, Z.V. Freudenburg, J.M. Jansma, E.J. Aarnoutse, M.J. Vansteensel, N.F. Ramsey, Give me a sign: decoding four complex hand gestures based on high-density ECoG. Brain Struct. Funct. 1–14 (2014)Google Scholar
  19. 19.
    T. Pistohl, T. Ball, A. Schulze-Bonhage, A. Aertsen, C. Mehring, Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neurosci. Methods 167(1), 105–114 (2008)CrossRefGoogle Scholar
  20. 20.
    T. Pistohl, A. Schulze-Bonhage, A. Aertsen, C. Mehring, T. Ball, Decoding natural grasp types from human ECoG. NeuroImage 59(1), 248–260 (2012)CrossRefGoogle Scholar
  21. 21.
    T. Yanagisawa et al., Real-time control of a prosthetic hand using human electrocorticography signals. J. Neurosurg. 114(6), 1715–1722 (2011)CrossRefGoogle Scholar
  22. 22.
    M.S. Fifer et al., Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial EEG. IEEE Trans. Neural Syst. Rehabil. Eng. 22(3), 695–705 (2014)CrossRefGoogle Scholar
  23. 23.
    L.R. Hochberg et al., Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398), 372–375 (2012)CrossRefGoogle Scholar
  24. 24.
    J.L. Collinger et al., High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet 381(9866), 557–564 (2013)CrossRefGoogle Scholar
  25. 25.
    T. Aflalo et al., Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348(6237), 906–910 (2015)CrossRefGoogle Scholar
  26. 26.
    B. Wodlinger, J.E. Downey, E.C. Tyler-Kabara, A.B. Schwartz, M.L. Boninger, J.L. Collinger, Ten-dimensional anthropomorphic arm control in a human brain–machine interface: difficulties, solutions, and limitations. J. Neural Eng. 12(1), 16011 (2015)CrossRefGoogle Scholar
  27. 27.
    C.E. Bouton et al., Restoring cortical control of functional movement in a human with quadriplegia. Nature 533(7602), 247–250 (2016)CrossRefGoogle Scholar
  28. 28.
    S.B. Hamed, M.H. Schieber, A. Pouget, Decoding M1 neurons during multiple finger movements. J. Neurophysiol. 98(1), 327–333 (2007)CrossRefGoogle Scholar
  29. 29.
    V. Aggarwal et al., Asynchronous decoding of dexterous finger movements using M1 neurons. IEEE Trans. Neural Syst. Rehabil. Eng. 16(1), 3–14 (2008)MathSciNetCrossRefGoogle Scholar
  30. 30.
    S. Acharya, F. Tenore, V. Aggarwal, R. Etienne-Cummings, M.H. Schieber, N.V. Thakor, Decoding individuated finger movements using volume-constrained neuronal ensembles in the M1 hand area. IEEE Trans. Neural Syst. Rehabil. Eng. 16(1), 15–23 (2008)CrossRefGoogle Scholar
  31. 31.
    K. Liao, R. Xiao, J. Gonzalez, L. Ding, Decoding individual finger movements from one hand using human EEG signals. PLoS ONE 9(1), e85192 (2014)CrossRefGoogle Scholar
  32. 32.
    A.Y. Paek, H.A. Agashe, J.L. Contreras-Vidal, Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography. Front. Neuroeng. 7 (2014)Google Scholar
  33. 33.
    S. Acharya, M.S. Fifer, H.L. Benz, N.E. Crone, N.V. Thakor, Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. J. Neural Eng. 7(4), 46002 (2010)CrossRefGoogle Scholar
  34. 34.
    R. Flamary, A. Rakotomamonjy, Decoding finger movements from ECoG signals using switching linear models. Front. Neurosci. 6 (2012)Google Scholar
  35. 35.
    N. Liang, L. Bougrain, Decoding finger flexion from band-specific ECoG signals in humans. Front. Neurosci. 6 (2012)Google Scholar
  36. 36.
    Y. Nakanishi et al., Decoding fingertip trajectory from electrocorticographic signals in humans. Neurosci. Res. 85, 20–27 (2014)CrossRefGoogle Scholar
  37. 37.
    J. Hammer et al., Predominance of movement speed over direction in neuronal population signals of motor cortex: intracranial EEG data and a simple explanatory model. Cereb. Cortex 26(6), 2863–2881 (2016)CrossRefGoogle Scholar
  38. 38.
    G. Schalk et al., Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J. Neural Eng. 4(3), 264 (2007)CrossRefGoogle Scholar
  39. 39.
    D.T. Bundy, M. Pahwa, N. Szrama, E.C. Leuthardt, Decoding three-dimensional reaching movements using electrocorticographic signals in humans. J. Neural Eng. 13(2), 26021 (2016)CrossRefGoogle Scholar
  40. 40.
    W. Penfield, E. Boldrey, Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation. Brain J. Neurol. (1937)Google Scholar
  41. 41.
    P. Hluštík, A. Solodkin, R.P. Gullapalli, D.C. Noll, S.L. Small, Somatotopy in human primary motor and somatosensory hand representations revisited. Cereb. Cortex 11(4), 312–321 (2001)CrossRefGoogle Scholar
  42. 42.
    R.M. Sanchez-Panchuelo, S. Francis, R. Bowtell, D. Schluppeck, Mapping human somatosensory cortex in individual subjects with 7T functional MRI. J. Neurophysiol. 103(5), 2544–2556 (2010)CrossRefGoogle Scholar
  43. 43.
    M.H. Schieber, Constraints on somatotopic organization in the primary motor cortex. J. Neurophysiol. 86(5), 2125–2143 (2001)Google Scholar
  44. 44.
    J.N. Sanes, J.P. Donoghue, V. Thangaraj, R.R. Edelman, S. Warach, Shared neural substrates controlling hand movements in human motor cortex. Science 268(5218), 1775–1777 (1995)CrossRefGoogle Scholar
  45. 45.
    I. Indovina, J.N. Sanes, On somatotopic representation centers for finger movements in human primary motor cortex and supplementary motor area. NeuroImage 13(6), 1027–1034 (2001)CrossRefGoogle Scholar
  46. 46.
    M.H. Schieber, Somatotopic gradients in the distributed organization of the human primary motor cortex hand area: evidence from small infarcts. Exp. Brain Res. 128(1–2), 139–148 (1999)CrossRefGoogle Scholar
  47. 47.
    C.E. Vargas-Irwin, G. Shakhnarovich, P. Yadollahpour, J.M.K. Mislow, M.J. Black, J.P. Donoghue, Decoding complete reach and grasp actions from local primary motor cortex populations. J. Neurosci. 30(29), 9659–9669 (2010)CrossRefGoogle Scholar
  48. 48.
    M. Saleh, K. Takahashi, Y. Amit, N.G. Hatsopoulos, Encoding of coordinated grasp trajectories in primary motor cortex. J. Neurosci. 30(50), 17079–17090 (2010)CrossRefGoogle Scholar
  49. 49.
    M. Saleh, K. Takahashi, N.G. Hatsopoulos, Encoding of coordinated reach and grasp trajectories in primary motor cortex. J. Neurosci. 32(4), 1220–1232 (2012)CrossRefGoogle Scholar
  50. 50.
    J.C. Kao, P. Nuyujukian, S.I. Ryu, M.M. Churchland, J.P. Cunningham, K.V. Shenoy, Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nat. Commun. 6, 7759 (2015)CrossRefGoogle Scholar
  51. 51.
    S.A. Overduin, A. d’Avella, J. Roh, J.M. Carmena, E. Bizzi, Representation of muscle synergies in the primate brain. J. Neurosci. 35(37), 12615–12624 (2015)Google Scholar
  52. 52.
    A. d’Avella, A. Portone, L. Fernandez, F. Lacquaniti, Control of fast-reaching movements by muscle synergy combinations. J. Neurosci. 26(30), 7791–7810 (2006)Google Scholar
  53. 53.
    S.A. Overduin, A. d’Avella, J. Roh, E. Bizzi, Modulation of muscle synergy recruitment in primate grasping. J. Neurosci. 28(4), 880–892 (2008)Google Scholar
  54. 54.
    A. d’Avella, A. Portone, F. Lacquaniti, Superposition and modulation of muscle synergies for reaching in response to a change in target location. J. Neurophysiol. 106(6), 2796–2812 (2011)Google Scholar
  55. 55.
    C.R. Mason, J.E. Gomez, T.J. Ebner, Hand synergies during reach-to-grasp. J. Neurophysiol. 86(6), 2896–2910 (2001)Google Scholar
  56. 56.
    P.H. Thakur, A.J. Bastian, S.S. Hsiao, Multidigit movement synergies of the human hand in an unconstrained haptic exploration task. J. Neurosci. 28(6), 1271–1281 (2008)CrossRefGoogle Scholar
  57. 57.
    R. Vinjamuri, M. Sun, C.-C. Chang, H.-N. Lee, R.J. Sclabassi, Z.-H. Mao, Dimensionality reduction in control and coordination of the human hand. IEEE Trans. Biomed. Eng. 57(2), 284–295 (2010)Google Scholar
  58. 58.
    S.A. Overduin, A. d’Avella, J.M. Carmena, E. Bizzi, Microstimulation activates a handful of muscle synergies. Neuron 76(6), 1071–1077. (2012)Google Scholar
  59. 59.
    S.A. Overduin, A. d’Avella, J.M. Carmena, E. Bizzi, Muscle synergies evoked by microstimulation are preferentially encoded during behavior. Front. Comput. Neurosci. 8 (2014)Google Scholar
  60. 60.
    M.S. Graziano, C.S. Taylor, T. Moore, Complex movements evoked by microstimulation of precentral cortex. Neuron 34(5), 841–851 (2002)CrossRefGoogle Scholar
  61. 61.
    M. Desmurget et al., Neural representations of ethologically relevant hand/mouth synergies in the human precentral gyrus. Proc. Natl. Acad. Sci. U.S.A. 111(15), 5718–5722 (2014)CrossRefGoogle Scholar
  62. 62.
    M. Mollazadeh, V. Aggarwal, N.V. Thakor, M.H. Schieber, Principal components of hand kinematics and neurophysiological signals in motor cortex during reach to grasp movements. J. Neurophysiol. 112(8), 1857–1870 (2014)CrossRefGoogle Scholar
  63. 63.
    E. Kirsch, G. Rivlis, M.H. Schieber, Primary motor cortex neurons during individuated finger and wrist movements: correlation of spike firing rates with the motion of individual digits versus their principal components. Front. Neurol. 5 (2014)Google Scholar
  64. 64.
    J.S. Duncan, X. Papademetris, J. Yang, M. Jackowski, X. Zeng, L.H. Staib, Geometric strategies for neuroanatomic analysis from MRI. NeuroImage 23(Supplement 1), S34–S45 (2004)CrossRefGoogle Scholar
  65. 65.
    Y. Guo, T. Hastie, R. Tibshirani, Regularized linear discriminant analysis and its application in microarrays. Biostatistics 8(1), 86–100 (2007)CrossRefMATHGoogle Scholar
  66. 66.
    M.S. Johannes, J.D. Bigelow, J.M. Burck, S.D. Harshbarger, M.V. Kozlowski, T. Van Doren, An overview of the developmental process for the modular prosthetic limb. Johns Hopkins APL Tech. Dig. 30(3), 207–216 (2011)Google Scholar
  67. 67.
    A. Harris, K. Katyal, M. Para, J. Thomas, Revolutionizing prosthetics software technology, in 2011 IEEE International Conference on Systems, Man, and Cybernetics (2011), pp. 2877–2884Google Scholar
  68. 68.
    M.M. Bridges, M.P. Para, M.J. Mashner, Control system architecture for the modular prosthetic limb. Johns Hopkins APL Tech. Dig. 30(3) (2011)Google Scholar
  69. 69.
    M.S. Fifer, S. Acharya, H.L. Benz, M. Mollazadeh, N.E. Crone, N.V. Thakor, Towards electrocorticographic control of a dexterous upper limb prosthesis. IEEE Pulse 3(1), 38–42 (2012)CrossRefGoogle Scholar
  70. 70.
    Amputee Makes History with APL’s Modular Prosthetic Limb, http://www.jhuapl.edu/newscenter/pressreleases/2014/141216.asp. Accessed 30 Nov 2016
  71. 71.
    APL’s Modular Prosthetic Limb Reaches New Levels of Operability, http://www.jhuapl.edu/newscenter/pressreleases/2016/160112.asp. Accessed 30 Nov 2016
  72. 72.
    D.P. McMullen et al., Demonstration of a semi-autonomous hybrid brain–machine Interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic. IEEE Trans. Neural Syst. Rehabil. Eng. 22(4), 784–796 (2014)CrossRefGoogle Scholar
  73. 73.
    A.J. Szameitat, S. Shen, A. Conforto, A. Sterr, Cortical activation during executed, imagined, observed, and passive wrist movements in healthy volunteers and stroke patients. NeuroImage 62(1), 266–280 (2012)CrossRefGoogle Scholar
  74. 74.
    R.J. Nelson, Interactions between motor commands and somatic perception in sensorimotor cortex. Curr. Opin. Neurobiol. 6(6), 801–810 (1996)CrossRefGoogle Scholar
  75. 75.
    M.S. Christensen, J. Lundbye-Jensen, S.S. Geertsen, T.H. Petersen, O.B. Paulson, J.B. Nielsen, Premotor cortex modulates somatosensory cortex during voluntary movements without proprioceptive feedback. Nat. Neurosci. 10(4), 417–419 (2007)Google Scholar
  76. 76.
    T.B. Crapse, M.A. Sommer, Corollary discharge circuits in the primate brain. Curr. Opin. Neurobiol. 18(6), 552–557 (2008)CrossRefGoogle Scholar
  77. 77.
    V. Gritsenko, N.I. Krouchev, J.F. Kalaska, Afferent input, efference copy, signal noise, and biases in perception of joint angle during active versus passive elbow movements. J. Neurophysiol. 98(3), 1140–1154 (2007)CrossRefGoogle Scholar
  78. 78.
    H. Sun et al., Sequential activation of premotor, primary somatosensory and primary motor areas in humans during cued finger movements. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. (2015)Google Scholar
  79. 79.
    G. Hotson, R.J. Smith, A.G. Rouse, M.H. Schieber, N.V. Thakor, B.A. Wester, High precision neural decoding of complex movement trajectories using recursive Bayesian estimation with dynamic movement primitives. IEEE Robot. Autom. Lett. 1(2), 676–683 (2016)CrossRefGoogle Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  • Tessy M. Lal
    • 1
  • Guy Hotson
    • 2
  • Matthew S. Fifer
    • 1
  • David P. McMullen
    • 3
  • Matthew S. Johannes
    • 4
  • Kapil D. Katyal
    • 4
  • Matthew P. Para
    • 4
  • Robert Armiger
    • 4
  • William S. Anderson
    • 3
  • Nitish V. Thakor
    • 1
  • Brock A. Wester
    • 4
  • Nathan E. Crone
    • 5
  1. 1.Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.Department of Electrical and Computer EngineeringJohns Hopkins UniversityBaltimoreUSA
  3. 3.Department of NeurosurgeryJohns Hopkins UniversityBaltimoreUSA
  4. 4.JHU Applied Physics Laboratory Applied NeuroscienceLaurelUSA
  5. 5.Department of NeurologyJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations