Brain Machine-Interfaces for Motor and Communication Control

Abstract

The brain-machine interface (BMI) is a technology which enables our brains to control external devices or to obtain sensory information by directly communicating with computers. BMIs are classified into two types: invasive type which uses intracranial electrodes, such as needle microelectrodes and brain surface electrodes, and noninvasive type which uses skin electrodes, near infrared spectroscopy, and etc. Noninvasive BMIs are promising for neurorehabilitation, while invasive BMIs are promising for neural prostheses for severely disabled people. BMIs using needle microelectrodes are characterized by high performance utilizing its detailed neural information, while BMIs using brain surface electrodes are noted for the high feasibility for clinical application based on its long term stability. We describe our development of a BMI using brain surface electrodes. It enables real time control of a robotic arm and a fully-implantable wireless system.

Keywords

Brain machine interface Functional restoration Electrocorticogram Robot Implant 

Notes

Acknowledgments

This work was partly supported by a grant for “Brain Machine Interface Development” from the Strategic Research Program for Brain Sciences of MEXT, and a Health Labour Sciences Research Grant (23100101) from the Ministry of Health, Labour, and Welfare of Japan. We would like to thank Toshiki Yoshimine, Takufumi Yanagisawa, Hisato Sugata, Morris Shayne, Takashi Moriwaki (Osaka Univ.), Yukiyasu Kamitani (ATR), and Hiroshi Yokoi (Univ. Electro. Com.). We would also like to thank Takafumi Suzuki, Hiroshi Ando (NICT), Takeshi Yoshida (Hiroshima Univ.), Fumihiro Sato (Tohoku Univ.), Yukio Nishimura, Tatsuya Umeda (NINS), Atsushi Iwata (A-R-Tec Corp), Shinichi Morikawa (Unique Medical), Naohiro Hayaishi (Keisuugiken Corp), Shinichi Yoshimura, Shuhei Kosaka (Aska Electric Co Ltd), and Hirofumi Itoh (Junkosha Inc), for prototype manufacturing of our implantable system. Our appreciation also goes to VLSI Design and Education Center (VDEC), University of Tokyo, for offering the chip fabrication program.

References

  1. Birbaumer, N., Ramos Murguialday, A., Weber, C., Montoya, P.: Neurofeedback and brain-computer interface clinical applications. Int. Rev. Neurobiol. 86, 107–117 (2009)CrossRefGoogle Scholar
  2. Chao, Z.C., Nagasaka, Y., Fujii, N.: Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys. Front. Neuroeng. 3, 3 (2010)Google Scholar
  3. Chapin, J.K., Moxon, K.A., Markowitz, R.S., Nicolelis, M.A.: Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664–670 (1999)CrossRefGoogle Scholar
  4. Collinger, J.L., Wodlinger, B., Downey, J.E., Wang, W., Tyler-Kabara, E.C., Weber, D.J., McMorland, A.J., Velliste, M., Boninger, M.L., Schwartz, A.B.: High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2013)CrossRefGoogle Scholar
  5. Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70, 510–523 (1988)CrossRefGoogle Scholar
  6. Georgopoulos, A.P., Kalaska, J.F., Caminiti, R., Massey, J.T.: On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982)Google Scholar
  7. Georgopoulos, A.P., Kettner, R.E., Schwartz, A.B.: Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population. J. Neurosci. 8, 2928–2937 (1988)Google Scholar
  8. Guenther, F.H., Brumberg, J.S., Wright, E.J., Nieto-Castanon, A., Tourville, J.A., Panko, M., Law, R., Siebert, S.A., Bartels, J.L., Andreasen, D.S., Ehirim, P., Mao, H., Kennedy, P.R.: A wireless brain-machine interface for real-time speech synthesis. PLoS One 4, e8218 (2009)CrossRefGoogle Scholar
  9. Hirata, M., Kato, A., Taniguchi, M., Ninomiya, H., Cheyne, D., Robinson, S.E., Maruno, M., Kumura, E., Ishii, R., Hirabuki, N., Nakamura, H., Yoshimine, T.: Frequency-dependent spatial distribution of human somatosensory evoked neuromagnetic fields. Neurosci. Lett. 318, 73–76 (2002)CrossRefGoogle Scholar
  10. Hirata, M., Kato, A., Taniguchi, M., Saitoh, Y., Ninomiya, H., Ihara, A., Kishima, H., Oshino, S., Baba, T., Yorifuji, S., Yoshimine, T.: Determination of language dominance with synthetic aperture magnetometry: comparison with the Wada test. Neuroimage 23, 46–53 (2004)CrossRefGoogle Scholar
  11. Hirata, M., Yoshimine, T., Saitoh, Y., Yanagisawa, T., Goto, T., Watanabe, Y., Saito, T.: Intracranial electrode and method for producing same. US Patent 7,860,577, 2010Google Scholar
  12. Hirata, M., Matsushita, K., Suzuki, T., Yoshida, T., Sato, F., Morris, S., Yanagisawa, M., Goto, T., Kawato, M., Yoshimine, T.: A fully-implantable wireless system for human brain-machine interfaces using brain surface electrodes: W-HERBS. IEICE Trans. Commun. E94-B, 2448–2453 (2011)CrossRefGoogle Scholar
  13. Hochberg, L.R., Serruya, M.D., Friehs, G.M., Mukand, J.A., Saleh, M., Caplan, A.H., Branner, A., Chen, D., Penn, R.D., Donoghue, J.P.: Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006)CrossRefGoogle Scholar
  14. Hochberg, L.R., Bacher, D., Jarosiewicz, B., Masse, N.Y., Simeral, J.D., Vogel, J., Haddadin, S., Liu, J., Cash, S.S., van der Smagt, P., Donoghue, J.P.: Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012)CrossRefGoogle Scholar
  15. Horikawa, T., Tamaki, M., Miyawaki, Y., Kamitani, Y.: Neural decoding of visual imagery during sleep. Science 340, 639–642 (2013)CrossRefGoogle Scholar
  16. Hosomi, K., Saitoh, Y., Kishima, H., Oshino, S., Hirata, M., Tani, N., Shimokawa, T., Yoshimine, T.: Electrical stimulation of primary motor cortex within the central sulcus for intractable neuropathic pain. Clin. Neurophysiol. 119, 993–1001 (2008)CrossRefGoogle Scholar
  17. Huggins, J.E., Levine, S.P., BeMent, S.L., Kushwaha, R.K., Schuh, L.A., Passaro, E.A., Rohde, M.M., Ross, D.A., Elisevich, K.V., Smith, B.J.: Detection of event-related potentials for development of a direct brain interface. J. Clin. Neurophysiol. 16, 448–455 (1999)CrossRefGoogle Scholar
  18. Ikegami, S., Takano, K., Wada, M., Saeki, N., Kansaku, K.: Effect of the green/blue flicker matrix for P300-based brain-computer interface: an EEG-fMRI study. Front. Neurol. 3, 113 (2012)CrossRefGoogle Scholar
  19. Ishii, R., Canuet, L., Herdman, A., Gunji, A., Iwase, M., Takahashi, H., Nakahachi, T., Hirata, M., Robinson, S.E., Pantev, C., Takeda, M.: Cortical oscillatory power changes during auditory oddball task revealed by spatially filtered magnetoencephalography. Clin. Neurophysiol. 120, 497–504 (2009)CrossRefGoogle Scholar
  20. Kamitani, Y., Tong, F.: Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8, 679–685 (2005)CrossRefGoogle Scholar
  21. Leuthardt, E.C., Schalk, G., Wolpaw, J.R., Ojemann, J.G., Moran, D.W.: A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63–71 (2004)CrossRefGoogle Scholar
  22. Levine, S.P., Huggins, J.E., BeMent, S.L., Kushwaha, R.K., Schuh, L.A., Passaro, E.A., Rohde, M.M., Ross, D.A.: Identification of electrocorticogram patterns as the basis for a direct brain interface. J. Clin. Neurophysiol. 16, 439–447 (1999)CrossRefGoogle Scholar
  23. Matsushita, K., Hirata, M., Suzuki, T., Ando, H., Ota, Y., Sato, F., Morris, S., Yoshida, T., Matsuki, H., Yoshimine, T.: Development of an implantable wireless ECoG 128ch recording device for clinical brain machine interface. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2013, 1867–1870 (2013)Google Scholar
  24. Miyawaki, Y., Uchida, H., Yamashita, O., Sato, M.A., Morito, Y., Tanabe, H.C., Sadato, N., Kamitani, Y.: Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron 60, 915–929 (2008)CrossRefGoogle Scholar
  25. Nakanishi, Y., Yanagisawa, T., Shin, D., Fukuma, R., Chen, C., Kambara, H., Yoshimura, N., Hirata, M., Yoshimine, T., Koike, Y.: Prediction of three-dimensional arm trajectories based on ECoG signals recorded from human sensorimotor cortex. PLoS One 8, e72085 (2013)CrossRefGoogle Scholar
  26. Ojemann, G.A.: Neurosurgical management of epilepsy: a personal perspective in 1983. Appl. Neurophysiol. 46, 11–18 (1983)Google Scholar
  27. Pfurtscheller, G., Neuper, C.: Future prospects of ERD/ERS in the context of brain-computer interface (BCI) developments. Prog. Brain Res. 159, 433–437 (2006)CrossRefGoogle Scholar
  28. Pistohl, T., Ball, T., Schulze-Bonhage, A., Aertsen, A., Mehring, C.: Prediction of arm movement trajectories from ECoG-recordings in humans. J. Neurosci. Methods 167, 105–114 (2008)CrossRefGoogle Scholar
  29. Ramos-Murguialday, A., Broetz, D., Rea, M., Laer, L., Yilmaz, O., Brasil, F.L., Liberati, G., Curado, M.R., Garcia-Cossio, E., Vyziotis, A., Cho, W., Agostini, M., Soares, E., Soekadar, S., Caria, A., Cohen, L.G., Birbaumer, N.: Brain-machine interface in chronic stroke rehabilitation: a controlled study. Ann. Neurol. 74, 100–108 (2013)CrossRefGoogle Scholar
  30. Rathelot, J.A., Strick, P.L.: Subdivisions of primary motor cortex based on cortico-motoneuronal cells. Proc. Natl. Acad. Sci. U. S. A. 106, 918–923 (2009)CrossRefGoogle Scholar
  31. Schalk, G., Kubanek, J., Miller, K.J., Anderson, N.R., Leuthardt, E.C., Ojemann, J.G., Limbrick, D., Moran, D., Gerhardt, L.A., Wolpaw, J.R.: Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J. Neural Eng. 4, 264–275 (2007)CrossRefGoogle Scholar
  32. Schalk, G., Miller, K.J., Anderson, N.R., Wilson, J.A., Smyth, M.D., Ojemann, J.G., Moran, D.W., Wolpaw, J.R., Leuthardt, E.C.: Two-dimensional movement control using electrocorticographic signals in humans. J. Neural Eng. 5, 75–84 (2008)CrossRefGoogle Scholar
  33. Serruya, M.D., Hatsopoulos, N.G., Paninski, L., Fellows, M.R., Donoghue, J.P.: Instant neural control of a movement signal. Nature 416, 141–142 (2002)CrossRefMATHGoogle Scholar
  34. Shibata, K., Watanabe, T., Sasaki, Y., Kawato, M.: Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation. Science 334, 1413–1415 (2011)CrossRefGoogle Scholar
  35. Shindo, K., Kawashima, K., Ushiba, J., Ota, N., Ito, M., Ota, T., Kimura, A., Liu, M.: Effects of neurofeedback training with an electroencephalogram-based brain-computer interface for hand paralysis in patients with chronic stroke: a preliminary case series study. J. Rehabil. Med. 43, 951–957 (2011)CrossRefGoogle Scholar
  36. Sitaram, R., Caria, A., Birbaumer, N.: Hemodynamic brain-computer interfaces for communication and rehabilitation. Neural Netw. 22, 1320–1328 (2009)CrossRefGoogle Scholar
  37. Sugata, H., Goto, T., Hirata, M., Yanagisawa, T., Shayne, M., Matsushita, K., Yoshimine, T., Yorifuji, S.: Neural decoding of unilateral upper limb movements using single trial MEG signals. Brain Res. 1468, 29–37 (2012a)CrossRefGoogle Scholar
  38. Sugata, H., Goto, T., Hirata, M., Yanagisawa, T., Shayne, M., Matsushita, K., Yoshimine, T., Yorifuji, S.: Movement-related neuromagnetic fields and performances of single trial classifications. Neuroreport 23, 16–20 (2012b)CrossRefGoogle Scholar
  39. Taniguchi, M., Kato, A., Fujita, N., Hirata, M., Tanaka, H., Kihara, T., Ninomiya, H., Hirabuki, N., Nakamura, H., Robinson, S.E., Cheyne, D., Yoshimine, T.: Movement-related desynchronization of the cerebral cortex studied with spatially filtered magnetoencephalography. Neuroimage 12, 298–306 (2000)CrossRefGoogle Scholar
  40. Taylor, D.M., Tillery, S.I., Schwartz, A.B.: Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002)CrossRefGoogle Scholar
  41. Toda, A., Imamizu, H., Kawato, M., Sato, M.A.: Reconstruction of two-dimensional movement trajectories from selected magnetoencephalography cortical currents by combined sparse Bayesian methods. Neuroimage 54, 892–905 (2011)CrossRefGoogle Scholar
  42. Velliste, M., Perel, S., Spalding, M.C., Whitford, A.S., Schwartz, A.B.: Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 (2008)CrossRefGoogle Scholar
  43. Wessberg, J., Stambaugh, C.R., Kralik, J.D., Beck, P.D., Laubach, M., Chapin, J.K., Kim, J., Biggs, S.J., Srinivasan, M.A., Nicolelis, M.A.: Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000)CrossRefGoogle Scholar
  44. Wolpaw, J.R., McFarland, D.J., Neat, G.W., Forneris, C.A.: An EEG-based brain-computer interface for cursor control. Electroencephalogr. Clin. Neurophysiol. 78, 252–259 (1991)CrossRefGoogle Scholar
  45. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002)CrossRefGoogle Scholar
  46. Yanagisawa, T., Hirata, M., Saitoh, Y., Kato, A., Shibuya, D., Kamitani, Y., Yoshimine, T.: Neural decoding using gyral and intrasulcal electrocorticograms. Neuroimage 45, 1099–1106 (2009)CrossRefGoogle Scholar
  47. Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., Yokoi, H., Kamitani, Y., Yoshimine, T.: Real-time control of a prosthetic hand using human electrocorticography signals. J. Neurosurg. 114, 1715–1722 (2011)CrossRefGoogle Scholar
  48. Yanagisawa, T., Yamashita, O., Hirata, M., Kishima, H., Saitoh, Y., Goto, T., Yoshimine, T., Kamitani, Y.: Regulation of motor representation by phase-amplitude coupling in the sensorimotor cortex. J. Neurosci. 32, 15467–15475 (2012a)CrossRefGoogle Scholar
  49. Yanagisawa, T., Hirata, M., Saitoh, Y., Kishima, H., Matsushita, K., Goto, T., Fukuma, R., Yokoi, H., Kamitani, Y., Yoshimine, T.: Electrocorticographic control of a prosthetic arm in paralyzed patients. Ann. Neurol. 71, 353–361 (2012b)CrossRefGoogle Scholar
  50. Yokoi, H., Kita, K., Nakamura, T.: Mutually adaptable EMG devices for prosthetic hand. Int. J. Fac. Autom. Robot. Soft Comput. 1, 74–83 (2009)Google Scholar
  51. Yoshida, T., Masui, Y., Eki, R., Iwata, A., Yoshida, M., Uematsu, K.: A neural recording amplifier with low-frequency noise suppression. IEICE Trans. Electrons E93-C, 849–854 (2010)CrossRefGoogle Scholar
  52. Yoshida, T., Sueishi, K., Iwata, A., Matsushita, K., Hirata, M., Suzuki, T.: A high-linearity low-noise amplifier with variable bandwidth for neural recording systems. Jpn. J. Appl. Phys. 50, 04DE07 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Japan 2016

Authors and Affiliations

  1. 1.Graduate School of MedicineOsaka UniversitySuitaJapan

Personalised recommendations