Advertisement

Real-Time fMRI-Based Brain Computer Interface: A Review

  • Yang Wang
  • Dongrui Wu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)

Abstract

In recent years, the rapid development of neuroimaging technology has been providing many powerful tools for cognitive neuroscience research. Among them, the functional magnetic resonance imaging (fMRI), which has high spatial resolution, acceptable temporal resolution, simple calibration, and short preparation time, has been widely used in brain research. Compared with the electroencephalogram (EEG), real-time fMRI-based brain computer interface (rtfMRI-BCI) not only can perform decoding analysis across the whole brain to control external devices, but also allows a subject to voluntarily self-regulate specific brain regions. This paper reviews the basic architecture of rtfMRI-BCI, the emerging machine learning based data analysis approaches (also known as multi-voxel pattern analysis), and the applications and recent advances of rtfMRI-BCI.

Keywords

Brain Computer Interface Functional Magnetic Resonance Imaging Machine learning Multi-voxel pattern analysis 

References

  1. 1.
    Andersson, P., Pluim, J.P., Viergever, M.A., Ramsey, N.F.: Navigation of a telepresence robot via covert visuospatial attention and real-time fMRI. Brain Topogr. 26(1), 177–185 (2013)CrossRefGoogle Scholar
  2. 2.
    Ashburner, J., Friston, K., et al.: The role of registration and spatial normalisation in detecting activations in functional imaging. Clin. MRI 7(1), 26–27 (1997)Google Scholar
  3. 3.
    Birbaumer, N., Cohen, L.G.: Brain-computer interfaces: communication and restoration of movement in paralysis. J. Physiol. 579(3), 621–636 (2007)CrossRefGoogle Scholar
  4. 4.
    Bleichner, M., Jansma, J., Salari, E., Freudenburg, Z., Raemaekers, M., Ramsey, N.: Classification of mouth movements using 7 T fMRI. J. Neural Eng. 12(6), 066026 (2015)CrossRefGoogle Scholar
  5. 5.
    Buyukturkoglu, K., Roettgers, H., Sommer, J., Rana, M., Dietzsch, L., Arikan, E.B., Veit, R., Malekshahi, R., Kircher, T., Birbaumer, N., et al.: Self-regulation of anterior insula with real-time fMRI and its behavioral effects in obsessive-compulsive disorder: a feasibility study. PLoS One 10(8), e0135872 (2015)CrossRefGoogle Scholar
  6. 6.
    Caria, A., Veit, R., Sitaram, R., Lotze, M., Weiskopf, N., Grodd, W., Birbaumer, N.: Regulation of anterior insular cortex activity using real-time fMRI. Neuroimage 35(3), 1238–1246 (2007)CrossRefGoogle Scholar
  7. 7.
    Carlson, T.A., Schrater, P., He, S.: Patterns of activity in the categorical representations of objects. J. Cogn. Neurosci. 15(5), 704–717 (2003)CrossRefGoogle Scholar
  8. 8.
    deCharms, R.C., Maeda, F., Glover, G.H., Ludlow, D., Pauly, J.M., Soneji, D., Gabrieli, J.D., Mackey, S.C.: Control over brain activation and pain learned by using real-time functional MRI. Proc. Natl. Acad. Sci. USA 102(51), 18626–18631 (2005)CrossRefGoogle Scholar
  9. 9.
    Cox, D.D., Savoy, R.L.: Functional magnetic resonance imaging (fMRI) brain reading: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19(2), 261–270 (2003)CrossRefGoogle Scholar
  10. 10.
    Davatzikos, C., Ruparel, K., Fan, Y., Shen, D., Acharyya, M., Loughead, J., Gur, R., Langleben, D.D.: Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. Neuroimage 28(3), 663–668 (2005)CrossRefGoogle Scholar
  11. 11.
    Deshpande, G., Rangaprakash, D., Oeding, L., Cichocki, A., Hu, X.P.: A new generation of brain-computer interfaces driven by discovery of latent EEG-fMRI linkages using tensor decomposition. Front. Neurosci. 11, 246 (2017)CrossRefGoogle Scholar
  12. 12.
    Eklund, A., Ohlsson, H., Andersson, M., Rydell, J., Ynnerman, A., Knutsson, H.: Using real-time fMRI to control a dynamical system by brain activity classification. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009. LNCS, vol. 5761, pp. 1000–1008. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04268-3_123 CrossRefGoogle Scholar
  13. 13.
    Friston, K.J., Frith, C.D., Frackowiak, R.S., Turner, R.: Characterizing dynamic brain responses with fMRI: a multivariate approach. Neuroimage 2(2), 166–172 (1995)CrossRefGoogle Scholar
  14. 14.
    Hanson, S.J., Matsuka, T., Haxby, J.V.: Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a face area? Neuroimage 23(1), 156–166 (2004)CrossRefGoogle Scholar
  15. 15.
    Haxby, J.V., Gobbini, M.I., Furey, M.L., Ishai, A., Schouten, J.L., Pietrini, P.: Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293(5539), 2425–2430 (2001)CrossRefGoogle Scholar
  16. 16.
    Haynes, J.D., Rees, G.: Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nat. Neurosci. 8(5), 686 (2005)CrossRefGoogle Scholar
  17. 17.
    Haynes, J.D., Rees, G.: Predicting the stream of consciousness from activity in human visual cortex. Curr. Biol. 15(14), 1301–1307 (2005)CrossRefGoogle Scholar
  18. 18.
    Haynes, J.D., Rees, G.: Decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 7(7), 523 (2006)CrossRefGoogle Scholar
  19. 19.
    Hollmann, M., Rieger, J.W., Baecke, S., Lützkendorf, R., Müller, C., Adolf, D., Bernarding, J.: Predicting decisions in human social interactions using real-time fMRI and pattern classification. PLoS One 6(10), e25304 (2011)CrossRefGoogle Scholar
  20. 20.
    Kamitani, Y., Tong, F.: Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8(5), 679–685 (2005)CrossRefGoogle Scholar
  21. 21.
    LaConte, S.M., Peltier, S.J., Hu, X.P.: Real-time fMRI using brain-state classification. Hum. Brain Mapp. 28(10), 1033–1044 (2007)CrossRefGoogle Scholar
  22. 22.
    Laufs, H.: A personalized history of EEG-fMRI integration. Neuroimage 62(2), 1056–1067 (2012)CrossRefGoogle Scholar
  23. 23.
    Lee, J.H., Ryu, J., Jolesz, F.A., Cho, Z.H., Yoo, S.S.: Brain-machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm. Neurosci. Lett. 450(1), 1–6 (2009)CrossRefGoogle Scholar
  24. 24.
    Mitchell, T.M., Hutchinson, R., Niculescu, R.S., Pereira, F., Wang, X., Just, M., Newman, S.: Learning to decode cognitive states from brain images. Mach. Learn. 57(1), 145–175 (2004)CrossRefMATHGoogle Scholar
  25. 25.
    Mwangi, B., Tian, T.S., Soares, J.C.: A review of feature reduction techniques in neuroimaging. Neuroinformatics 12(2), 229–244 (2014)CrossRefGoogle Scholar
  26. 26.
    Naito, M., Michioka, Y., Ozawa, K., Ito, Y., Kiguchi, M., Kanazawa, T.: A communication means for totally locked-in ALS patients based on changes in cerebral blood volume measured with near-infrared light. IEICE Trans. Inf. Syst. 90(7), 1028–1037 (2007)CrossRefGoogle Scholar
  27. 27.
    O’toole, A.J., Jiang, F., Abdi, H., Haxby, J.V.: Partially distributed representations of objects and faces in ventral temporal cortex. J. Cogn. Neurosci. 17(4), 580–590 (2005)CrossRefGoogle Scholar
  28. 28.
    Polyn, S.M., Natu, V.S., Cohen, J.D., Norman, K.A.: Category-specific cortical activity precedes retrieval during memory search. Science 310(5756), 1963–1966 (2005)CrossRefGoogle Scholar
  29. 29.
    Posse, S., Fitzgerald, D., Gao, K., Habel, U., Rosenberg, D., Moore, G.J., Schneider, F.: Real-time fMRI of temporolimbic regions detects Amygdala activation during single-trial self-induced sadness. Neuroimage 18(3), 760–768 (2003)CrossRefGoogle Scholar
  30. 30.
    Rota, G., Sitaram, R., Veit, R., Erb, M., Weiskopf, N., Dogil, G., Birbaumer, N.: Self-regulation of regional cortical activity using real-time fMRI: the right inferior frontal gyrus and linguistic processing. Hum. Brain Mapp. 30(5), 1605–1614 (2009)CrossRefGoogle Scholar
  31. 31.
    Ruiz, S., Birbaumer, N., Sitaram, R.: Abnormal neural connectivity in Schizophrenia and fMRI-brain-computer interface as a potential therapeutic approach. Front. Psychiatry 4, 17 (2013)CrossRefGoogle Scholar
  32. 32.
    Ruiz, S., Buyukturkoglu, K., Rana, M., Birbaumer, N., Sitaram, R.: Real-time fMRI brain computer interfaces: self-regulation of single brain regions to networks. Biol. Psychol. 95, 4–20 (2014)CrossRefGoogle Scholar
  33. 33.
    Sitaram, R., Caria, A., Veit, R., Gaber, T., Ruiz, S., Birbaumer, N.: Volitional control of the anterior insula in criminal psychopaths using real-time fMRI neurofeedback: a pilot study. Front. Behav. Neurosci. 8, 344 (2014)CrossRefGoogle Scholar
  34. 34.
    Sitaram, R., Weiskopf, N., Caria, A., Veit, R., Erb, M., Birbaumer, N.: fMRI brain-computer interfaces. IEEE Signal Process. Mag. 25(1), 95–106 (2008)CrossRefGoogle Scholar
  35. 35.
    Sitaram, R., Zhang, H., Guan, C., Thulasidas, M., Hoshi, Y., Ishikawa, A., Shimizu, K., Birbaumer, N.: Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage 34(4), 1416–1427 (2007)CrossRefGoogle Scholar
  36. 36.
    Sladky, R., Friston, K.J., Tröstl, J., Cunnington, R., Moser, E., Windischberger, C.: Slice-timing effects and their correction in functional MRI. Neuroimage 58(2), 588–594 (2011)CrossRefGoogle Scholar
  37. 37.
    Sorger, B., Dahmen, B., Reithler, J., Gosseries, O., Maudoux, A., Laureys, S., Goebel, R.: Another kind of bold response: answering multiple-choice questions via online decoded single-trial brain signals. Prog. Brain Res. 177, 275–292 (2009)CrossRefGoogle Scholar
  38. 38.
    Sorger, B., Reithler, J., Dahmen, B., Goebel, R.: A real-time fMRI-based spelling device immediately enabling robust motor-independent communication. Curr. Biol. 22(14), 1333–1338 (2012)CrossRefGoogle Scholar
  39. 39.
    Spiridon, M., Kanwisher, N.: How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35(6), 1157–1165 (2002)CrossRefGoogle Scholar
  40. 40.
    Subramanian, L., Hindle, J.V., Johnston, S., Roberts, M.V., Husain, M., Goebel, R., Linden, D.: Real-time functional magnetic resonance imaging neurofeedback for treatment of Parkinson’s disease. J. Neurosci. 31(45), 16309–16317 (2011)CrossRefGoogle Scholar
  41. 41.
    Sulzer, J., Haller, S., Scharnowski, F., Weiskopf, N., Birbaumer, N., Blefari, M.L., Bruehl, A.B., Cohen, L.G., Gassert, R., Goebel, R., et al.: Real-time fMRI neurofeedback: progress and challenges. Neuroimage 76, 386–399 (2013)CrossRefGoogle Scholar
  42. 42.
    Weiskopf, N., Mathiak, K., Bock, S.W., Scharnowski, F., Veit, R., Grodd, W., Goebel, R., Birbaumer, N.: Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 51(6), 966–970 (2004)CrossRefGoogle Scholar
  43. 43.
    Weiskopf, N., Scharnowski, F., Veit, R., Goebel, R., Birbaumer, N., Mathiak, K.: Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI). J. Physiol. Paris 98(4), 357–373 (2004)CrossRefGoogle Scholar
  44. 44.
    Weiskopf, N., Veit, R., Erb, M., Mathiak, K., Grodd, W., Goebel, R., Birbaumer, N.: Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data. Neuroimage 19(3), 577–586 (2003)CrossRefGoogle Scholar
  45. 45.
    Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Med. Image Anal. 1(1), 35–51 (1996)CrossRefGoogle Scholar
  46. 46.
    Yoo, S.S., Fairneny, T., Chen, N.K., Choo, S.E., Panych, L.P., Park, H., Lee, S.Y., Jolesz, F.A.: Brain-computer interface using fMRI: spatial navigation by thoughts. NeuroReport 15(10), 1591–1595 (2004)CrossRefGoogle Scholar
  47. 47.
    Yoo, S.S., Jolesz, F.A.: Functional mri for neurofeedback: feasibility studyon a hand motor task. NeuroReport 13(11), 1377–1381 (2002)CrossRefGoogle Scholar
  48. 48.
    Yu-Feng, Z., Yong, H., Chao-Zhe, Z., Qing-Jiu, C., Man-Qiu, S., Meng, L., Li-Xia, T., Tian-Zi, J., Yu-Feng, W.: Altered baseline brain activity in children with adhd revealed by resting-state functional MRI. Brain Dev. 29(2), 83–91 (2007)CrossRefGoogle Scholar
  49. 49.
    Zang, Y., Jiang, T., Lu, Y., He, Y., Tian, L.: Regional homogeneity approach to fMRI data analysis. Neuroimage 22(1), 394–400 (2004)CrossRefGoogle Scholar
  50. 50.
    Zhang, J., Wang, J., Wu, Q., Kuang, W., Huang, X., He, Y., Gong, Q.: Disrupted brain connectivity networks in drug-naive, first-episode major depressive disorder. Biol. Psychiatry 70(4), 334–342 (2011)CrossRefGoogle Scholar
  51. 51.
    Zich, C., Debener, S., Kranczioch, C., Bleichner, M.G., Gutberlet, I., De Vos, M.: Real-time EEG feedback during simultaneous EEG-fMRI identifies the cortical signature of motor imagery. Neuroimage 114, 438–447 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of AutomationHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations