Decoding brain states using functional magnetic resonance imaging

  • Dongha Lee
  • Bumhee Park
  • Changwon Jang
  • Hae-Jeong Park
Review Article
First Online:
Received:
Accepted:

Abstract

Most leading research in basic and clinical neuroscience has been carried out by functional magnetic resonance imaging (fMRI), which detects the blood oxygenation level dependent signals associated with neural activities. Among new fMRI applications, brain decoding is an emerging research area, which infers mental states from fMRI signals. Brain decoding using fMRI includes classification, identification, and reconstruction of brain states. It is generally conducted using multi-voxel pattern analysis based on neuroscientific evidence that brain functions are mediated by distributed activation patterns. Brain decoding techniques have been successful in diverse applications such as the brain computer interface, patient monitoring, and neurofeedback. These techniques have expanded our understanding of how the brain encodes distinct information. In the current paper, we reviewed recent fMRI-based brain decoding techniques and applications. We also discussed the potential implications of brain decoding in neuroscience.

Keywords

fMRI Mind reading Brain decoding Neurofeedback Real time fMRI 
This is a preview of subscription content, log in to check access.

References

  1. [1]
    Ogawa S, Tank DW, Menon R, Ellermann JM, Kim SG, Merkle H, Ugurbil K. Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci USA. 1992; 89(13):5951–5955.CrossRefGoogle Scholar
  2. [2]
    Desmond JE, Sum JM, Wagner AD, Demb JB, Shear PK, Glover GH, Gabrieli JD, Morrell MJ. Functional MRI measurement of language lateralization in Wada-tested patients. Brain. 1995; 118(Pt 6):1411–1419.CrossRefGoogle Scholar
  3. [3]
    Roux FE, Boulanouar K, Ibarrola D, Tremoulet M, Chollet F, Berry I. Functional MRI and intraoperative brain mapping to evaluate brain plasticity in patients with brain tumours and hemiparesis. J Neurol Neurosur Psychiatr. 2000; 69(4):453–463.CrossRefGoogle Scholar
  4. [4]
    Johansen-Berg H, Dawes H, Guy C, Smith SM, Wade DT, Matthews PM. Correlation between motor improvements and altered fMRI activity after rehabilitative therapy. Brain. 2002; 125(Pt 12):2731–2742.CrossRefGoogle Scholar
  5. [5]
    Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science. 2001; 293(5539):2425–2430.CrossRefGoogle Scholar
  6. [6]
    Peters BO, Pfurtscheller G, Flyvbjerg H. Mining multi-channel EEG for its information content: an ANN-based method for a brain-computer interface. Neural Netw. 1998; 11(7–8):1429–1433.CrossRefGoogle Scholar
  7. [7]
    Hinterberger T, Veit R, Wilhelm B, Weiskopf N, Vatine JJ, Birbaumer N. Neuronal mechanisms underlying control of a brain-computer interface. Eur J Neurosci. 2005; 21(11):3169–3181.CrossRefGoogle Scholar
  8. [8]
    Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002; 113(6):767–791.CrossRefGoogle Scholar
  9. [9]
    Obermaier B, Muller GR, Pfurtscheller G. “Virtual keyboard” controlled by spontaneous EEG activity. IEEE Trans Neural Syst Rehabil Eng. 2003; 11(4):422–426.CrossRefGoogle Scholar
  10. [10]
    Kubler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, Birbaumer NP. Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch Phys Med Rehabil. 2001; 82(11):1533–1539.CrossRefGoogle Scholar
  11. [11]
    Yoo SS, Fairneny T, Chen NK, Choo SE, Panych LP, Park H, Lee SY, Jolesz FA. Brain-computer interface using fMRI: spatial navigation by thoughts. Neuroreport. 2004; 15(10):1591–1595.CrossRefGoogle Scholar
  12. [12]
    Lee JH, Ryu J, Jolesz FA, Cho ZH, Yoo SS. Brain-machine interface via real-time fMRI: preliminary study on thoughtcontrolled robotic arm. Neurosci Lett. 2009; 450(1):1–6.CrossRefGoogle Scholar
  13. [13]
    Owen AM, Coleman MR, Boly M, Davis MH, Laureys S, Pickard JD. Detecting awareness in the vegetative state. Science. 2006; 313(5792):1402.CrossRefGoogle Scholar
  14. [14]
    Monti MM, Vanhaudenhuyse A, Coleman MR, Boly M, Pickard JD, Tshibanda L, Owen AM, Laureys S. Willful modulation of brain activity in disorders of consciousness. New Engl J Med. 2010; 362(7):579–589.CrossRefGoogle Scholar
  15. [15]
    Misaki M, Kim Y, Bandettini PA, Kriegeskorte N. Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. NeuroImage. 2010; 53(1):103–118.CrossRefGoogle Scholar
  16. [16]
    Norman KA, Polyn SM, Detre GJ, Haxby JV. Beyond mindreading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci. 2006; 10(9):424–430.CrossRefGoogle Scholar
  17. [17]
    Haynes JD, Sakai K, Rees G, Gilbert S, Frith C, Passingham RE. Reading hidden intentions in the human brain. Curr Biol. 2007; 17(4):323–328.CrossRefGoogle Scholar
  18. [18]
    Haynes JD, Rees G. Predicting the stream of consciousness from activity in human visual cortex. Curr Biol. 2005; 15(14):1301–1307.CrossRefGoogle Scholar
  19. [19]
    Mitchell TM, Shinkareva SV, Carlson A, Chang KM, Malave VL, Mason RA, Just MA. Predicting human brain activity associated with the meanings of nouns. Science. 2008; 320(5880):1191–1195.CrossRefGoogle Scholar
  20. [20]
    Chadwick MJ, Hassabis D, Weiskopf N, Maguire EA. Decoding individual episodic memory traces in the human hippocampus. Curr Biol. 2010; 20(6):544–547.CrossRefGoogle Scholar
  21. [21]
    Meyer K, Kaplan JT, Essex R, Webber C, Damasio H, Damasio A. Predicting visual stimuli on the basis of activity in auditory cortices. Nat Neurosci. 2010; 13(6):667–668.CrossRefGoogle Scholar
  22. [22]
    Polyn SM, Natu VS, Cohen JD, Norman KA. Category-specific cortical activity precedes retrieval during memory search. Science. 2005; 310(5756):1963–1966.CrossRefGoogle Scholar
  23. [23]
    Kay KN, Naselaris T, Prenger RJ, Gallant JL. Identifying natural images from human brain activity. Nature. 2008; 452(7185):352–355.CrossRefGoogle Scholar
  24. [24]
    Naselaris T, Prenger RJ, Kay KN, Oliver M, Gallant JL. Bayesian reconstruction of natural images from human brain activity. Neuron. 2009; 63(6):902–915.CrossRefGoogle Scholar
  25. [25]
    Thirion B, Duchesnay E, Hubbard E, Dubois J, Poline JB, Lebihan D, Dehaene S. Inverse retinotopy: inferring the visual content of images from brain activation patterns. NeuroImage. 2006; 33(4):1104–1116.CrossRefGoogle Scholar
  26. [26]
    Miyawaki Y, Uchida H, Yamashita O, Sato MA, Morito Y, Tanabe HC, Sadato N, Kamitani Y. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron. 2008; 60(5):915–929.CrossRefGoogle Scholar
  27. [27]
    Friston KJ, Holmes AP, Poline JB, Grasby PJ, Williams SC, Frackowiak RS, Turner R. Analysis of fMRI time-series revisited. NeuroImage. 1995; 2(1):45–53.CrossRefGoogle Scholar
  28. [28]
    Cox RW, Jesmanowicz A, Hyde JS. Real-time functional magnetic resonance imaging. Magn Reson Med. 1995; 33(2):230–236.CrossRefGoogle Scholar
  29. [29]
    Park HJ, Park B, Kim DJ. Real-time functional MRI for patient monitoring during a language task. Conf Proc IEEE Eng Med Biol Soc. 2009; 1:5389–5392.Google Scholar
  30. [30]
    Weiskopf N, Mathiak K, Bock SW, Scharnowski F, Veit R, Grodd W, Goebel R, Birbaumer N. Principles of a braincomputer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE T Bio-Med Eng. 2004; 51(6):966–970.CrossRefGoogle Scholar
  31. [31]
    Kotchoubey B, Strehl U, Uhlmann C, Holzapfel S, Konig M, Froscher W, Blankenhorn V, Birbaumer N. Modification of slow cortical potentials in patients with refractory epilepsy: a controlled outcome study. Epilepsia. 2001; 42(3):406–416.CrossRefGoogle Scholar
  32. [32]
    Egner T, Gruzelier JH. Ecological validity of neurofeedback: modulation of slow wave EEG enhances musical performance. Neuroreport. 2003; 14(9):1221–1224.CrossRefGoogle Scholar
  33. [33]
    deCharms RC, Christoff K, Glover GH, Pauly JM, Whitfield S, Gabrieli JD. Learned regulation of spatially localized brain activation using real-time fMRI. NeuroImage. 2004; 21(1):436–443.CrossRefGoogle Scholar
  34. [34]
    Rota G, Handjaras G, Sitaram R, Birbaumer N, Dogil G. Reorganization of functional and effective connectivity during real-time fMRI-BCI modulation of prosody processing. Brain Lang. 2011; 117(3):123–132.CrossRefGoogle Scholar
  35. [35]
    Liu KP, Chan CC, Lee TM, Hui-Chan CW. Mental imagery for promoting relearning for people after stroke: a randomized controlled trial. Arch Phys Med Rehab. 2004; 85(9):1403–1408.CrossRefGoogle Scholar
  36. [36]
    Haller S, Birbaumer N, Veit R. Real-time fMRI feedback training may improve chronic tinnitus. Eur Radio. 2010; 20(3):696–703.CrossRefGoogle Scholar
  37. [37]
    Hinds O, Ghosh S, Thompson TW, Yoo JJ, Whitfield-Gabrieli S, Triantafyllou C, Gabrieli JD. Computing moment-to-moment BOLD activation for real-time neurofeedback. NeuroImage. 2011; 54(1):361–368.CrossRefGoogle Scholar
  38. [38]
    Gazzaniga MS. The law and neuroscience. Neuron. 2008; 60(3):412–415.CrossRefGoogle Scholar
  39. [39]
    Meegan DV. Neuroimaging techniques for memory detection: scientific, ethical, and legal issues. Am J Bioeth. 2008; 8(1):9–20.CrossRefGoogle Scholar
  40. [40]
    Sip KE, Roepstorff A, McGregor W, Frith CD. Detecting deception: the scope and limits. Trends Cognitive Science. 2008; 12(2):48–53.CrossRefGoogle Scholar
  41. [41]
    Gamer M, Bauermann T, Stoeter P, Vossel G. Covariations among fMRI, skin conductance, and behavioral data during processing of concealed information. Hum Brain Mapp. 2007; 28(12):1287–1301.CrossRefGoogle Scholar
  42. [42]
    Nose I, Murai J, Taira M. Disclosing concealed information on the basis of cortical activations. NeuroImage. 2009; 44(4):1380–1386.CrossRefGoogle Scholar

Copyright information

© © Korean Society of Medical and Biological Engineering and Springer 2011

Authors and Affiliations

  • Dongha Lee
    • 1
  • Bumhee Park
    • 1
  • Changwon Jang
    • 1
  • Hae-Jeong Park
    • 1
    • 2
  1. 1.Brain Korea 21 Project for Medical ScienceYonsei University College of MedicineSeoulRepublic of Korea
  2. 2.Department of Radiology, Nuclear Medicine, Severance Biomedical Science InstituteYonsei University College of MedicineSeoulRepublic of Korea

Personalised recommendations