Real-Time Functional MRI Classification of Brain States Using Markov-SVM Hybrid Models: Peering Inside the rt-fMRI Black Box

  • Ariana Anderson
  • Dianna Han
  • Pamela K. Douglas
  • Jennifer Bramen
  • Mark S. Cohen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7263)


Real-time functional MRI (rt-fMRI) methods provide the ability to predict and detect online changes in cognitive states. Applications require appropriate selection of features, preprocessing routines, and efficient computational models in order to be both practical to implement and deliver interpretable results. We predict video activity in nicotine-addicted subjects using both regional spatial averages and pre-constructed independent component spatial maps we refer to as an ”IC dictionary.” We found that this dictionary predicted better than the anatomical summaries and was less sensitive to preprocessing steps. When prior state information was incorporated using hybrid SVM-Markov models, the online models were able to predict even more accurately in real-time whether an individual was viewing a video while either resisting or indulging in nicotine cravings. Collectively, this work proposes and evaluates models that could be used for biofeedback. The IC dictionary offered an interpretable feature set proposing functional networks responsible for cognitive activity. We explore what is inside the black box of real-time fMRI, and examine both the advantages and shortcomings when machine learning methods are applied to predict and interpret cognitive states in the real-time context.


Independent Component Analysis Cognitive State Support Vector Machine Model fMRI Data Independent Component Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cohen, M.S., Weisskoff, R.M.: Ultra-fast imaging. Magn. Reson. Imaging 9, 1–37 (1991)CrossRefGoogle Scholar
  2. 2.
    Cox, R.W., Jesmanowicz, A., Hyde, J.S.: Real-time functional magnetic resonance imaging. Magn. Reson. Med. 33, 230–236 (1995)CrossRefGoogle Scholar
  3. 3.
    Gembris, D., Taylor, J.G., Schor, S., Frings, W., Suter, D., Posse, S.: Functional magnetic resonance imaging in real time (FIRE): sliding-window correlation analysis and reference-vector optimization. Magn. Reson. Med. 43, 259–268 (2000)CrossRefGoogle Scholar
  4. 4.
    Weiskopf, N., Sitaram, R., Josephs, O., Veit, R., Scharnowski, F., Goebel, R., Birbaumer, N., Deichmann, R., Mathiak, K.: Real-time functional magnetic resonance imaging: methods and applications. Magn. Reson. Imaging 25, 989–1003 (2007)CrossRefGoogle Scholar
  5. 5.
    Bleier, A.R., Jolesz, F.A., Cohen, M.S., Weisskoff, R.M., Dalcanton, J.J., Higuchi, N., Feinberg, D.A., Rosen, B.R., McKinstry, R.C., Hushek, S.G.: Real-time magnetic resonance imaging of laser heat deposition in tissue. Magn. Reson. Med. 21, 132–137 (1991)CrossRefGoogle Scholar
  6. 6.
    Voyvodic, J.T.: Real-time fMRI paradigm control, physiology, and behavior combined with near real-time statistical analysis. Neuroimage 10, 91–106 (1999)CrossRefGoogle Scholar
  7. 7.
    Cohen, M.S.: Real-time functional magnetic resonance imaging. Methods 25(2), 201–220 (2001)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Goddard, N.H., Cohen, J.D., Eddy, W.F., Genovese, C.R., Noll, D.C., Nystrom, L.E.: Online analysis of functional MRI datasets on parallel platforms. The Journal of Supercomputing 11, 295–318 (1997)CrossRefGoogle Scholar
  9. 9.
    Grill-Spector, K., Sayres, R., Ress, D.: High-resolution imaging reveals highly selective nonface clusters in the fusiform face area. Nat. Neurosci. 9, 1177–1185 (2006)CrossRefGoogle Scholar
  10. 10.
    Schneider, W., Noll, D.C., Cohen, J.D.: Functional topographic mapping of the cortical ribbon in human vision with conventional MRI scanners. Nature 365, 150–153 (1993)CrossRefGoogle Scholar
  11. 11.
    Gasser, T., Ganslandt, O., Sandalcioglu, E., Stolke, D., Fahlbusch, R., Nimsky, C.: Intraoperative functional MRI: implementation and preliminary experience. Neuroimage 26, 685–693 (2005)CrossRefGoogle Scholar
  12. 12.
    Gasser, T., Sandalcioglu, E., Schoch, B., Gizewski, E., Forsting, M., Stolke, D., Wiedemayer, H.: Functional magnetic resonance imaging in anesthetized patients: a relevant step toward real-time intraoperative functional neuroimaging. Neurosurgery 57, 94–99 (2005)Google Scholar
  13. 13.
    Gasser, T., Szelenyi, A., Senft, C., Muragaki, Y., Sandalcioglu, I.E., Sure, U., Nimsky, C., Seifert, V.: Intraoperative MRI and functional mapping. Acta Neurochir. Suppl. 109, 61–65 (2011)CrossRefGoogle Scholar
  14. 14.
    Schwindack, C., Siminotto, E., Meyer, M., McNamara, A., Marshall, I., Wardlaw, J.M., Whittle, I.R.: Real-time functional magnetic resonance imaging (rt-fMRI) in patients with brain tumours: preliminary findings using motor and language paradigms. Br. J. Neurosurg. 19, 25–32 (2005)CrossRefGoogle Scholar
  15. 15.
    Gering, D.T., Weber, D.M.: Intraoperative, real-time, functional MRI. J. Magn. Reson. Imaging 8, 254–257 (1998)CrossRefGoogle Scholar
  16. 16.
    Moller, M., Freund, M., Greiner, C., Schwindt, W., Gaus, C., Heindel, W.: Real time fMRI: a tool for the routine presurgical localisation of the motor cortex. Eur. Radiol. 15, 292–295 (2005)CrossRefGoogle Scholar
  17. 17.
    Hong, X., Rohan, M., Cohen, M.S., Terwilliger, R., Roemer, P.: Real-time observation of mental activity: the autocerebroscope (1997)Google Scholar
  18. 18.
    DeCharms, C.R.: Applications of real-time fMRI. Nat. Rev. Neurosci. 9(9), 720–729 (2008)CrossRefGoogle Scholar
  19. 19.
    LaConte, S.M.: Decoding fMRI brain states in real-time. Neuroimage 56, 440–454 (2011)CrossRefGoogle Scholar
  20. 20.
    Caria, A., Veit, R., Sitaram, R., Lotze, M., Weiskopf, N., Grodd, W., Birbaumer, N.: Regulation of anterior insular cortex activity using real-time fMRI. Neuroimage 35, 1238–1246 (2007)CrossRefGoogle Scholar
  21. 21.
    Yoo, S.-S., Fairneny, T., Chen, N.-K., Choo, S.-E., Panych, L.P., Park, H., Lee, S.-Y., Jolesz, F.A.: Brain-computer interface using fMRI: Spatial navigation by thoughts 15(10), 1591–1595 (2004)Google Scholar
  22. 22.
    LaConte, S.M., Peltier, S.J., Hu, X.P.: Real-time fMRI using brain-state classification. Hum. Brain Mapp. 28, 1033–1044 (2007)CrossRefGoogle Scholar
  23. 23.
    Ohlsson, H., Rydell, J., Brun, A., Roll, J., Andersson, M., Ynnerman, A., Knutsson, H.: Enabling bio-feedback using real-time fMRIGoogle Scholar
  24. 24.
    DeCharms, R.C.: Reading and controlling human brain activation using real-time functional magnetic resonance imaging. Trends Cogn. Sci. (2007)Google Scholar
  25. 25.
    DeCharms, R.C., Maeda, F., Glover, G.H., Ludlow, D., Pauly, J.M., Soneji, D., Gabrieli, J.D., Mackey, S.C.: Control over brain activation and pain learned by using real-time functional MRI. Proc. Natl. Acad. Sci. U.S.A. 102, 18626–18631 (2005)CrossRefGoogle Scholar
  26. 26.
    Weiskopf, N., Sitaram, R., Josephs, O., Veit, R., Scharnowski, F., Goebel, R., Birbaumer, N., Deichmann, R., Mathiak, K.: Real-time functional magnetic resonance imaging: methods and applications. Magn. Reson. Imaging (2007)Google Scholar
  27. 27.
    Zarahn, E., Aguirre, G.K., D’Esposito, M.: Empirical analyses of BOLD fMRI statistics. NeuroImage 5, 179–197 (1997)CrossRefGoogle Scholar
  28. 28.
    Cohen, M.S., DuBois, R.M.: Stability, repeatability, and the expression of signal magnitude in functional magnetic resonance imaging. J. Magn. Reson. Imaging 10, 33–40 (1999)CrossRefGoogle Scholar
  29. 29.
    Monti, M.M.: Statistical analysis of fMRI time-series: A critical evaluation of the GLM approach. Preprint Submitted to Frontiers Special Topics (2006)Google Scholar
  30. 30.
    Pereira, F., Mitchell, T., Botvinick, M.: Machine learning classifiers and fMRI: a tutorial overview. NeuroImage 45(1 suppl.) (March 2009)Google Scholar
  31. 31.
    Douglas, P.K., Harris, S., Yuille, A., Cohen, M.S.: Performance comparison of machine learning algorithms and number of independent components used in fMRI decoding of belief vs. disbelief. Neuroimage 56, 544–553 (2011)CrossRefGoogle Scholar
  32. 32.
    Liu, W., Zhu, P., Anderson, J.S., Yurgelun-Todd, D., Fletcher, P.T.: Spatial Regularization of Functional Connectivity Using High-Dimensional Markov Random Fields. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 363–370. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  33. 33.
    Svensén, M., Kruggel, F., von Cramon, D.Y.: Markov Random Field Modelling of fMRI Data Using a Mean Field EM-algorithm. In: Hancock, E.R., Pelillo, M. (eds.) EMMCVPR 1999. LNCS, vol. 1654, pp. 317–330. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  34. 34.
    Nagori, M.B., Mane, T.N., Agrawal, S.A., Joshi, M.S.: Evaluation of markov blanket algorithms for fMRI data analysis. In: International Conference on Information and Network Technology (2011)Google Scholar
  35. 35.
    Nan, F., Wang, Y., Ma, X.: fMRI Activation Detection by MultiScale Hidden Markov Model. In: Rajasekaran, S. (ed.) BICoB 2009. LNCS, vol. 5462, pp. 295–306. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  36. 36.
    Woolrich, M.W., Behrens, T.E., Beckmann, C.F., Jenkinson, M., Smith, S.M.: Multilevel linear modelling for FMRI group analysis using Bayesian inference. Neuroimage 21, 1732–1747 (2004)CrossRefGoogle Scholar
  37. 37.
    Hyvärinen, A., Oja, E.: Independent component analysis: Algorithms and applications. Neural Networks 13(4-5), 411–430 (2000)CrossRefGoogle Scholar
  38. 38.
    Mckeown, M., Makeig, S., Brown, G., Jung, T., Kindermann, S., Bell, A., Sejnowski, T.: Analysis of fMRI data by blind separation into independent spatial components (1998)Google Scholar
  39. 39.
    Esposito, F., Seifritz, E., Formisano, E., Morrone, R., Scarabino, T., Tedeschi, G., Cirillo, S., Goebel, R., Di Salle, F.: Real-time independent component analysis of fMRI time-series. Neuroimage 20, 2209–2224 (2003)CrossRefGoogle Scholar
  40. 40.
    De Martino, F., Valente, G., Staeren, N., Ashburner, J., Goebel, R., Formisano, E.: Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage 43, 44–58 (2008)CrossRefGoogle Scholar
  41. 41.
    Anderson, A., Bramen, J., Douglas, P.K., Lenartowicz, A., Cho, A., Culbertson, C., Brody, A.L., Yuille, A.L., Cohen, M.S.: Large sample group independent component analysis of functional magnetic resonance imaging using anatomical atlas-based reduction and bootstrapped clustering. International Journal of Imaging Systems and Technology 21(2), 223–231 (2011)CrossRefGoogle Scholar
  42. 42.
    Brody, A.L., Mandelkern, M.A., Olmstead, R.E., Jou, J., Tiongson, E., Allen, V., Scheibal, D., London, E.D., Monterosso, J.R., Tiffany, S.T., Korb, A., Gan, J.J., Cohen, M.S.: Neural substrates of resisting craving during cigarette cue exposure. Biol. Psychiatry 62, 642–651 (2007)CrossRefGoogle Scholar
  43. 43.
    Smith, S.M., et al.: Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage 23, 208–219 (2004)CrossRefGoogle Scholar
  44. 44.
    Desikan, R.S., Segonne, F., Fischl, B., Quinn, B.T., Dickerson, B.C., Blacker, D., Buckner, R.L., Dale, A.M., Maguire, R.P., Hyman, B.T., Albert, M.S., Killiany, R.J.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006)CrossRefGoogle Scholar
  45. 45.
    Smith, S.M., Fox, P.T., Miller, K.L., Glahn, D.C., Mickle Fox, P., Mackay, C.E., Filippini, N., Watkins, K.E., Toro, R., Laird, A.R., Beckmann, C.F.: Correspondence of the brain’s functional architecture during activation and rest. Proceedings of the National Academy of Sciences of the United States of America 106(31), 13040–13045 (2009)CrossRefGoogle Scholar
  46. 46.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning, 273–297 (1995)Google Scholar
  47. 47.
    Garczarek, U.M.: Classification rules in standardized partition spaces. Doctoral Dissertation: University of Dortmund (2002)Google Scholar
  48. 48.
    Chang, Y.-W., Lin, C.-J.: Feature ranking using linear svm. Journal of Machine Learning Research - Proceedings Track 3, 53–64 (2008)Google Scholar
  49. 49.
    Maldjian, J.A., Laurienti, P.J., Kraft, R.A., Burdette, J.H.: An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. Neuroimage 19, 1233–1239 (2003)CrossRefGoogle Scholar
  50. 50.
    Marrelec, G., Fransson, P.: Assessing the influence of different ROI selection strategies on functional connectivity analyses of fMRI data acquired during steady-state conditions. PLoS ONE 6, e14788 (2011)CrossRefGoogle Scholar
  51. 51.
    Margulies, D.S., Vincent, J.L., Kelly, C., Lohmann, G., Uddin, L.Q., Biswal, B.B., Villringer, A., Castellanos, F.X., Milham, M.P., Petrides, M.: Precuneus shares intrinsic functional architecture in humans and monkeys. Proc. Natl. Acad. Sci. U.S.A. 106, 20069–20074 (2009)CrossRefGoogle Scholar
  52. 52.
    Barkhof, F., Scheltens, P., Stam, C.J., Smith, S.M., Beckmann, C.F., Damoiseaux, J.S., Rombouts, S.A.: Consistent resting-state networks across healthy subjects. Proc. National Academy of Science (2006)Google Scholar
  53. 53.
    Scoville, W.B., Milner, B., Scoville, W.B., Miller, B.: Loss of recent memory after bilateral hippocampal lesions. J. Neuropsychiatry Clin. Neurosci. 12, 103–113 (1957, 2000)CrossRefGoogle Scholar
  54. 54.
    Ekstrom, A.D., Kahana, M.J., Caplan, J.B., Fields, T.A., Isham, E.A., Newman, E.L., Fried, I.: Cellular networks underlying human spatial navigation. Nature 425, 184–188 (2003)CrossRefGoogle Scholar
  55. 55.
    Squire, L.R., Knowlton, B.J.: The medial temporal lobe, the hippocampus, and the memory systems of the brain. Memory, 765–779Google Scholar
  56. 56.
    Shams, L., Kamitani, Y., Shimojo, S.: Illusions. What you see is what you hear. Nature 408, 788 (2000)CrossRefGoogle Scholar
  57. 57.
    Gall, F.J.: Anatomie et physiologie du système nerveux en général et du cerveau en particulier: avec des observations sur la possibilité de reconnoitre plusieurs dispositions intellectuelles et morales de l’homme et des animaux. Chez N. Maze, libraire (1819)Google Scholar
  58. 58.
    Pribram, K.H.: Languages of the brain: experimental paradoxes and principles in neuropsychology. Brandon House (1981)Google Scholar
  59. 59.
    Kriegeskorte, N., Goebel, R., Bandettini, P.: Information-based functional brain mapping. Proc. Natl. Acad. Sci. U.S.A. 103, 3863–3868 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ariana Anderson
  • Dianna Han
  • Pamela K. Douglas
  • Jennifer Bramen
  • Mark S. Cohen

There are no affiliations available

Personalised recommendations