Understanding the Semantic Structure of Human fMRI Brain Recordings with Formal Concept Analysis

  • Dominik Endres
  • Ruth Adam
  • Martin A. Giese
  • Uta Noppeney
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7278)


We investigate whether semantic information related to object categories can be obtained from human fMRI BOLD responses with Formal Concept Analysis (FCA). While the BOLD response provides only an indirect measure of neural activity on a relatively coarse spatio-temporal scale, it has the advantage that it can be recorded from humans, who can be questioned about their perceptions during the experiment, thereby obviating the need of interpreting animal behavioral responses. Furthermore, the BOLD signal can be recorded from the whole brain simultaneously. In our experiment, a single human subject was scanned while viewing 72 gray-scale pictures of animate and inanimate objects in a target detection task. These pictures comprise the formal objects for FCA. We computed formal attributes by learning a hierarchical Bayesian classifier, which maps BOLD responses onto binary features, and these features onto object labels. The connectivity matrix between the binary features and the object labels can then serve as the formal context. In line with previous reports, FCA revealed a clear dissociation between animate and inanimate objects with the inanimate category also including plants. Furthermore, we found that the inanimate category was subdivided between plants and non-plants when we increased the number of attributes extracted from the BOLD response. FCA also allows for the display of organizational differences between high-level and low-level visual processing areas. We show that subjective familiarity and similarity ratings are strongly correlated with the attribute structure computed from the BOLD signal.


fMRI inferior temporal cortex semantic neural decoding 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bell, A.H., Hadj-Bouziane, F., Frihauf, J.B., Tootell, R.B.H., Ungerleider, L.G.: Object representations in the temporal cortex of monkeys and humans as revealed by functional magnetic resonance imaging. Journal of Neurophysiology 101(2), 688–700 (2009), CrossRefGoogle Scholar
  2. 2.
    Bell, A.H., Malecek, N.J., Morin, E.L., Hadj-Bouziane, F., Tootell, R.B.H., Ungerleider, L.G.: Relationship between functional magnetic resonance imaging-identified regions and neuronal category selectivity. The Journal of Neuroscience 31(34), 12229–12240 (2011), CrossRefGoogle Scholar
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)Google Scholar
  4. 4.
    Brett, M., Anton, J., Valabregue, R., Poline, J.: Region of interest analysis using an SPM toolbox. Neuroimage 16(2) (2002); 8th International Conference on Functional Mapping of the Human BrainGoogle Scholar
  5. 5.
    Eger, E., Ashburner, J., Haynes, J.D., Dolan, R.J., Rees, G.: fMRI activity patterns in human LOC carry information about object exemplars within category. J. Cogn. Neurosci. 20(2), 356–370 (2008)CrossRefGoogle Scholar
  6. 6.
    Endres, D., Földiák, P., Priss, U.: An application of formal concept analysis to semantic neural decoding. Annals of Mathematics and Artificial Intelligence 57(3-4), 233–248 (2010), doi:10.1007/s10472-010-9196-8CrossRefGoogle Scholar
  7. 7.
    Evans, A.C., Marrett, S., Neelin, P., Collins, L., Worsley, K., Dai, W., Milot, S., Meyer, E., Bub, D.: Anatomical mapping of functional activation in stereotactic coordinate space. Neuroimage 1(1), 43–53 (1992)CrossRefGoogle Scholar
  8. 8.
    Földiák, P.: Sparse coding in the primate cortex. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn. MIT Press, Cambridge (2002)Google Scholar
  9. 9.
    Friston, K.J., Holmes, A.P., Price, C.J., Buchel, C., Worsley, K.J.: Multisubject fMRI studies and conjunction analyses. Neuroimage 10(4), 385–396 (1999)CrossRefGoogle Scholar
  10. 10.
    Friston, K., Holmes, A., Worsley, K., Poline, J., Frith, C., Frackowiak, R.: Statistical parametric mapping: a general linear approach. Hum. Brain Mapping 2, 189–210 (1995)CrossRefGoogle Scholar
  11. 11.
    Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical foundations. Springer (1999)Google Scholar
  12. 12.
    Harpur, G.F., Prager, R.W.: Experiments with low-entropy neural networks. In: Baddeley, R., Hancock, P., Földiák, P. (eds.) Information Theory and the Brain, ch. 5, pp. 84–100. Cambridge University Press, New York (2000)CrossRefGoogle Scholar
  13. 13.
    Haxby, J., Gobbini, M., Furey, M., Ishai, A., Schouten, J., Pietrini, P.: Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 5539(293), 2425–2430 (2001)CrossRefGoogle Scholar
  14. 14.
    Josephs, O., Turner, R., Friston, K.J.: Event-related fMRI. Human Brain Mapping 5, 243–248 (1997)CrossRefGoogle Scholar
  15. 15.
    Kandel, E.R., Schwartz, J.H., Jessell, T.M. (eds.): Principles of Neural Science, ch. 25-29. McGraw-Hill Education (2000)Google Scholar
  16. 16.
    Kiani, R., Esteky, H., Mirpour, K., Tanaka, K.: Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. Journal of Neurophysiology 97(6), 4296–4309 (2007)CrossRefGoogle Scholar
  17. 17.
    Knebel, J.F., Toepel, U., Hudry, J., le Coutre, J., Murray, M.M.: Generating controlled image sets in cognitive neuroscience research. Brain Topogr. 20(4), 284–289 (2008)CrossRefGoogle Scholar
  18. 18.
    Kriegeskorte, N., Mur, M., Ruff, D.A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., Bandettini, P.A.: Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60(6), 1126–1141 (2008)CrossRefGoogle Scholar
  19. 19.
    Lengnink, K.: Formalisierungen von Ähnlichkeit aus Sicht der Formalen Begriffsanalyse. Ph.D. thesis, Technische Hochschule Darmstadt, Fachbereich Mathematik (1996)Google Scholar
  20. 20.
    Lin, C.J.: Projected gradient methods for non-negative matrix factorization. Neural Computation 19, 2756–2779 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Mishkin, M., Ungerleider, L.G., Macko, K.A.: Object vision and spatial vision: two cortical pathways. Trends in Neurosciences 6, 414–417 (1983), CrossRefGoogle Scholar
  22. 22.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996)CrossRefGoogle Scholar
  23. 23.
    Op de Beeck, H.P., Haushofer, J., Kanwisher, N.G.: Interpreting fMRI data: maps, modules and dimensions. Nat. Rev. Neurosci. 9(2), 123–135 (2008)CrossRefGoogle Scholar
  24. 24.
    Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical recipes in C++: the art of scientific computing, 3rd edn. Cambridge University Press, New York (2007)Google Scholar
  25. 25.
    Tversky, A.: Features of similarity. Psychological Review 84(4), 327–352 (1977)CrossRefGoogle Scholar
  26. 26.
    Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the mni MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)CrossRefGoogle Scholar
  27. 27.
    Uludag, K., Dubowitz, D., Buxton, R.: Basic principals of functional MRI, pp. 249–287. Elsevier (2005)Google Scholar
  28. 28.
    Walther, D.B., Caddigan, E., Fei-Fei, L., Beck, D.M.: Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29(34), 10573–10581 (2009)CrossRefGoogle Scholar
  29. 29.
    Wille, R.: Restructuring lattice theory: an approach based on hierarchies of concepts. In: Rival, I. (ed.) Ordered Sets, pp. 445–470. Reidel, Dordrecht (1982)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dominik Endres
    • 1
  • Ruth Adam
    • 2
  • Martin A. Giese
    • 1
  • Uta Noppeney
    • 2
  1. 1.Sect. Computational Sensomotorics, Dept. Cognitive NeurologyCIN, HIH, BCCN and University Clinic TübingenTübingenGermany
  2. 2.Cognitive Neuroimaging GroupMax Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations