Cross-decoding supramodal information in the human brain
- 268 Downloads
Perceptual decision making is the cognitive process wherein the brain classifies stimuli into abstract categories for more efficient downstream processing. A system that, during categorization, can process information regardless of the information’s original sensory modality (i.e., a supramodal system) would have a substantial advantage over a system with dedicated processes for specific sensory modalities. While many studies have probed decision processes through the lens of one sensory modality, it remains unclear whether there are such supramodal brain areas that can flexibly process task-relevant information regardless of the original “format” of the information. To investigate supramodality, one must ensure that supramodal information exists somewhere within the functional architecture by rendering information from multiple sensory systems necessary but insufficient for categorization. To this aim, we tasked participants with categorizing auditory and tactile frequency-modulated sweeps according to learned, supramodal categories in a delayed match-to-category paradigm while we measured their blood-oxygen-level dependent signal with functional MRI. To detect supramodal information, we implemented a set of cross-modality pattern classification analyses, which demonstrated that the left caudate nucleus encodes category-level information but not stimulus-specific information (such as spatial directions and stimulus modalities), while the right inferior frontal gyrus, showing the opposite pattern, encodes stimulus-specific information but not category-level information. Given our paradigm, these results reveal abstract representations in the brain that are independent of motor, semantic, and sensory-specific processing, instead reflecting supramodal, categorical information, which points to the caudate nucleus as a locus of cognitive processes involved in complex behavior.
KeywordsCategorization Cross-decoding fMRI Perceptual decision making Supramodal
SML and JVS designed the experiment and acquired data. SML analyzed the data. SML and JVS wrote the manuscript.
Compliance with ethical standards
Conflict of interest
The authors declare no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535–574. https://doi.org/10.1146/annurev.neuro.29.051605.113038 CrossRefPubMedGoogle Scholar
- Handjaras G, Leo A, Cecchetti L et al (2017) Modality-independent encoding of individual concepts in the left parietal cortex. Neuropsychologia 105:39–49. https://doi.org/10.1016/j.neuropsychologia.2017.05.001 CrossRefPubMedGoogle Scholar
- Macmillan NA, Creelman CD (2005) Detection theory: a user’s guide, 2nd edn. Psychology, ErlbaumGoogle Scholar
- Menon V, Adleman NE, White CD et al (2001) Error-related brain activation during a Go/NoGo response inhibition task. Hum Brain Mapp 12:131–143. https://doi.org/10.1002/1097-0193(200103)12:3%3C131::AID-HBM1010%3E3.0.CO;2-C CrossRefPubMedGoogle Scholar
- Vettel JM, Green JR, Heller L, Tarr MJ (2016) Temporal and semantic effects on multisensory integration. arXiv:1606.05004
- Zaitsev M, Hennig J, Speck O (2004) Point spread function mapping with parallel imaging techniques and high acceleration factors: fast, robust, and flexible method for echo-planar imaging distortion correction. Magn Reson Med 52:1156–1166. https://doi.org/10.1002/mrm.20261 CrossRefPubMedGoogle Scholar