Abstract
Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.
Article PDF
Similar content being viewed by others
References
Alivisatos, B., & Petrides, M. (1997). Functional activation of the human brain during mental rotation. Neuropsychologia, 36, 111–118.
Amedi, A., Jacobson, G., Hendler, T., Malach, R., & Zohary, E. (2002). Convergence of visual and tactile shape processing in the human lateral occipital complex. Cerebral Cortex, 12, 1202–1212.
Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001). Visuo-haptic object-related activation in the ventral visual pathway. Nature Neuroscience, 4, 324–330.
Binkofski, F., Buccino, G., Posse, S., Seitz, R. J., Rizzolatti, G., & Freund, H.-J. (1999). A fronto-parietal circuit for object manipulation in man: Evidence from an f MRI-study. European Journal of Neuroscience, 11, 3276–3286.
Blake, R., Sobel, K. V., & James, T. W. (2004). Neural synergy between kinetic vision and touch. Psychological Science, 15, 397–402.
Blossfeldt, K. (1997a). The alphabet of plants (A. Wilde & J. Wilde, Eds.). New York: te Neues Publishing.
Blossfeldt, K. (1997b). Photography (A. Wilde & J. Wilde, Eds.). New York: Distributed Art Publishers.
Bodegård, A., Geyer, S., Grefkes, C., Zilles, K., & Roland, P. E. (2001). Hierarchical processing of tactile shape in the human brain. Neuron, 31, 317–328.
Bremmer, F., Schlack, A., Shah, N. J., Zafiris, O., Kubischik, M., Hoffmann, K., Zilles, K., & Fink, G. R. (2001). Polymodal motion processing in posterior parietal and premotor cortex: A human fMRI study strongly implies equivalencies between humans and monkeys. Neuron, 29, 287–296.
Brodatz, P. (1966). Textures: A photographic album for artists and designers. Mineola, NY: Dover.
Cohen, M. S., Kosslyn, S. M., Breiter, H. C., DiGirolamo, G. J., Thompson, W. L., Anderson, A. K., Bookheimer, S. Y., Rosen, B. R., & Belliveau, J. W. (1996). Changes in cortical activity during mental rotation: A mapping study using functional MRI. Brain, 119, 89–100.
De Volder, A. G., Toyama, H., Kimura, Y., Kiyosawa, M., Nakano, H., Vanlierde, A., Wanet-Defalque, M. C., Mishina, M., Oda, K., Ishiwata, K., & Senda, M. (2001). Auditory triggered mental imagery of shape involves visual association areas in early blind humans. Neuro-Image, 14, 129–139.
Duvernoy, H. M. (1999). The human brain: Surface, blood supply and three-dimensional sectional anatomy (2nd ed.). New York: Springer-Verlag.
Easton, R. D., Greene, A. J., & Srinivas, K. (1997). Transfer between vision and haptics: Memory for 2-D patterns and 3-D objects. Psychonomic Bulletin & Review, 4, 403–410.
Easton, R. D., Srinivas, K., & Greene, A. J. (1997). Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. Journal of Experimental Psycholology: Learning, Memory, & Cognition, 23, 153–163.
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22, 5749–5759.
Feinberg, T. E., Rothi, L. J., & Heilman, K. M. (1986). Multimodal agnosia after unilateral left hemisphere lesion. Neurology, 36, 864–867.
Feinstein, H. (2001). Foliage. Boston: Bulfinch Press.
Freides, D. (1974). Human information processing and sensory modality: Cross-modal functions, information complexity, memory and deficit. Psychological Bulletin, 81, 284–310.
Grant, A. C., Zangaladze, A., Thiagarajah, M. C., & Sathian, K. (1999). Tactile perception in developmental dyslexia: A psychophysical study using gratings. Neuropsychologia, 37, 1201–1211.
Grefkes, C., Weiss, P. H., Zilles, K., & Fink, G. R. (2002). Crossmodal processing of object features in human anterior intraparietal cortex: An f MRI study implies equivalencies between humans and monkeys. Neuron, 35, 173–184.
Grill-Spector, K., Kushnir, T., Hendler, T., Edelman, S., Itzchak, Y., & Malach, R. (1998). A sequence of object-processing stages revealed by f MRI in the human occipital lobe. Human Brain Mapping, 6, 316–328.
Hagen, M. C., Franzen, O., McGlone, F., Essick, G., Dancer, C., & Pardo, J. V. (2002). Tactile motion activates the human middle temporal/V5 (MT/V5) complex. European Journal of Neuroscience, 16, 957–964.
James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon, R. S., & Goodale, M. A. (2002). Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia, 40, 1706–1714.
Kilgour, A. R., de Gelder, B., & Lederman, S. J. (2004). Haptic face recognition and prosopagnosia. Neuropsychologia, 42, 707–712.
Klatzky, R. L., Lederman, S. J., & Reed, C. (1987). There’s more to touch than meets the eye: The salience of object attributes for haptics with and without vision. Journal of Experimental Psychology: General, 116, 356–369.
Lederman, S. J., Klatzky, R. L., Chataway, C., & Summers, C. D. (1990). Visual mediation and the haptic recognition of two-dimensional pictures of common objects. Perception & Psychophysics, 47, 54–64.
Leonards, U., Sunaert, S., Van Hecke, P., & Orban, G. A. (2000). Attention mechanisms in visual search: An f MRI study. Journal of Cognitive Neuroscience, 12, 61–75.
Lloyd, D. M., Shore, D. I., Spence, C., & Calvert, G. A. (2003). Multisensory representation of limb position in human premotor cortex. Nature Neuroscience, 6, 17–18.
Macaluso, E., Frith, C. D., & Driver, J. (2002). Directing attention to locations and to sensory modalities: Multiple levels of selective processing revealed with PET. Cerebral Cortex, 12, 357–368.
Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., Ledden, P. J., Brady, T. J. Rosen, B. R., & Tootell, R. B. H. (1995). Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proceedings of the National Academy of the Sciences, 92, 8135–8139.
Marks, D. F. (1973). Visual imagery differences in the recall of pictures. British Journal of Psychology, 64, 17–24.
Mugler, J. P., & Brookman, J. R. (1990). Three dimensional magnetization-prepared rapid gradient-echo imaging (3D MPRAGE). Magnetic Resonance Medicine, 15, 152–157.
Prather, S. C., Votaw, J. R., & Sathian, K. (2004). Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia, 42, 1079–1087.
Raczkowski, D., Kalat, J. W., & Nebes, R. (1974). Reliability and validity of some handedness questionnaire items. Neuropsychologia, 12, 43–47.
Reales, J. M., & Ballesteros, S. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 644–663.
Rock, I., & Victor, J. (1964). Vision and touch: An experimentally created conflict between the two senses. Science, 143, 594–596.
Rockland, K. S., & Ojima, H. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology, 50, 19–26.
Sathian, K., Cascio, C., Rice, D., Morris, M., Dancer, C., & Mc-Glone, F. (2003). Is tactile temporal processing impaired in developmental dyslexia? Cognitive Neuroscience Society Abstracts, E323. p. 192.
Sathian, K., Prather, S. C., & Zhang, M. (2004). Visual cortical involvement in normal tactile perception. In G. Calvert, C. Spence, & B. Stein (Eds.), The handbook of multisensory processes (pp. 703–709). Cambridge, MA: MIT Press.
Sathian, K., & Zangaladze, A. (2001). Feeling with the mind’s eye: The role of visual imagery in tactile perception. Optometry & Vision Science, 78, 276–281.
Sathian, K., Zangaladze, A., Hoffman, J. M., & Grafton, S. T. (1997). Feeling with the mind’s eye. NeuroReport, 8, 3877–3881.
Schroeder, C. E., Smiley, J., Fu, K. G., McGinnis, T., O’Connell, M. N., & Hackett, T. A. (2003). Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. International Journal of Psychophysiology, 50, 5–17.
Sergent, J., Ohta, S., & MacDonald, B. (1992). Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain, 115, 15–36.
Stoeckel, M. C., Weder, B., Binkofski, F., Buccino, G., Shah, N. J., & Seitz, R. J. (2003). A fronto-parietal circuit for tactile object discrimination: An event-related f MRI study. NeuroImage, 19, 1103–1114.
Stoesz, M., Zhang, M., Weisser, V. D., Prather, S. C., Mao, H., & Sathian, K. (2003). Neural networks active during tactile form perception: Common and differential activity during macrospatial and microspatial tasks. International Journal of Psychophysiology, 50, 41–49.
Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the brain. New York: Thieme Medical.
Vandenberghe, R., Gitelman, D. R., Parrish, T. B., & Mesulam, M. M. (2001). Functional specificity of superior parietal mediation of spatial shifting. NeuroImage, 14, 661–673.
Zangaladze, A., Epstein, C. M., Grafton, S. T., & Sathian, K. (1999). Involvement of visual cortex in tactile discrimination of orientation. Nature, 401, 587–590.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by Grant RO1 EY 12440 to K.S. from the National Eye Institute at the NIH.
Rights and permissions
About this article
Cite this article
Zhang, M., Weisser, V.D., Stilla, R. et al. Multisensory cortical processing of object shape and its relation to mental imagery. Cognitive, Affective, & Behavioral Neuroscience 4, 251–259 (2004). https://doi.org/10.3758/CABN.4.2.251
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/CABN.4.2.251