Brain Topography

, Volume 21, Issue 3–4, pp 269–274 | Cite as

A Putative Model of Multisensory Object Representation

  • Simon Lacey
  • Noa Tal
  • Amir Amedi
  • K. SathianEmail author


This review surveys the recent literature on visuo-haptic convergence in the perception of object form, with particular reference to the lateral occipital complex (LOC) and the intraparietal sulcus (IPS) and discusses how visual imagery or multisensory representations might underlie this convergence. Drawing on a recent distinction between object- and spatially-based visual imagery, we propose a putative model in which LOtv, a subregion of LOC, contains a modality-independent representation of geometric shape that can be accessed either bottom-up from direct sensory inputs or top-down from frontoparietal regions. We suggest that such access is modulated by object familiarity: spatial imagery may be more important for unfamiliar objects and involve IPS foci in facilitating somatosensory inputs to the LOC; by contrast, object imagery may be more critical for familiar objects, being reflected in prefrontal drive to the LOC.


Cross-modal Visual Haptic Imagery 


  1. Alivisatos B, Petrides M (1997) Functional activation of the human brain during mental rotation. Neuropsychologia 36:11–118Google Scholar
  2. Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330PubMedCrossRefGoogle Scholar
  3. Amedi A, Jacobson G, Hendler T, Malach R, Zohary E (2002) Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb Cortex 12:1202–1212PubMedCrossRefGoogle Scholar
  4. Amedi A, Raz N, Pianka P, Malach R, Zohary E (2003) Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind. Nat Neurosci 6:758–766PubMedCrossRefGoogle Scholar
  5. Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci 10:687–689PubMedCrossRefGoogle Scholar
  6. Arno P, De Volder AG, Vanlierde A, Wanet-Defalque M-C, Streel E, Robert A, Sanabria-Bohórquez S, Veraart C (2001) Occipital activation by pattern recognition in the early blind using auditory substitution for vision. Neuroimage 13:632–645PubMedCrossRefGoogle Scholar
  7. Banati RB, Goerres GW, Tjoa C, Aggleton JP, Grasby P (2000) The functional anatomy of visual-tactile integration in man: a study using positron emission tomography. Neuropsychologia 38:115–124PubMedCrossRefGoogle Scholar
  8. Blajenkova O, Kozhevnikov M, Motes MA (2006) Object-spatial imagery: a new self-report imagery questionnaire. Appl Cognit Psychol 20:239–263CrossRefGoogle Scholar
  9. Burton H, Snyder AZ, Conturo TE, Akbudak E, Ollinger JM, Raichle ME (2002) Adaptive changes in early and late blind: a fMRI study of Braille reading. J Neurophysiol 87:589–607PubMedGoogle Scholar
  10. Carpenter PA, Eisenberg P (1978) Mental rotation and the frame of reference in blind and sighted individuals. Percept Psychophys 23:117–124PubMedGoogle Scholar
  11. De Volder AG, Toyama H, Kimura Y, Kiyosawa M, Nakano H, Vanlierde A, Wanet-Defalque M-C, Mishima M, Oda K, Ishiwata K, Senda M (2001) Auditory triggered mental imagery of shape involves visual association areas in early blind humans. Neuroimage 14:129–139PubMedCrossRefGoogle Scholar
  12. Dellantonio A, Spagnolo F (1990) Mental rotation of tactual stimuli. Acta Psychol 73:245–257CrossRefGoogle Scholar
  13. Deshpande G, Hu X, Stilla R, Sathian K (2008) Effective connectivity during haptic perception: a study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 40:1807–1814PubMedCrossRefGoogle Scholar
  14. Easton RD, Greene AJ, Srinivas K (1997a) Transfer between vision and haptics: memory for 2-D patterns and 3-D objects. Psychon Bull Rev 4:403–410Google Scholar
  15. Easton RD, Srinivas K, Greene AJ (1997b) Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. J Exp Psychol Learn 23:153–163CrossRefGoogle Scholar
  16. Feinberg TE, Rothi LJ, Heilman KM (1986) Multimodal agnosia after unilateral left hemisphere lesion. Neurology 36:864–867PubMedGoogle Scholar
  17. Gibson G, Stilla R, Sathian K (2008) Segregated visuo-haptic processing of texture and location. Abstract, Human Brain MappingGoogle Scholar
  18. Grefkes C, Fink G (2005) The functional organization of the intraparietal sulcus in humans and monkeys. J Anat 207:3–17PubMedCrossRefGoogle Scholar
  19. Grefkes C, Geyer S, Schormann T, Roland P, Zilles K (2001) Human somatosensory area 2: observer-independent cytoarchitectonic mapping, interindividual variability, and population map. Neuroimage 14:617–631PubMedCrossRefGoogle Scholar
  20. Grefkes C, Weiss PH, Zilles K, Fink GR (2002) Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron 35:173–184PubMedCrossRefGoogle Scholar
  21. Grefkes C, Ritzl A, Zilles K, Fink GR (2004) Human medial intraparietal cortex subserves visuomotor coordinate transformation. Neuroimage 23:1494–1506PubMedCrossRefGoogle Scholar
  22. Grill-Spector K, Henson R, Martin A (2006) Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn Sci 10:14–23PubMedCrossRefGoogle Scholar
  23. Hadjikhani N, Roland PE (1998) Cross-modal transfer of information between the tactile and the visual representations in the human brain: a positron emission tomographic study. J Neurosci 18:1072–1084PubMedGoogle Scholar
  24. Hollins M (1986) Haptic mental rotation: more consistent in blind subjects? J Visual Impair Blin 80:950–952Google Scholar
  25. Iwamura Y (1998) Hierarchical somatosensory processing. Curr Opin Neurobiol 8:522–528PubMedCrossRefGoogle Scholar
  26. James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714PubMedCrossRefGoogle Scholar
  27. James TW, James KH, Humphrey GK, Goodale MA (2006) Do visual and tactile object representations share the same neural substrate? In: Heller MA, Ballesteros S (eds) Touch and blindness: psychology and neuroscience. Lawrence Erlbaum Associates, Mahwah, NJ, pp 139–155Google Scholar
  28. Kosslyn SM (1973) Scanning visual images: some structural implications. Percept Psychophys 14:90–94Google Scholar
  29. Kosslyn SM, Ball TM, Reiser BJ (1978) Visual images preserve metric spatial information: evidence from studies of image scanning. J Exp Psychol Human 4:47–60CrossRefGoogle Scholar
  30. Kourtzi Z, Kanwisher N (2000) Cortical regions involved in perceiving object shape. J Neurosci 20:3310–3318PubMedGoogle Scholar
  31. Kourtzi Z, Kanwisher N (2001) Representation of perceived object shape by the human lateral occipital complex. Science 293:1506–1509PubMedCrossRefGoogle Scholar
  32. Kourtzi Z, Tolias AS, Altmann CF, Augath M, Logothetis NK (2003) Integration of local features into global shapes: monkey and human fMRI studies. Neuron 37:333–346PubMedCrossRefGoogle Scholar
  33. Kozhevnikov M, Hegarty M, Mayer RE (2002) Revising the visualiser-verbaliser dimension: evidence for two types of visualisers. Cognit Instruct 20:47–77CrossRefGoogle Scholar
  34. Kozhevnikov M, Kosslyn SM, Shephard J (2005) Spatial versus object visualisers: a new characterisation of cognitive style. Mem Cognit 33:710–726PubMedGoogle Scholar
  35. Krekelberg B, Boynton GM, van Wezel RJA (2006) Adaptation: from single cells to BOLD signals. Trends Neurosci 29:250–256PubMedCrossRefGoogle Scholar
  36. Lacey S, Campbell C (2006) Mental representation in visual/haptic crossmodal memory: evidence from interference effects. Q J Exp Psychol 59:361–376CrossRefGoogle Scholar
  37. Lacey S, Campbell C, Sathian K (2007a) Vision and touch: multiple or multisensory representations of objects? Perception 36:1513–1521PubMedCrossRefGoogle Scholar
  38. Lacey S, Peters A, Sathian K (2007b) Cross-modal object representation is viewpoint-independent. PLoS ONE 2:e890. doi: 10.1371/journal.pone0000890 PubMedCrossRefGoogle Scholar
  39. Large M-E, Aldcroft A, Vilis T (2007) Task-related laterality effects in the lateral occipital complex. Brain Res 1128:130–138PubMedCrossRefGoogle Scholar
  40. Lucan JN, Foxe JJ, Gomez-Ramirez M, Sathian K, Molholm S (2008) The spatio-temporal dynamics of somatosensory shape discrimination. Abstract, International Multisensory Research ForumGoogle Scholar
  41. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RBH (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA 92:8135–8139PubMedCrossRefGoogle Scholar
  42. Marmor GS, Zaback LA (1976) Mental rotation by the blind: does mental rotation depend on visual imagery? J Exp Psychol Human 2:515–521CrossRefGoogle Scholar
  43. Motes MA, Malach R, Kozhevnikov M (2008) Object-processing neural efficiency differentiates object from spatial visualizers. NeuroReport 19:1727–1731PubMedCrossRefGoogle Scholar
  44. Newman SD, Klatzky RL, Lederman SJ, Just MA (2005) Imagining material versus geometric properties of objects: an fMRI study. Cognit Brain Res 23:235–246CrossRefGoogle Scholar
  45. Pascual-Leone A, Hamilton RH (2001) The metamodal organization of the brain. Prog Brain Res 134:427–445PubMedCrossRefGoogle Scholar
  46. Pascual-Leone A, Amedi A, Fregni F, Merabet LB (2005) The plastic human brain. Annu Rev Neurosci 28:377–401PubMedCrossRefGoogle Scholar
  47. Peltier S, Stilla R, Mariola E, LaConte S, Hu X, Sathian K (2007) Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia 45:476–483PubMedCrossRefGoogle Scholar
  48. Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu W-HC, Cohen L, Guazzelli M, Haxby JV (2004) Beyond sensory images: object-based representation in the human ventral pathway. Proc Natl Acad Sci USA 101:5658–5663PubMedCrossRefGoogle Scholar
  49. Prather SC, Votaw JR, Sathian K (2004) Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia 42:1079–1087PubMedCrossRefGoogle Scholar
  50. Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn 25:644–663CrossRefGoogle Scholar
  51. Reed CL, Shoham S, Halgren E (2004) Neural substrates of tactile object recognition: an fMRI study. Hum Brain Mapp 21:236–246PubMedCrossRefGoogle Scholar
  52. Renier L, Collignon O, Tranduy D, Poirier C, Vanlierde A, Veraart C, De Volder A (2004) Visual cortex activation in early blind and sighted subjects using an auditory visual substitution device to perceive depth. Neuroimage 22:S1CrossRefGoogle Scholar
  53. Renier L, Collignon O, Poirier C, Tranduy D, Vanlierde A, Bol A, Veraart C, De Volder AG (2005) Cross modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26:573–580PubMedCrossRefGoogle Scholar
  54. Röder B, Rösler F (1998) Visual input does not facilitate the scanning of spatial images. J Ment Imag 22:165–181Google Scholar
  55. Saito DN, Okada T, Morita Y, Yonekura Y, Sadato N (2003) Tactile-visual cross-modal shape matching: a functional MRI study. Cognit Brain Res 17:14–25CrossRefGoogle Scholar
  56. Sathian K (2004) Modality, quo vadis?: comment. Behav Brain Sci 27:413–414CrossRefGoogle Scholar
  57. Sathian K (2005) Visual cortical activity during tactile perception in the sighted and the visually deprived. Dev Psychobiol 46:279–286PubMedCrossRefGoogle Scholar
  58. Sathian K, Lacey S (2007) Journeying beyond classical somatosensory cortex. Can J Exp Psychol 61:254–264PubMedGoogle Scholar
  59. Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8:3877–3881PubMedCrossRefGoogle Scholar
  60. Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171:701–703PubMedCrossRefGoogle Scholar
  61. Shikata E, McNamara A, Sprenger A, Hamzei F, Glauche V, Büchel C, Binkofski F (2008) Localization of human intraparietal areas AIP, CIP, and LIP using surface orientation and saccadic eye movement tasks. Hum Brain Mapp 29:411–421PubMedCrossRefGoogle Scholar
  62. Stilla R, Sathian K (2008) Selective visuo-haptic processing of shape and texture. Hum Brain Mapp 29:1123–1138PubMedCrossRefGoogle Scholar
  63. Stilla R, Hanna R, Hu X, Mariola E, Deshpande G, Sathian K (2008a) Neural processing underlying tactile microspatial discrimination in the blind: a functional magnetic resonance imaging study. J Vis 8:1–19. doi: 10.1167/8.10.13 PubMedCrossRefGoogle Scholar
  64. Stilla R, Lacey S, Deshpande G, Hu X, Sathian K (2008b) Effective connectivity during haptic shape perception is modulated by object familiarity. Abstract, Society for NeuroscienceGoogle Scholar
  65. Stoesz M, Zhang M, Weisser VD, Prather SC, Mao H, Sathian K (2003) Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks. Int J Psychophysiol 50:41–49PubMedCrossRefGoogle Scholar
  66. Swisher JD, Halko MA, Merabet LB, McMains SA, Somers DC (2007) Visual topography of human intraparietal sulcus. J Neurosci 27:5326–5337PubMedCrossRefGoogle Scholar
  67. Tal N, Amedi A (2008) Using fMR-adaptation to study visual-tactile integration of objects in humans. Abstract, Israel Society for NeuroscienceGoogle Scholar
  68. Zhang M, Weisser VD, Stilla R, Prather SC, Sathian K (2004) Multisensory cortical processing of object shape and its relation to mental imagery. Cogn Affect Behav Ne 4:251–259CrossRefGoogle Scholar
  69. Zhou Y-D, Fuster JM (1997) Neuronal activity of somatosensory cortex in a cross-modal (visuo-haptic) memory task. Exp Brain Res 116:551–555PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Simon Lacey
    • 1
  • Noa Tal
    • 2
  • Amir Amedi
    • 2
    • 3
  • K. Sathian
    • 1
    • 4
    • 5
    • 6
    Email author
  1. 1.Department of NeurologyEmory University School of MedicineAtlantaUSA
  2. 2.Physiology Department, Faculty of MedicineThe Hebrew University of JerusalemJerusalemIsrael
  3. 3.Cognitive Science ProgramThe Hebrew University of JerusalemJerusalemIsrael
  4. 4.Department of Rehabilitation MedicineEmory UniversityAtlantaUSA
  5. 5.Department of PsychologyEmory UniversityAtlantaUSA
  6. 6.Rehabilitation R&D Center of ExcellenceAtlanta VAMCDecaturUSA

Personalised recommendations