Experimental Brain Research

, Volume 202, Issue 4, pp 747–754 | Cite as

Can you hear shapes you touch?

  • Jung-Kyong KimEmail author
  • Robert J. Zatorre
Research Article


Shape is an inherent property of objects existing in both vision and touch but not audition. Can shape then be represented by sound artificially? It has previously been shown that sound can convey visual information by means of image-to-sound coding, but whether sound can code tactile information is not clear. Blindfolded sighted individuals were trained to recognize tactile spatial information using sounds mapped from abstract shapes. After training, subjects were able to match auditory input to tactually discerned shapes and showed generalization to novel auditory–tactile pairings. Furthermore, they showed complete transfer to novel visual shapes, despite the fact that training did not involve any visual exposure. In addition, we found enhanced tactile acuity specific to the training stimuli. The present study demonstrates that as long as tactile space is coded in a systematic way, shape can be conveyed via a medium that is not spatial, suggesting a metamodal representation.


Shape Sensory substitution Learning Auditory perception Tactile 



We thank Dr. Peter Meijer for his helpful discussions, Nick Foster and Patrick Bermudez for their technical assistance and Jana Levene for her stimuli preparation. This research was supported by grants from the Canadian Institutes of Health Research and the Natural Sciences and Engineering Research Council of Canada.


  1. Ahissar M, Hochstein S (1993) Attentional control of early perceptual learning. Proc Natl Acad Sci USA 90:5718–5722CrossRefPubMedGoogle Scholar
  2. Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330CrossRefPubMedGoogle Scholar
  3. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559–571CrossRefPubMedGoogle Scholar
  4. Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci 10:687–689CrossRefPubMedGoogle Scholar
  5. Arno P, Capelle C, Wanet-Defalque MC, Catalan-Ahumada M, Veraart C (1999) Auditory coding of visual patterns for the blind. Perception 28:1013–1029CrossRefPubMedGoogle Scholar
  6. Auvray M, Hanneton S, O’Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the vOICe’. Perception 36:416–430CrossRefPubMedGoogle Scholar
  7. Bach-y-Rita P (1972) Brain mechanisms in sensory substitution. Academic Press, LondonGoogle Scholar
  8. Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221:963–964CrossRefPubMedGoogle Scholar
  9. Ball K, Sekuler R (1987) Direction-specific improvement in motion discrimination. Vision Res 27:953–965CrossRefPubMedGoogle Scholar
  10. Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123CrossRefPubMedGoogle Scholar
  11. Capelle C, Trullemans C, Arno P, Veraart C (1998) A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Trans Biomed Eng 45:1279–1293CrossRefPubMedGoogle Scholar
  12. Carello C, Anderson KL, Kunkler-Peck AJ (1998) Perception of object length by sound. Psychol Sci 9:211–214CrossRefGoogle Scholar
  13. Crist RE, Kapadia MK, Westheimer G, Gilbert CD (1997) Perceptual learning of spatial localization: specificity for orientation, position, and context. J Neurophysiol 78:2889–2894PubMedGoogle Scholar
  14. Crist RE, Li W, Gilbert CD (2001) Learning to see: experience and attention in primary visual cortex. Nat Neurosci 4:519–525PubMedGoogle Scholar
  15. Deibert E, Kraut M, Kremen S, Hart J Jr (1999) Neural pathways in tactile object recognition. Neurology 52:1413–1417PubMedGoogle Scholar
  16. Fiorentini A, Berardi N (1980) Perceptual learning specific for orientation and spatial frequency. Nature 287:43–44CrossRefPubMedGoogle Scholar
  17. Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, BostonGoogle Scholar
  18. Grassi M (2005) Do we hear size or sound? Balls dropped on plates. Percept Psychophys 67:274–284PubMedGoogle Scholar
  19. Irvine DR, Martin RL, Klimkeit E, Smith R (2000) Specificity of perceptual learning in a frequency discrimination task. J Acoust Soc Am 108:2964–2968CrossRefPubMedGoogle Scholar
  20. James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714CrossRefPubMedGoogle Scholar
  21. Kim JK, Zatorre RJ (2008) Generalized learning of visual-to-auditory substitution in sighted individuals. Brain Res 1242:263–275CrossRefPubMedGoogle Scholar
  22. Kunkler-Peck AJ, Turvey MT (2000) Hearing shape. J Exp Psychol Hum Percept Perform 26:279–294CrossRefPubMedGoogle Scholar
  23. Li W, Piech V, Gilbert CD (2004) Perceptual learning and top-down influences in primary visual cortex. Nat Neurosci 7:651–657CrossRefPubMedGoogle Scholar
  24. Meijer PB (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39:112–121CrossRefPubMedGoogle Scholar
  25. Meinhardt G (2002) Learning to discriminate simple sinusoidal gratings is task specific. Psychol Res 66:143–156CrossRefPubMedGoogle Scholar
  26. Mesulam MM (1998) From sensation to cognition. Brain 121:1013–1052CrossRefPubMedGoogle Scholar
  27. Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and haptic perception of natural object shape. Percept Psychophys 66:342–351PubMedGoogle Scholar
  28. Norman JF, Clayton AM, Norman HF, Crabtree CE (2008) Learning to perceive differences in solid shape through vision and touch. Perception 37:185–196CrossRefPubMedGoogle Scholar
  29. Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. Prog Brain Res 134:427–445CrossRefPubMedGoogle Scholar
  30. Polley DB, Steinberg EE, Merzenich MM (2006) Perceptual learning directs auditory cortical map reorganization through top-down influences. J Neurosci 26:4970–4982CrossRefPubMedGoogle Scholar
  31. Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn Mem Cogn 25:644–663CrossRefGoogle Scholar
  32. Recanzone GH, Merzenich MM, Jenkins WM, Grajski KA, Dinse HR (1992) Topographic reorganization of the hand representation in cortical area 3b owl monkeys trained in a frequency-discrimination task. J Neurophysiol 67:1031–1056PubMedGoogle Scholar
  33. Recanzone GH, Schreiner CE, Merzenich MM (1993) Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys. J Neurosci 13:87–103PubMedGoogle Scholar
  34. Renier L, Collignon O, Poirier C, Tranduy D, Vanlierde A, Bol A, Veraart C, De Volder AG (2005a) Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26:573–580CrossRefPubMedGoogle Scholar
  35. Renier L, Laloyaux C, Collignon O, Tranduy D, Vanlierde A, Bruyer R, De Volder AG (2005b) The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects. Perception 34:857–867CrossRefPubMedGoogle Scholar
  36. Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8:3877–3881CrossRefPubMedGoogle Scholar
  37. Teich AF, Qian N (2003) Learning and adaptation in a recurrent model of V1 orientation selectivity. J Neurophysiol 89:2086–2100CrossRefPubMedGoogle Scholar
  38. Van Boven RW, Johnson KO (1994) The limit of tactile spatial resolution in humans: grating orientation discrimination at the lip, tongue, and finger. Neurology 44:2361–2366PubMedGoogle Scholar
  39. Zangaladze A, Epstein CM, Grafton ST, Sathian K (1999) Involvement of visual cortex in tactile discrimination of orientation. Nature 401:587–590CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Department of Neuropsychology, Montreal Neurological InstituteMcGill UniversityMontrealCanada

Personalised recommendations