Skip to main content
SpringerLink
Log in

Can you hear shapes you touch?

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

Shape is an inherent property of objects existing in both vision and touch but not audition. Can shape then be represented by sound artificially? It has previously been shown that sound can convey visual information by means of image-to-sound coding, but whether sound can code tactile information is not clear. Blindfolded sighted individuals were trained to recognize tactile spatial information using sounds mapped from abstract shapes. After training, subjects were able to match auditory input to tactually discerned shapes and showed generalization to novel auditory–tactile pairings. Furthermore, they showed complete transfer to novel visual shapes, despite the fact that training did not involve any visual exposure. In addition, we found enhanced tactile acuity specific to the training stimuli. The present study demonstrates that as long as tactile space is coded in a systematic way, shape can be conveyed via a medium that is not spatial, suggesting a metamodal representation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Ahissar M, Hochstein S (1993) Attentional control of early perceptual learning. Proc Natl Acad Sci USA 90:5718–5722

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559–571

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci 10:687–689

    Article  PubMed  CAS  Google Scholar 

  • Arno P, Capelle C, Wanet-Defalque MC, Catalan-Ahumada M, Veraart C (1999) Auditory coding of visual patterns for the blind. Perception 28:1013–1029

    Article  PubMed  CAS  Google Scholar 

  • Auvray M, Hanneton S, O’Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the vOICe’. Perception 36:416–430

    Article  PubMed  Google Scholar 

  • Bach-y-Rita P (1972) Brain mechanisms in sensory substitution. Academic Press, London

    Google Scholar 

  • Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221:963–964

    Article  PubMed  CAS  Google Scholar 

  • Ball K, Sekuler R (1987) Direction-specific improvement in motion discrimination. Vision Res 27:953–965

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123

    Article  PubMed  CAS  Google Scholar 

  • Capelle C, Trullemans C, Arno P, Veraart C (1998) A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Trans Biomed Eng 45:1279–1293

    Article  PubMed  CAS  Google Scholar 

  • Carello C, Anderson KL, Kunkler-Peck AJ (1998) Perception of object length by sound. Psychol Sci 9:211–214

    Article  Google Scholar 

  • Crist RE, Kapadia MK, Westheimer G, Gilbert CD (1997) Perceptual learning of spatial localization: specificity for orientation, position, and context. J Neurophysiol 78:2889–2894

    PubMed  CAS  Google Scholar 

  • Crist RE, Li W, Gilbert CD (2001) Learning to see: experience and attention in primary visual cortex. Nat Neurosci 4:519–525

    PubMed  CAS  Google Scholar 

  • Deibert E, Kraut M, Kremen S, Hart J Jr (1999) Neural pathways in tactile object recognition. Neurology 52:1413–1417

    PubMed  CAS  Google Scholar 

  • Fiorentini A, Berardi N (1980) Perceptual learning specific for orientation and spatial frequency. Nature 287:43–44

    Article  PubMed  CAS  Google Scholar 

  • Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston

    Google Scholar 

  • Grassi M (2005) Do we hear size or sound? Balls dropped on plates. Percept Psychophys 67:274–284

    PubMed  Google Scholar 

  • Irvine DR, Martin RL, Klimkeit E, Smith R (2000) Specificity of perceptual learning in a frequency discrimination task. J Acoust Soc Am 108:2964–2968

    Article  PubMed  CAS  Google Scholar 

  • James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714

    Article  PubMed  Google Scholar 

  • Kim JK, Zatorre RJ (2008) Generalized learning of visual-to-auditory substitution in sighted individuals. Brain Res 1242:263–275

    Article  PubMed  CAS  Google Scholar 

  • Kunkler-Peck AJ, Turvey MT (2000) Hearing shape. J Exp Psychol Hum Percept Perform 26:279–294

    Article  PubMed  CAS  Google Scholar 

  • Li W, Piech V, Gilbert CD (2004) Perceptual learning and top-down influences in primary visual cortex. Nat Neurosci 7:651–657

    Article  PubMed  CAS  Google Scholar 

  • Meijer PB (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39:112–121

    Article  PubMed  CAS  Google Scholar 

  • Meinhardt G (2002) Learning to discriminate simple sinusoidal gratings is task specific. Psychol Res 66:143–156

    Article  PubMed  CAS  Google Scholar 

  • Mesulam MM (1998) From sensation to cognition. Brain 121:1013–1052

    Article  PubMed  Google Scholar 

  • Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and haptic perception of natural object shape. Percept Psychophys 66:342–351

    PubMed  Google Scholar 

  • Norman JF, Clayton AM, Norman HF, Crabtree CE (2008) Learning to perceive differences in solid shape through vision and touch. Perception 37:185–196

    Article  PubMed  Google Scholar 

  • Pascual-Leone A, Hamilton R (2001) The metamodal organization of the brain. Prog Brain Res 134:427–445

    Article  PubMed  CAS  Google Scholar 

  • Polley DB, Steinberg EE, Merzenich MM (2006) Perceptual learning directs auditory cortical map reorganization through top-down influences. J Neurosci 26:4970–4982

    Article  PubMed  CAS  Google Scholar 

  • Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn Mem Cogn 25:644–663

    Article  Google Scholar 

  • Recanzone GH, Merzenich MM, Jenkins WM, Grajski KA, Dinse HR (1992) Topographic reorganization of the hand representation in cortical area 3b owl monkeys trained in a frequency-discrimination task. J Neurophysiol 67:1031–1056

    PubMed  CAS  Google Scholar 

  • Recanzone GH, Schreiner CE, Merzenich MM (1993) Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys. J Neurosci 13:87–103

    PubMed  CAS  Google Scholar 

  • Renier L, Collignon O, Poirier C, Tranduy D, Vanlierde A, Bol A, Veraart C, De Volder AG (2005a) Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26:573–580

    Article  PubMed  Google Scholar 

  • Renier L, Laloyaux C, Collignon O, Tranduy D, Vanlierde A, Bruyer R, De Volder AG (2005b) The Ponzo illusion with auditory substitution of vision in sighted and early-blind subjects. Perception 34:857–867

    Article  PubMed  CAS  Google Scholar 

  • Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8:3877–3881

    Article  PubMed  CAS  Google Scholar 

  • Teich AF, Qian N (2003) Learning and adaptation in a recurrent model of V1 orientation selectivity. J Neurophysiol 89:2086–2100

    Article  PubMed  Google Scholar 

  • Van Boven RW, Johnson KO (1994) The limit of tactile spatial resolution in humans: grating orientation discrimination at the lip, tongue, and finger. Neurology 44:2361–2366

    PubMed  Google Scholar 

  • Zangaladze A, Epstein CM, Grafton ST, Sathian K (1999) Involvement of visual cortex in tactile discrimination of orientation. Nature 401:587–590

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

We thank Dr. Peter Meijer for his helpful discussions, Nick Foster and Patrick Bermudez for their technical assistance and Jana Levene for her stimuli preparation. This research was supported by grants from the Canadian Institutes of Health Research and the Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. Department of Neuropsychology, Montreal Neurological Institute, McGill University, 3801 University, Room 276, Montreal, QC, H3A 2B4, Canada

    Jung-Kyong Kim & Robert J. Zatorre

Authors
  1. Jung-Kyong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert J. Zatorre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jung-Kyong Kim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, JK., Zatorre, R.J. Can you hear shapes you touch?. Exp Brain Res 202, 747–754 (2010). https://doi.org/10.1007/s00221-010-2178-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-010-2178-6

Keywords

Search

Navigation