Biology & Philosophy

, Volume 28, Issue 4, pp 639–655 | Cite as

Neither touch nor vision: sensory substitution as artificial synaesthesia?

  • Mirko FarinaEmail author


Block (Trends Cogn Sci 7:285–286, 2003) and Prinz (PSYCHE 12:1–19, 2006) have defended the idea that SSD perception remains in the substituting modality (auditory or tactile). Hurley and Noë (Biol Philos 18:131–168, 2003) instead argued that after substantial training with the device, the perceptual experience that the SSD user enjoys undergoes a change, switching from tactile/auditory to visual. This debate has unfolded in something like a stalemate where, I will argue, it has become difficult to determine whether the perception acquired through the coupling with an SSD remains in the substituting or the substituted modality. Within this puzzling deadlock two new approaches have been recently suggested. Ward and Meijer (Conscious Cogn 19:492–500, 2010) describe SSD perception as visual-like but characterize it as a kind of artificially induced synaesthesia. Auvray et al. (Perception 36:416–430, 2007) and Auvray and Myin (Cogn Sci 33:1036–1058, 2009) suggest that SSDs let their users experience a new kind of perception. Deroy and Auvray (forthcoming) refine this position, and argue that this new kind of perception depends on pre-existing senses without entirely aligning with any of them. So, they have talked about perceptual experience in SSDs as going "beyond vision". In a similar vein, MacPherson (Oxford University Press, New York, 2011a) claims that “if the subjects (SSD users) have experiences with both vision-like and touch-like representational characteristics then perhaps they have a sense that ordinary humans do not” (MacPherson in Oxford University Press, New York, 2011a, p. 139).


Sensory substitution Individuation of the senses Human sensory modalities Phenomenology Synaesthesia 



I would like to express my appreciation to the ARC Centre of Excellence in Cognition and its Disorders (CCD) and to Macquarie University for generously financing my research. A special and rather large thanks goes to John Sutton for his invaluable support through the preparation of this paper. Thanks also to Malika Auvray, Anina Rich, Andy Clark, Kim Sterelny, Julian Kiverstein, Peter Meijer, and the anonymous reviewer for their stimulating feedback on earlier drafts of this manuscript. Needless to say, any remaining errors are mine alone.


  1. Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neurosci 10:687–689CrossRefGoogle Scholar
  2. Auvray M, Deroy O (forthcoming) Synesthesia. In: Matthen M (ed) Oxford handbook of philosophy of perception. Oxford University Press, OxfordGoogle Scholar
  3. Auvray M, Myin E (2009) Perception with compensatory devices. From sensory substitution to sensorimotor extension. Cogn Sci 33:1036–1058CrossRefGoogle Scholar
  4. Auvray M, Hanneton S, O’Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localization and object recognition with the voice. Perception 36:416–430CrossRefGoogle Scholar
  5. Bach-y-Rita P, Kercel SW (2003) Sensory substitution and the human-machine interface. Trends Cogn Sci 7:541–546CrossRefGoogle Scholar
  6. Block N (2003) Tactile sensation via spatial perception. Trends Cogn Sci 7:285–286CrossRefGoogle Scholar
  7. Capelle C, Trullemans C, Arno P, Veraart C (1998) A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Transact Biomed Eng 45:1279–1293CrossRefGoogle Scholar
  8. Chirimuuta M, Paterson MWD (forthcoming) A methodological molyneux question: sensory substitution, plasticity and the unification of perceptual theory. In: Stokes D, Biggs S, Matthen M (eds) Perception and its modalities. Oxford University Press, OxfordGoogle Scholar
  9. Cohen Kadosh R, Henik A (2007) Can synaesthesia research inform cognitive science? Trends Cogn Sci 11:177–184CrossRefGoogle Scholar
  10. Cohen Kadosh R, Walsh V (2006) Cognitive neuroscience: rewired or crosswired brains? Curr Biol 16:R962–R963CrossRefGoogle Scholar
  11. Cytowic R (2003) The Man Who Tasted Shapes. MIT Press, CambridgeGoogle Scholar
  12. Cytowic R, Eagleman DM (2009) Wednesday is Indigo Blue: Discovering the Brain of Synesthesia. MIT Press, CambridgeGoogle Scholar
  13. Deroy O, Auvray M (forthcoming) Quasi-vision: the sensory substitution dilemma. In: Stokes D, Biggs S, Matthen M (eds) The senses. Oxford University Press, OxfordGoogle Scholar
  14. Dixon MJ, Smilek D, Merikle PM (2004) Not all synaesthetes are created equal: projector versus associator synesthetes. Cogn Affect Behav Neurosci 4:335–343CrossRefGoogle Scholar
  15. Ericsson KA, Krampe RTh, Tesch-Romer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:393–394CrossRefGoogle Scholar
  16. Farina M, Auvray M (forthcoming) Patrolling the boundaries of synaethesia: a critical appraisal of transient and artificiallyinduced forms of synaesthetic experiences. In: Deroy O (ed) Synaesthesia: Philosophical and Psychological Perspectives. Oxford University Press, OxfordGoogle Scholar
  17. Grice HP (1962) Some Remarks About the Senses. In: Butler RJ (ed) Analytic Philosophy. Basil Blackwell, First Series, OxfordGoogle Scholar
  18. Grossenbacher PG, Lovelace CT (2001) Mechanisms of synesthesia: cognitive and physiological constraints. Trends in Cogn Sci 5:36–41CrossRefGoogle Scholar
  19. Guarniero G (1974) Experience of tactile vision. Perception 3:101–104CrossRefGoogle Scholar
  20. Hanneton S, Auvray M, Durette B (2010) The vibe: a versatile vision-to-audition sensory substitution device. Appl Bionics Biomech 7:269–276CrossRefGoogle Scholar
  21. Humphrey N (1992) A History of the Mind: Evolution and the Birth of Consciousness. Springer, New YorkCrossRefGoogle Scholar
  22. Hurley S, Noë A (2003) Neural plasticity and consciousness. Biol Philos 18:131–168CrossRefGoogle Scholar
  23. Kaczmarek KA, Bach-y-Rita P (1995) Tactile displays. In: Barfield W, Furness T (eds) Virtual Environments and Advanced Interface Design. Oxford University Press, New York, pp 393–414Google Scholar
  24. Kamiyama K, Kajimoto H, Inami M, Kawakami N, Tachi S (2001) A vision-based tactile sensor. In: International conference on artificial reality and telexistence, Proceedings (ICAT 2001), pp 127–134Google Scholar
  25. Keeley B (2002) Making sense of the senses: individuating modalities in humans and other animals. J Philos 99:5–28CrossRefGoogle Scholar
  26. Keeley B (2009) The role of neurobiology in differentiating the senses. In: Bickle J (ed) Oxford handbook of philosophy and neuroscience. Oxford University Press, Oxford, pp 226–250Google Scholar
  27. Kiverstein J (2007) Could a robot have a subjective point of view? J Conscious Stud 14:127–139Google Scholar
  28. Kiverstein J, Farina M, Clark A (forthcoming) Substituing the senses. In: Matthen M (ed) The Oxford handbook of philosophy of perception. Oxford University Press, OxfordGoogle Scholar
  29. Kupers R, Pietrini P, Ricciardi E, Ptito M (2011) The nature of consciousness in the visually deprived brain. Frontiers Psychol 2:1–14Google Scholar
  30. Loomis JM (2010) Sensory Substiution for Orientation and Mobility: What Progress are we making? Sidebar 1.1. (pp. 7–10) to Chapter 1, Perceiving to Move and Moving to Perceive: Contorl of Locomotion by Students with Vision Loss by DA Guthm, JJ Rieser, and DH Ashmead pp. 3–44). In: Wiener WR, Welsh RL, Blasch BB (eds) Foundations of Orientation and Mobility, Third Edition, Volume 1 (History and Theory). AFB Press, New YorkGoogle Scholar
  31. Loomis JM, Klatzky RL, Giudice NA (in press) Sensory substitution of vision: importance of perceptual and cognitive processing. In: Manduchi R, Kurniawan S (ed) Assistive technology for blindness and low vision. CRC Press, Boca Raton, pp 161–193Google Scholar
  32. Macpherson F (2011a) Taxonomising the senses. Philos Stud 153:123–142CrossRefGoogle Scholar
  33. Macpherson F (2011b) The senses: classical and contemporary philosophical perspectives. Oxford University Press, New YorkGoogle Scholar
  34. Matthen M (forthcoming) The individuation of the senses. In: Matthen M (ed) The Oxford handbook of the philosophy of perception. Oxford University Press, OxfordGoogle Scholar
  35. Meijer PBL (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39:112–121CrossRefGoogle Scholar
  36. Nudds M (2004) The significance of the senses. Proceed Aristot Soc 104:31–51CrossRefGoogle Scholar
  37. O’Callaghan C (2012) Perception and Multimodality. In: Margolis E, Samuels R, Stich S (eds) The Oxford Handbook of Philosophy of Cognitive Science. Oxford University Press, OxfordGoogle Scholar
  38. O’Regan JK (2011) Why Red Doesn’t Sound Like a Bell: Explaining the Feel of Consciousness. Oxford University Press, OxfordCrossRefGoogle Scholar
  39. O’Regan JK, Noe¨ A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24:939–973CrossRefGoogle Scholar
  40. Ortiz T, Poch J, Santos JM, Requena C, Martínez AM et al (2011) Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness. PLoS One 6(8):e23264CrossRefGoogle Scholar
  41. Prinz J (2006) Putting the brakes on enactive perception. Psychedelic 12:1–19Google Scholar
  42. Ptito M, Moesgaard SM, Gjedde A, Kupers R (2005) Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 128:606–614CrossRefGoogle Scholar
  43. Ramachandran VS, Hubbard EM (2001) Synaesthesia: a window into perception, thought and language. J Conscious Stud 8:3–34Google Scholar
  44. Rich AN, Mattingley JB (2002) Anomalous perception in synaesthesia: a cognitive neuroscience perspective. Nat Rev Neurosci 3:43–52CrossRefGoogle Scholar
  45. Rich AN, Mattingley JB (forthcoming) The role of attention in synaesthesia. In Simner J, Hubbard E (eds) The Oxford handbook of synaesthesia. Oxford University Press, OxfordGoogle Scholar
  46. Rodríguez-hernández AF, Merino C, Casanova O, Modrono C, Torres MÁ (2010) Sensory substitution for visually disabled people: computer solutions. WSEAS Transact Biol Biomed 7:1–10Google Scholar
  47. Simner J (2012) Defining synaesthesia. Br J Psychol 103:1–15CrossRefGoogle Scholar
  48. Stokes S, Biggs S, Matthen M (forthcoming) Perception and its modalities. Oxford University Press, OxfordGoogle Scholar
  49. Ward J, Meijer P (2010) Visual experiences in the blind induced by an auditory sensory substitution device. Conscious Cogn 19:492–500CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Macquarie UniversitySydneyAustralia

Personalised recommendations