Skip to main content
SpringerLink
Log in

Neither touch nor vision: sensory substitution as artificial synaesthesia?

  • Published:
Biology & Philosophy Aims and scope Submit manuscript

Abstract

Block (Trends Cogn Sci 7:285–286, 2003) and Prinz (PSYCHE 12:1–19, 2006) have defended the idea that SSD perception remains in the substituting modality (auditory or tactile). Hurley and Noë (Biol Philos 18:131–168, 2003) instead argued that after substantial training with the device, the perceptual experience that the SSD user enjoys undergoes a change, switching from tactile/auditory to visual. This debate has unfolded in something like a stalemate where, I will argue, it has become difficult to determine whether the perception acquired through the coupling with an SSD remains in the substituting or the substituted modality. Within this puzzling deadlock two new approaches have been recently suggested. Ward and Meijer (Conscious Cogn 19:492–500, 2010) describe SSD perception as visual-like but characterize it as a kind of artificially induced synaesthesia. Auvray et al. (Perception 36:416–430, 2007) and Auvray and Myin (Cogn Sci 33:1036–1058, 2009) suggest that SSDs let their users experience a new kind of perception. Deroy and Auvray (forthcoming) refine this position, and argue that this new kind of perception depends on pre-existing senses without entirely aligning with any of them. So, they have talked about perceptual experience in SSDs as going "beyond vision". In a similar vein, MacPherson (Oxford University Press, New York, 2011a) claims that “if the subjects (SSD users) have experiences with both vision-like and touch-like representational characteristics then perhaps they have a sense that ordinary humans do not” (MacPherson in Oxford University Press, New York, 2011a, p. 139).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Thanks also to John Sutton for clarifying these issues.

  2. So, we probably need a substantial amount of reports in order to make a strong statistical argument based on them.

  3. The question of how to individuate the senses has a long history. Grice (1962) discussed four criteria (the proper objection criterion, the character of experience criterion, the physical features criterion, and the sensory organ criterion) that one might use to distinguish the senses and argued that there is an ineliminable role for the phenomenal character of experiences in individuating them. More recently, Nudds (2004) has argued that sense is an intuitive concept that we employ for certain everyday purposes. In particular, he suggests that the concept of sense relies on societal agreement and ‘convention’. Nudds thus embraces a folk-psychological understanding of the senses. Keeley (2002) has defended a neurobiological criterion for counting the senses, according to which token perceptions count as instances of different sense modalities because of the “character of the putative sense organs and their modes of connection with the brain” (2002: 13). Instead of differentiating the senses just on the basis of whether experience begins in the eyes, ears, nose, tongue or skin, Keeley argues we must also take into account whether the sense organ is, as he puts it, ‘appropriately wired up’ (Keeley 2009). So Keeley defends a scientific account of the senses. The question of how to individuate the senses has taken on fresh interest (Macpherson Macpherson 2011b) and a series of philosophers have recently proposed other views to specify what counts as a sense and how to individuate sensory modalities. See in particular O’Callaghan (2012); Matthen (forthcoming) and Stokes et al. (forthcoming).

  4. In Kiverstein et al. (forthcoming), we address the debate on human sensory modalities and its link with sensory substitution, and offer a critical analysis of Keeley’s notion of dedication with respect to SSDs.

  5. TDU or Tongue Display Unit is a sensory substitution device that belongs to the category of visual-to-tactile substitution features. The FSRS discussed above and the Brain Port are probably the most successful example of a TDU.

  6. Ericsson et al. (1993) claims that it takes 10,000 h [20 h for 50 weeks a year for 10 years] of deliberate practice to become an expert in any particular activity. I do not wish to evaluate this controversial assumption, but rather use it to emphasise the role of experience in the development of skills and expertise.

  7. For an in-depth analysis of the limitations of perceptual and cognitive processing associated with current sensory substitution devices see (Loomis 2010; Loomis et al. in press).

  8. 15 h of training are most probably not enough to turn a naïve subject into an expert user (see note 5 above). But 15 h of training are better than just 6–8 h (the standard amount of practice to which many TDU users were normally exposed in the past). So, even if the labels “expert users” or “over-trained participants” are not, by Ericsson’s standards, appropriate to define the subjects involved in this experiment; their usage appears legitimate when we compared these subjects to standard TDU users, who were tested after a much reduced period of training.

  9. Not all forms of synaesthesia are laden with affect. So, this definitional aspect doesn’t seem to be strictly necessary.

  10. Perhaps the most discussed of all these definitional aspects is the consistency criterion. Consistency or persistency over time is the idea that the same inducer always triggers the same concurrent type of synaesthetic experience. See Auvray and Deroy (forthcoming). For some criticism of the idea that consistency can be used as a gold-standard criterion for establishing the authenticity of an individual’s synesthesia see Simner (2012) instead.

  11. Is the synesthetic occurrence experienced internally or rather perceived as having a spatial connotation? This is a much debated issue in the literature. Saying that synaesthesia is spatially extended might indeed depend on the projector/associator divide. It is important to note however that some authors, such as Simner (2012) and Rich and Mattingley (forthcoming) reject this divide and instead claim that there is a continuum between projectors and associators.

  12. PF became blind at the age of 21 after an accident. She first used The vOICe in 1998. CC was born with rod dystrophy. She had very bad eyesight as a child and as a teenager but was registered blind only at the age of 33. She first used The vOICe in 2001.

References

  • Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neurosci 10:687–689

    Article  Google Scholar 

  • Auvray M, Deroy O (forthcoming) Synesthesia. In: Matthen M (ed) Oxford handbook of philosophy of perception. Oxford University Press, Oxford

  • Auvray M, Myin E (2009) Perception with compensatory devices. From sensory substitution to sensorimotor extension. Cogn Sci 33:1036–1058

    Article  Google Scholar 

  • Auvray M, Hanneton S, O’Regan JK (2007) Learning to perceive with a visuo-auditory substitution system: localization and object recognition with the voice. Perception 36:416–430

    Article  Google Scholar 

  • Bach-y-Rita P, Kercel SW (2003) Sensory substitution and the human-machine interface. Trends Cogn Sci 7:541–546

    Article  Google Scholar 

  • Block N (2003) Tactile sensation via spatial perception. Trends Cogn Sci 7:285–286

    Article  Google Scholar 

  • Capelle C, Trullemans C, Arno P, Veraart C (1998) A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Transact Biomed Eng 45:1279–1293

    Article  Google Scholar 

  • Chirimuuta M, Paterson MWD (forthcoming) A methodological molyneux question: sensory substitution, plasticity and the unification of perceptual theory. In: Stokes D, Biggs S, Matthen M (eds) Perception and its modalities. Oxford University Press, Oxford

  • Cohen Kadosh R, Henik A (2007) Can synaesthesia research inform cognitive science? Trends Cogn Sci 11:177–184

    Article  Google Scholar 

  • Cohen Kadosh R, Walsh V (2006) Cognitive neuroscience: rewired or crosswired brains? Curr Biol 16:R962–R963

    Article  Google Scholar 

  • Cytowic R (2003) The Man Who Tasted Shapes. MIT Press, Cambridge

    Google Scholar 

  • Cytowic R, Eagleman DM (2009) Wednesday is Indigo Blue: Discovering the Brain of Synesthesia. MIT Press, Cambridge

    Google Scholar 

  • Deroy O, Auvray M (forthcoming) Quasi-vision: the sensory substitution dilemma. In: Stokes D, Biggs S, Matthen M (eds) The senses. Oxford University Press, Oxford

  • Dixon MJ, Smilek D, Merikle PM (2004) Not all synaesthetes are created equal: projector versus associator synesthetes. Cogn Affect Behav Neurosci 4:335–343

    Article  Google Scholar 

  • Ericsson KA, Krampe RTh, Tesch-Romer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:393–394

    Article  Google Scholar 

  • Farina M, Auvray M (forthcoming) Patrolling the boundaries of synaethesia: a critical appraisal of transient and artificiallyinduced forms of synaesthetic experiences. In: Deroy O (ed) Synaesthesia: Philosophical and Psychological Perspectives. Oxford University Press, Oxford

  • Grice HP (1962) Some Remarks About the Senses. In: Butler RJ (ed) Analytic Philosophy. Basil Blackwell, First Series, Oxford

    Google Scholar 

  • Grossenbacher PG, Lovelace CT (2001) Mechanisms of synesthesia: cognitive and physiological constraints. Trends in Cogn Sci 5:36–41

    Article  Google Scholar 

  • Guarniero G (1974) Experience of tactile vision. Perception 3:101–104

    Article  Google Scholar 

  • Hanneton S, Auvray M, Durette B (2010) The vibe: a versatile vision-to-audition sensory substitution device. Appl Bionics Biomech 7:269–276

    Article  Google Scholar 

  • Humphrey N (1992) A History of the Mind: Evolution and the Birth of Consciousness. Springer, New York

    Book  Google Scholar 

  • Hurley S, Noë A (2003) Neural plasticity and consciousness. Biol Philos 18:131–168

    Article  Google Scholar 

  • Kaczmarek KA, Bach-y-Rita P (1995) Tactile displays. In: Barfield W, Furness T (eds) Virtual Environments and Advanced Interface Design. Oxford University Press, New York, pp 393–414

    Google Scholar 

  • Kamiyama K, Kajimoto H, Inami M, Kawakami N, Tachi S (2001) A vision-based tactile sensor. In: International conference on artificial reality and telexistence, Proceedings (ICAT 2001), pp 127–134

  • Keeley B (2002) Making sense of the senses: individuating modalities in humans and other animals. J Philos 99:5–28

    Article  Google Scholar 

  • Keeley B (2009) The role of neurobiology in differentiating the senses. In: Bickle J (ed) Oxford handbook of philosophy and neuroscience. Oxford University Press, Oxford, pp 226–250

    Google Scholar 

  • Kiverstein J (2007) Could a robot have a subjective point of view? J Conscious Stud 14:127–139

    Google Scholar 

  • Kiverstein J, Farina M, Clark A (forthcoming) Substituing the senses. In: Matthen M (ed) The Oxford handbook of philosophy of perception. Oxford University Press, Oxford

  • Kupers R, Pietrini P, Ricciardi E, Ptito M (2011) The nature of consciousness in the visually deprived brain. Frontiers Psychol 2:1–14

    Google Scholar 

  • Loomis JM (2010) Sensory Substiution for Orientation and Mobility: What Progress are we making? Sidebar 1.1. (pp. 7–10) to Chapter 1, Perceiving to Move and Moving to Perceive: Contorl of Locomotion by Students with Vision Loss by DA Guthm, JJ Rieser, and DH Ashmead pp. 3–44). In: Wiener WR, Welsh RL, Blasch BB (eds) Foundations of Orientation and Mobility, Third Edition, Volume 1 (History and Theory). AFB Press, New York

    Google Scholar 

  • Loomis JM, Klatzky RL, Giudice NA (in press) Sensory substitution of vision: importance of perceptual and cognitive processing. In: Manduchi R, Kurniawan S (ed) Assistive technology for blindness and low vision. CRC Press, Boca Raton, pp 161–193

  • Macpherson F (2011a) Taxonomising the senses. Philos Stud 153:123–142

    Article  Google Scholar 

  • Macpherson F (2011b) The senses: classical and contemporary philosophical perspectives. Oxford University Press, New York

    Google Scholar 

  • Matthen M (forthcoming) The individuation of the senses. In: Matthen M (ed) The Oxford handbook of the philosophy of perception. Oxford University Press, Oxford

  • Meijer PBL (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39:112–121

    Article  Google Scholar 

  • Nudds M (2004) The significance of the senses. Proceed Aristot Soc 104:31–51

    Article  Google Scholar 

  • O’Callaghan C (2012) Perception and Multimodality. In: Margolis E, Samuels R, Stich S (eds) The Oxford Handbook of Philosophy of Cognitive Science. Oxford University Press, Oxford

    Google Scholar 

  • O’Regan JK (2011) Why Red Doesn’t Sound Like a Bell: Explaining the Feel of Consciousness. Oxford University Press, Oxford

    Book  Google Scholar 

  • O’Regan JK, Noe¨ A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24:939–973

    Article  Google Scholar 

  • Ortiz T, Poch J, Santos JM, Requena C, Martínez AM et al (2011) Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness. PLoS One 6(8):e23264

    Article  Google Scholar 

  • Prinz J (2006) Putting the brakes on enactive perception. Psychedelic 12:1–19

    Google Scholar 

  • Ptito M, Moesgaard SM, Gjedde A, Kupers R (2005) Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 128:606–614

    Article  Google Scholar 

  • Ramachandran VS, Hubbard EM (2001) Synaesthesia: a window into perception, thought and language. J Conscious Stud 8:3–34

    Google Scholar 

  • Rich AN, Mattingley JB (2002) Anomalous perception in synaesthesia: a cognitive neuroscience perspective. Nat Rev Neurosci 3:43–52

    Article  Google Scholar 

  • Rich AN, Mattingley JB (forthcoming) The role of attention in synaesthesia. In Simner J, Hubbard E (eds) The Oxford handbook of synaesthesia. Oxford University Press, Oxford

  • Rodríguez-hernández AF, Merino C, Casanova O, Modrono C, Torres MÁ (2010) Sensory substitution for visually disabled people: computer solutions. WSEAS Transact Biol Biomed 7:1–10

    Google Scholar 

  • Simner J (2012) Defining synaesthesia. Br J Psychol 103:1–15

    Article  Google Scholar 

  • Stokes S, Biggs S, Matthen M (forthcoming) Perception and its modalities. Oxford University Press, Oxford

  • Ward J, Meijer P (2010) Visual experiences in the blind induced by an auditory sensory substitution device. Conscious Cogn 19:492–500

    Article  Google Scholar 

Download references

Acknowledgments

I would like to express my appreciation to the ARC Centre of Excellence in Cognition and its Disorders (CCD) and to Macquarie University for generously financing my research. A special and rather large thanks goes to John Sutton for his invaluable support through the preparation of this paper. Thanks also to Malika Auvray, Anina Rich, Andy Clark, Kim Sterelny, Julian Kiverstein, Peter Meijer, and the anonymous reviewer for their stimulating feedback on earlier drafts of this manuscript. Needless to say, any remaining errors are mine alone.

Author information

Authors and Affiliations

  1. Macquarie University, Sydney, NSW, Australia

    Mirko Farina

Authors
  1. Mirko Farina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mirko Farina.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Farina, M. Neither touch nor vision: sensory substitution as artificial synaesthesia?. Biol Philos 28, 639–655 (2013). https://doi.org/10.1007/s10539-013-9377-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10539-013-9377-z

Keywords

Search

Navigation