Investigating Sensorimotor Contingencies in the Enactive Interface

  • Janet K. Gibbs
  • Kate Devlin
Part of the Studies in Applied Philosophy, Epistemology and Rational Ethics book series (SAPERE, volume 15)


Sensorimotor theories of perception have been widely investigated in the context of the perceiver’s normal environment, but not in the context of virtual environments. There are clearly identified differences between perception of pictures and that of a real-world environment, but these differences have not been studied in the light of sensorimotor theory. Nagel et al.’s studies of sensory augmentation included a trial of their feelSpace belt in a computer-game environment, but with inconclusive results. We propose that the sensorimotor contingencies that apply in the context of a virtual environment are significantly different from those in the ‘real world’, and might account for the differences found between ‘normal’ and picture perception. Building on Froese et al.’s work on Enactive Interfaces, and on Visell’s structure for sensory substitution, we consider how interfacing a sensory augmentation device with a computer game environment might provide the basis for fruitful research in this area.


Virtual Environment Perceptual Experience Motion Sickness Phenomenal Experience Picture Perception 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Froese, T., McGann, M., Bigge, W., Spiers, A., Seth, A.K.: The enactive torch: a new tool for the science of perception. IEEE Transactions on Haptics 5(4), 365–375 (2012)CrossRefGoogle Scholar
  2. 2.
    Schmidmaier, M.: Sensory substitution systems. In: Media Informatics Advanced Seminar on Multimodal Human-Computer Interaction (2011)Google Scholar
  3. 3.
    Visell, Y.: Tactile sensory substitution: models for enaction in HCI. Interacting with Computers 21, 38–53 (2009)CrossRefGoogle Scholar
  4. 4.
    Bach-y Rita, P., Kercel, S.J.: Sensory substitution and the human-machine interface. Trends in Cognitive Sciences 7(12) (December 2003)Google Scholar
  5. 5.
    Nagel, S., Carl, C., Kringe, T., Maertin, R., Koenig, P.: Beyond sensory substitution: flearning the sixth sense. Journal of Neural Engineering 2, R12–R26 (2005)Google Scholar
  6. 6.
    Noë, A.: Action in Perception. Representation and Mind. The MIT Press, Cambridge (2004)Google Scholar
  7. 7.
    O’Regan, J.K.: Why Red Doesn’t Sound Like a Bell: Understanding the Feel of Consciousness. Oxford University Press (2011)Google Scholar
  8. 8.
    O’Regan, J., Noë, A.: What it is like to see: A sensorimotor theory of perceptual experience. Synthese 129(1), 79–103 (2001)CrossRefGoogle Scholar
  9. 9.
    O’Regan, J., Noë, A., et al.: A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24(5), 939–972 (2001)CrossRefGoogle Scholar
  10. 10.
    Sensory Substitution and Augmentation. Unpublished Conference organised by the Centre for the Study of Perceptual Experience and the Network for Sensory Research, London (March 2013)Google Scholar
  11. 11.
    Bach-y Rita, P.: Brain mechanisms in sensory substition. Academic Press, New York (1972)Google Scholar
  12. 12.
    Ward, J., Meijer, P.: Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness & Cognition 19(1), 492–500 (2010)CrossRefGoogle Scholar
  13. 13.
    Guarniero, G.: Tactile vision: a personal view. Journal of Visual Impairment & Blindness 71(3), 125–130 (1977)Google Scholar
  14. 14.
    Hecht, H., Schwartz, R., Atherton, M. (eds.): Looking into pictures: an interdisciplinary approach to pictorial space. The MIT Press, Cambridge (2003)Google Scholar
  15. 15.
    Gibson, J.J.: The ecological approach to visual perception. Lawrence Erlbaum Assoc., Hillsdale (1979)Google Scholar
  16. 16.
    Wollheim, R.: In defense of seeing-in. In: Hecht, H., Schwartz, R., Atherton, M. (eds.) Looking into Pictures: An Interdisciplinary Approach to Pictorial Space. The MIT Press, Cambridge (2003)Google Scholar
  17. 17.
    Mausfeld, R.: Conjoint representations and the mental capacity for multiple simultaneous perspectives. In: Hecht, H., Schwartz, R., Atherton, M. (eds.) Looking into Pictures: An Interdisciplinary Approach to Pictorial Space. The MIT Press, Cambridge (2003)Google Scholar
  18. 18.
    Cooper, E.A., Banks, M.: Perception of depth in pictures when viewed from the wrong distance. Journal of Vision 12(9) (August 2012)Google Scholar
  19. 19.
    Sedgewick, H.A.: The effects of viewpoint on the virtual space of pictures. In: Ellis, S.R. (ed.) Pictorial Communication in Virtual and Real Environments, pp. 460–479. Taylor & Francis, New YorkGoogle Scholar
  20. 20.
    Sedgewick, H.A.: Relating direct and indirect perception of spatial layout. In: Hecht, H., Schwartz, R., Atherton, M. (eds.) Looking into Pictures: An Interdisciplinary Approach to Pictorial Space. The MIT Press, Cambridge (2003)Google Scholar
  21. 21.
    Pirenne, M.: Optics, painting and photography. Cambridge University Press, Cambridge (1970)Google Scholar
  22. 22.
    Keshavarz, B., Hecht, H.: Visually induced motion sickness and presence in video games. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56, p. 1753 (2012)Google Scholar
  23. 23.
    Merhi, O.: Motion sickness, console video games and head-mounted displays. Human Factors: Journal of the Human Factors and Ergonomics Society 49(5), 920–935 (2007)CrossRefGoogle Scholar
  24. 24.
    Levinson, S.C.: Language and cognition: the cognitive consequences of spatial description in guugu yimithirr. Journal of Linguistic Anthropology 7(1), 8–13 (1997)CrossRefGoogle Scholar
  25. 25.
    Kaercher, S.M., Fenzlaff, S., Hartmann, D., Nagel, S.K., Koenig, P.: Sensory augmentation for the blind. In: Frontiers in Human Neuroscience (March 2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Janet K. Gibbs
    • 1
  • Kate Devlin
    • 1
  1. 1.Goldsmiths, University of LondonLondonUK

Personalised recommendations