Skip to main content
SpringerLink
Log in
Book cover

Multisensory Object Perception in the Primate Brain pp 251–271Cite as

Visuo-haptic Perception of Objects and Scenes

  • Chapter
  • First Online:

Abstract

Although many high-level perceptual tasks can be achieved on the basis of information encoded through one sensory modality, it is increasingly evident that the maintenance of a robust, coherent perception of the objects that surround us depends on multisensory integration. Consequently, multisensory representations of object information in memory, particularly those based on vision and touch, result in more efficient object recognition and spatial localisation. The following chapter reviews evidence on how multisensory object information can, for example, resolve problems often associated with unisensory processing such as maintaining shape constancy with changes in object viewpoint or motion, and updating spatial representations with changes in observer position. Further evidence from neuroimaging studies suggests that the perceptual processes involved in object and spatial recognition are underpinned by shared neural resources. Taken together, these studies suggest that the traditional view of sensory systems processing object information in an independent manner is breaking down such that, conversely, the wealth of evidence now lies firmly in favour of sensory systems which are highly interactive all along the information processing hierarchy, and which can modulate and affect high-level perceptual outcomes.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Non-informative visual information is information that would not, on its own, be sufficient to solve the task. For example, seeing the surrounding room but without seeing the test stimuli would be considered ‘non-informative’ visual information.

References

  • Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic, object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330

    Article  PubMed  CAS  Google Scholar 

  • Andresen DR, Vinberg J, Grill-Spector K (2009) The representation of object viewpoint in human visual cortex. Neuroimage 45(2):522–536

    Article  PubMed  Google Scholar 

  • Berkeley G (1709) An essay towards a new theory of vision. Jeremy Pepyat Booksellers, Dublin, Ireland

    Google Scholar 

  • Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 94:115–147

    Article  PubMed  CAS  Google Scholar 

  • Biederman I, Gerhardstein PC (1993) Recognizing depth-view, rotated objects: evidence and conditions for three-dimensional viewpoint invariance. J Exp Psychol Hum Percept Perform 19:1162–1182

    Article  PubMed  CAS  Google Scholar 

  • Biederman I, Ju G (1988) Surface versus edge-based determinants of visual recognition. Cogn Psychol 20(1):38–64

    Article  PubMed  CAS  Google Scholar 

  • Biederman I, Rabinowitz JC, Glass AL, Stacy EW Jr (1974) On the information extracted from a glance at a scene. J Exp Psychol 103(3):597–600

    Article  PubMed  CAS  Google Scholar 

  • Bülthoff HH, Edelman S (1992) Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc Natl Acad Sci USA 89:60–64

    Article  PubMed  Google Scholar 

  • Chan JS, Newell FN (2008) Behavioral evidence for task-dependent “what” versus “where” processing within and across modalities. Percept Psychophys 70(1):36–49

    Article  PubMed  Google Scholar 

  • Chan JS, Whitaker TA, Simões-Franklin C, Garavan H, Newell FN (2010) Implied haptic object motion activates visual area MT/MST. Neuroimage, 49(2):1708–1716

    Google Scholar 

  • Diwadkar VA, McNamara TP (1997) Viewpoint dependence in scene recognition. Psycholog Sci 8:302–307

    Article  Google Scholar 

  • Easton RD, Srinivas K, Greene AJ (1997) Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. J Exp Psychol Learn Mem Cogn 23(1):153–163

    Article  PubMed  CAS  Google Scholar 

  • Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433

    Article  PubMed  CAS  Google Scholar 

  • Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169

    Article  PubMed  Google Scholar 

  • Ernst MO, Lange C, Newell FN (2007) Multisensory recognition of actively explored objects. Can J Exp Psychol 61(3):242–253

    Article  PubMed  Google Scholar 

  • Gibson JJ (1962) Observations on active touch. Psycholog Rev 69:477–491

    Article  CAS  Google Scholar 

  • Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15:20–25

    Article  PubMed  CAS  Google Scholar 

  • Gori M, Del Viva M, Sandini G, Burr DC (2008) Young children do not integrate visual and haptic form information. Curr Biol 18(9):694–698

    Article  PubMed  CAS  Google Scholar 

  • Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R (1999) Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 24:187–203

    Article  PubMed  CAS  Google Scholar 

  • Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P (2001) Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293:2425–2430

    Article  PubMed  CAS  Google Scholar 

  • Hayward WG, Williams P (2000) Viewpoint dependence and object discriminability. Psychol Sci 11:7–12

    Article  PubMed  CAS  Google Scholar 

  • James TW Humphrey GK Gati JS Menon RS, Goodale MA (2002) Differential effects of viewpoint on object-driven activation in dorsal and ventral streams. Neuron 35:793–801

    Article  PubMed  CAS  Google Scholar 

  • Kaas AL, van Mier HI, Lataster J, Fingal M, Sack AT (2007) The effect of visuo-haptic congruency on haptic spatial matching. Exp Brain Res 183(1):75–85

    Article  PubMed  Google Scholar 

  • Klatzky RL, Lederman SJ (1995) Identifying objects from a haptic glance. Percept Psychophys 57(8):1111–1123

    Article  PubMed  CAS  Google Scholar 

  • Klatzky RL, Lederman SJ, Metzger V (1985) Identifying objects by touch: an expert system. Percept Psychophys 37:299–302

    Article  PubMed  CAS  Google Scholar 

  • Kourtzi Z, Kanwisher N (2000) Activation in human MT/MST by static images with implied motion. J Cogn Neurosci 12(1):48–55

    Article  PubMed  CAS  Google Scholar 

  • Lacey S, Peters A, Sathian K (2007) Cross-modal object recognition is viewpoint-independent. PLoS ONE 2(9):e890

    Article  PubMed  Google Scholar 

  • Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21(3–4):269–274

    Google Scholar 

  • Lewis TL, Maurer D (2005) Multiple sensitive periods in human visual development: evidence from visually deprived children. Dev Psychobiol 46(3):163–183

    Article  PubMed  Google Scholar 

  • Logothetis NK, Pauls J (1995) Psychophysical and physiological evidence for viewer-centered object representations in the primate. Cereb Cortex 5(3):270–288

    Article  PubMed  CAS  Google Scholar 

  • Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RBH (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA 92:8135–8139

    Article  PubMed  CAS  Google Scholar 

  • Marr D (1982) Vision. Freeman Publishers, San Francisco

    Google Scholar 

  • Meltzoff AN, Borton RW (1979) Intermodal matching by human neonates. Nature 282:403–404

    Article  PubMed  CAS  Google Scholar 

  • Mishkin M, Ungerleider LG, Macko KA (1983) Object vision and spatial vision: two cortical pathways. Trends Neurosci 6:414–417

    Article  Google Scholar 

  • Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 4:452–465

    Article  Google Scholar 

  • Newell FN (1998) Stimulus context and view dependence in object recognition. Perception 27(1):47–68

    Article  PubMed  CAS  Google Scholar 

  • Newell FN, Ernst MO, Tjan BS, Bülthoff HH (2001) Viewpoint dependence in visual and haptic object recognition. Psychol Sci 12(1):37–42

    Article  PubMed  CAS  Google Scholar 

  • Newell FN, Findlay JM (1997) The effect of depth rotation on object identification. Perception 26(10):1231–1257

    Article  PubMed  CAS  Google Scholar 

  • Newell FN, Finucane CM, Lisiecka D, Pasqualotto A, Vendrell I (2010a) Active perception allows for spatial updating of object locations across modalities. Submitted

    Google Scholar 

  • Newell, Hansen, Steven, Calvert (2010b) Tactile discrimination and localisation of novel object features activates both ventral and dorsal streams. Manuscript in preparation

    Google Scholar 

  • Newell FN, Sheppard DM, Edelman S, Shapiro KL (2005) The interaction of shape- and location-based priming in object categorisation: evidence for a hybrid "what + where" representation stage. Vision Res 45(16):2065–2080

    Article  PubMed  Google Scholar 

  • Newell FN, Wallraven C, Huber S (2004) The role of characteristic motion in object categorization. J Vis 4(2):118–129

    Article  PubMed  Google Scholar 

  • Newell FN, Woods AT, Mernagh M, Bülthoff HH (2005) Visual, haptic and crossmodal recognition of scenes. Exp Brain Res 161(2):233–242

    Article  PubMed  Google Scholar 

  • Newport R, Rabb B, Jackson SR (2002) Noninformative vision improves haptic spatial perception. Curr Biol 12(19):1661–1664

    Article  PubMed  CAS  Google Scholar 

  • Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and haptic perception of natural object shape. Percept Psychophys 66(2):342–351

    Article  PubMed  Google Scholar 

  • Palmer S, Rosch E, Chase P (1981) Canonical perspective and the perception of objects. In: Long JB, Baddeley AD (eds) Attention and performance, vol. IX. Erlbaum, Hillsdale, NJ, pp 135–151

    Google Scholar 

  • Pasqualotto A, Finucane CM, Newell FN (2005) Visual and haptic representations of scenes are updated with observer movement. Exp Brain Res 166(3–4):481–488

    Article  PubMed  Google Scholar 

  • Pasqualotto A, Newell FN (2007) The role of visual experience on the representation and updating of novel haptic scenes. Brain Cogn 65(2):184–194

    Article  PubMed  Google Scholar 

  • Postma A, Zuidhoek S, Noordzij ML, Kappers AM (2008) Keep an eye on your hands: on the role of visual mechanisms in processing of haptic space. Cogn Proc 9(1):63–68

    Article  Google Scholar 

  • Postma A, Zuidhoek S, Noordzij ML, Kappers AM (2008) Haptic orientation perception benefits from visual experience: evidence from early-blind, late-blind, and sighted people. Percept Psychophys 70(7):1197–1206

    Article  PubMed  Google Scholar 

  • Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435:1102–1107

    Article  PubMed  CAS  Google Scholar 

  • Romanski LM, Tian B, Fritz J, Mishkin M, Goldman-Rakic PS, Rauschecker JP (1999) Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat Neurosci 2:1131–1136

    Article  PubMed  CAS  Google Scholar 

  • Reales JM, Ballestersos S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn Mem Cogn 25_(3)644–663

    Article  Google Scholar 

  • Reddy L, Kanwisher N (2006) Coding of visual objects in the ventral stream. Curr Opin Neurobiol 16:408–414

    Article  PubMed  CAS  Google Scholar 

  • Reed CL, Klatzky RL, Halgren E (2005) What vs. where in touch: an fMRI study. Neuroimage 25:718–726

    Article  PubMed  Google Scholar 

  • Rousselet GA, Fabre-Thorpe M, Thorpe SJ (2002) Parallel processing in high-level categorization of natural images. Nat Neurosci 5(7):629–630

    PubMed  CAS  Google Scholar 

  • Sann C, Streri A (2007) Perception of object shape and texture in human newborns: evidence from cross-modal transfer tasks. Dev Sci 10(3):399–410

    Article  PubMed  Google Scholar 

  • Schendan HE, Stern CE (2007) Mental rotation and object categorization share a common network of prefrontal and dorsal and ventral regions of posterior cortex. Neuroimage 35(3):1264–1277

    Article  PubMed  Google Scholar 

  • Schendan HE, Stern CE (2008) Where vision meets memory: prefrontal-posterior networks for visual object constancy during categorization and recognition. Cereb Cortex 18(7):1695–1711

    Article  PubMed  Google Scholar 

  • Setti A, Newell FN (2009) The effect of body and part-based motion on the recognition of unfamiliar objects. Vis Cogn 18(3):456–480

    Google Scholar 

  • Simons DJ, Wang RF, Roddenberry D (2002) Object recognition is mediated by extraretinal information. Percept Psychophys 64(4):521–530

    Article  PubMed  Google Scholar 

  • Stone JV (1999) Object recognition: view-specificity and motion-specificity. Vis Res 39(24):4032–4044

    Article  PubMed  CAS  Google Scholar 

  • Stone JV (1998) Object recognition using spatiotemporal signatures. Vis Res 38(7):947–951

    Article  PubMed  CAS  Google Scholar 

  • Tarr MJ (1995) Rotating objects to recognize them: a case study of the role of viewpoint dependency in the recognition of three-dimensional objects. Psychon Bull Rev 2:55–82

    Article  Google Scholar 

  • Tarr MJ, Bülthoff HH (1998) Image-based object recognition in man, monkey and machine. Cognition 67:1–20

    Article  PubMed  CAS  Google Scholar 

  • Thinus-Blanc C, Gaunet F (1997) Representation of space in blind persons: vision as a spatial sense? Psychol Bull 121(1):20–42

    Article  PubMed  CAS  Google Scholar 

  • Tootell RB, Reppas JB, Kwong KK, Malach R, Born RT, Brady TJ, Rosen BR, Belliveau JW (1995) Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J Neurosci 15(4):3215–3230

    PubMed  CAS  Google Scholar 

  • Ungerleider LG, Haxby JV (1994) ‘What’ and ‘where’ in the human brain. Curr Opin Neurobiol 4, 157–165

    Article  PubMed  CAS  Google Scholar 

  • Ullman S (1998) Three-dimensional object recognition based on the combination of views. Cogn 67:21–44

    Article  CAS  Google Scholar 

  • Van Boven RW, Ingeholm JE, Beauchamp MS, Bikle PC, Ungerleider LG (2005) Tactile form and location processing in the human brain. Proc Natl Acad Sci U S A 102(35):12601–12605

    Article  PubMed  Google Scholar 

  • Volcic R, van Rheede JJ, Postma A, Kappers AM (2008) Differential effects of non-informative vision and visual interference on haptic spatial processing. Exp Brain Res 190(1):31–41

    Article  PubMed  Google Scholar 

  • Vuong QC, Tarr MJ (2004) Rotation direction affects object recognition. Vis Res 44(14):1717–1730

    Article  PubMed  Google Scholar 

  • Welch RB, Warren DH (1986) Intersensory interactions. In: Boff KR, Kaufman L, Thomas JP (eds) Handbook of perception and performance, vol. 1. Sensory processes and perception. John Wiley and Sons, New York, pp 25–1–25–36

    Google Scholar 

  • Whitaker TA, Chan J, Newell FN (2010) The role of characteristic motion in haptic and visuo-haptic object recognition. Manuscript in preparation

    Google Scholar 

  • Wang RF, Simons DJ (1999) Active and passive scene recognition across views. Cognition 70(2):191–210

    Article  PubMed  CAS  Google Scholar 

  • Woods AT, Moore A, Newell FN (2009) Canonical views in haptic object perception. Perception 37(12):1867–1878

    Article  Google Scholar 

  • Young MP (1992) Objective analysis of the topological organization of the primate cortical visual system. Nature 358(6382):152–155

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Psychology and Institute of Neuroscience, Lloyd Building, Trinity College, Dublin 2, Ireland

    Fiona N. Newell

Authors
  1. Fiona N. Newell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fiona N. Newell .

Editor information

Editors and Affiliations

  1. Inst. Medizinische Psychologie (IMP), Universität Frankfurt, Heinrich-Hoffmann-Str. 10, Frankfurt, 60528, Germany

    Jochen Kaiser

  2. Abt. Neurophysiologie, MPI für Hirnforschung, Deutschordenstr. 46, Frankfurt/Main, 60528, Germany

    Marcus Johannes Naumer

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science + Business Media, LLC

About this chapter

Cite this chapter

Newell, F.N. (2010). Visuo-haptic Perception of Objects and Scenes. In: Kaiser, J., Naumer, M. (eds) Multisensory Object Perception in the Primate Brain. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5615-6_14

Download citation

Publish with us

Policies and ethics

Search

Navigation