Perception & Psychophysics

, Volume 70, Issue 1, pp 36–49 | Cite as

Behavioral evidence for task-dependent “what” versus “where” processing within and across modalities

  • Jason S. Chan
  • Fiona N. Newell


Task-dependent information processing for the purpose of recognition or spatial perception is considered a principle common to all the main sensory modalities. Using a dual-task interference paradigm, we investigated the behavioral effects of independent information processing for shape identification and localization of object features within and across vision and touch. In Experiment 1, we established that color and texture processing (i.e., a “what” task) interfered with both visual and haptic shape-matching tasks and that mirror image and rotation matching (i.e., a “where” task) interfered with a feature-location-matching task in both modalities. In contrast, interference was reduced when a “where” interference task was embedded in a “what” primary task and vice versa. In Experiment 2, we replicated this finding within each modality, using the same interference and primary tasks throughout. In Experiment 3, the interference tasks were always conducted in a modality other than the primary task modality. Here, we found that resources for identification and spatial localization are independent of modality. Our findings further suggest that multisensory resources for shape recognition also involve resources for spatial localization. These results extend recent neuropsychological and neuroimaging findings and have important implications for our understanding of high-level information processing across the human sensory systems.


Primary Task Location Task Shape Task Haptic Stimulus Ence Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001). Visuo-haptic, object-related activation in the ventral visual pathway. Nature Neuroscience, 4, 324–330.PubMedCrossRefGoogle Scholar
  2. Avillac, M., Denève, S., Olivier, E., Pouget, A., & Duhamel, J. R. (2005). Reference frames for representing visual and tactile locations in parietal cortex. Nature Neuroscience, 8, 941–949.PubMedGoogle Scholar
  3. Belin, P., & Zatorre, R. J. (2000). “What,” “where” and “how” in auditory cortex. Nature Neuroscience, 3, 965–966.PubMedCrossRefGoogle Scholar
  4. Calvert, G. A., Hansen, P. C., Steven, M. S., & Newell, F. N. (2007). An fMRI comparative study of visuotactile “what” and “where” systems. Manuscript in preparation.Google Scholar
  5. Clarke, S., Thiran, A. B., Maeder, P., Adriani, M., Vernet, O., Regli, L., et al. (2002). What and where in human audition: Selective deficits following focal hemispheric lesions. Experimental Brain Research, 147, 8–15.CrossRefGoogle Scholar
  6. Desimone, R., Schein, S. J., Moran, J., & Ungerleider, L. G. (1985). Contour, color and shape analysis beyond the striate cortex. Vision Research, 25, 441–452.PubMedCrossRefGoogle Scholar
  7. Desimone, R., & Ungerleider, L. G. (1986). Multiple visual areas in the caudal superior temporal sulcus of the macaque. Journal of Comparative Neurology, 248, 164–189.PubMedCrossRefGoogle Scholar
  8. Duncan, J. (1984). Selective attention and the organization of visual information. Journal of Experimental Psychology: General, 113, 501–517.CrossRefGoogle Scholar
  9. Easton, R. D., Srinivas, K., & Greene, A. J. (1997). Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. Journal of Experimental Psychology: Learning, Memory, & Cognition, 23, 153–163.CrossRefGoogle Scholar
  10. Egly, R., Driver, J., & Rafal, R. D. (1994). Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology: General, 123, 161–177.CrossRefGoogle Scholar
  11. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1–47.PubMedCrossRefGoogle Scholar
  12. Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15, 20–25.PubMedCrossRefGoogle Scholar
  13. Haxby, J. V., Grady, C. L., Horwitz, B., Ungerleider, L. G., Mishkin, M., Carson, R. E., et al. (1991). Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proceedings of the National Academy of Sciences, 88, 1621–1625.CrossRefGoogle Scholar
  14. Haxby, J. V., Horwitz, B., Ungerleider, L. G., Maisog, J. M., Pietrini, P., & Grady, C. L. (1994). The functional organization of human extrastriate cortex: A PET-rCBF study of selective attention to faces and locations. Journal of Neuroscience, 14, 6336–6353.PubMedGoogle Scholar
  15. Hirst, W., & Kalmar, D. (1987). Characterizing attentional resources. Journal of Experimental Psychology: General, 116, 68–81.CrossRefGoogle Scholar
  16. Humphreys, G. W., & Riddoch, M. J. (2003). From what to where: Neuropsychological evidence for implicit interactions between objectand space-based attention. Psychological Science, 14, 487–492.PubMedCrossRefGoogle Scholar
  17. Irwin, D. E., & Brockmole, J. R. (2004). Suppressing where but not what: The effect of saccades on dorsal- and ventral-stream visual processing. Psychological Science, 15, 467–473.PubMedCrossRefGoogle Scholar
  18. James, T. W., Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: An fMRI study. Brain, 126, 2463–2475.PubMedCrossRefGoogle Scholar
  19. James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon, R. S., & Goodale, M. A. (2002). Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia, 40, 1706–1714.PubMedCrossRefGoogle Scholar
  20. Kappers, A. M. L. (1999). Large systematic deviations in the haptic perception of parellelity. Perception, 28, 1001–1012.PubMedCrossRefGoogle Scholar
  21. Kerzel, D. (2001). Visual short-term memory is influenced by haptic perception. Journal of Experimental Psychology: Learning, Memory, & Cognition, 27, 1101–1109.CrossRefGoogle Scholar
  22. Kinsbourne, M. (1980). Mapping a behavioral cerebral space. International Journal of Neuroscience, 11, 45–50.PubMedCrossRefGoogle Scholar
  23. Kitada, R., Kito, T., Saito, D. N., Kochiyama, T., Matsumura, M., Sadato, N., & Lederman, S. J. (2006). Multisensory activation of the intraparietal area when classifying grating orientation: A functional magnetic resonance imaging study. Journal of Neuroscience, 26, 7491–7501.PubMedCrossRefGoogle Scholar
  24. Landau, B., Hoffman, J. E., & Kurz, N. (2006). Object recognition with severe spatial deficits in Williams syndrome: Sparing and breakdown. Cognition, 100, 483–510.PubMedCrossRefGoogle Scholar
  25. Lederman, S. J., & Klatzky, R. J. (1987). Hand movements: A window into haptic object recognition. Cognitive Psychology, 19, 342–368.PubMedCrossRefGoogle Scholar
  26. Liu, T., Slotnick, S. D., Serences, J. T., & Yantis, S. (2003). Cortical mechanisms of feature-based attentional control. Cerebral Cortex, 13, 1334–1343.PubMedCrossRefGoogle Scholar
  27. Logie, R. H., & Marchetti, C. (1991). Visuo-spatial working memory: Visual, spatial or central executive? In R. H. Logie & M. Denis (Eds.), Mental images in human cognition (pp. 105–115). Amsterdam: North-Holland.CrossRefGoogle Scholar
  28. Marois, R., Leung, H.-C., & Gore, J. (2000). A stimulus-driven approach to object identity and location processing in the human brain. Neuron, 25, 717–728.PubMedCrossRefGoogle Scholar
  29. Merigan, W. H., & Maunsell, J. H. (1993). How parallel are the primate visual pathways? Annual Review of Neuroscience, 16, 369–402.PubMedCrossRefGoogle Scholar
  30. Milner, D. A., & Goodale, M. A. (1995). The visual brain in action. New York: Oxford University Press.Google Scholar
  31. Mishkin, M., Ungerleider, L. G., & Macko, K. A. (1983). Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences, 6, 414–417.CrossRefGoogle Scholar
  32. Newcombe, F., & Russell, W. R. (1969). Dissociated visual perceptual and spatial deficits in focal lesions of the right hemisphere. Journal of Neurology, Neurosurgery, & Psychiatry, 32, 73–81.CrossRefGoogle Scholar
  33. Newell, F. N., Ernst, M. O., Tjan, B. S., & Bülthoff, H. H. (2001). Viewpoint dependence in visual and haptic object recognition. Psychological Science, 12, 37–42.PubMedCrossRefGoogle Scholar
  34. Newell, F. N., Woods, A. T., Mernagh, M., & Bülthoff, H. H. (2005). Visual, haptic and crossmodal recognition of scenes. Experimental Brain Research, 161, 233–242.CrossRefGoogle Scholar
  35. Poremba, A., Saunders, R. C., Crane, A. M., Cook, M., Sokoloff, L., & Mishkin, M. (2003). Functional mapping of the primate auditory system. Science, 299, 568–572.PubMedCrossRefGoogle Scholar
  36. Prather, S. C., Votaw, J. R., & Sathian, K. (2004). Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia, 42, 1079–1087.PubMedCrossRefGoogle Scholar
  37. Reales, J. M., & Ballesteros, S. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 644–663.CrossRefGoogle Scholar
  38. Reed, C. L., & Caselli, R. J. (1994)0. The nature of tactile agnosia: A case study. Neuropsychologia, 32, 527–539.PubMedCrossRefGoogle Scholar
  39. Reed, C. L., Caselli, R. J., & Farah, M. J. (1996). Tactile agnosia: Underlying impairment and implications for normal tactile object recognition. Brain, 119, 875–888.PubMedCrossRefGoogle Scholar
  40. Reed, C. L., Klatzky, R. L., & Halgren, E. (2005). What vs. where in touch: An fMRI study. NeuroImage, 25, 718–726.PubMedCrossRefGoogle Scholar
  41. Reed, C. L., Shoham, S., & Halgren, E. (2004). Neural substrates of tactile object recognition: An fMRI study. Human Brain Mapping, 21, 236–246.PubMedCrossRefGoogle Scholar
  42. Romanski, L. M., Tian, B., Fritz, J., Mishkin, M., Goldman-Rakic, P. S., & Rauschecker, J. P. (1999). Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nature Neuroscience, 2, 1131–1136.PubMedCrossRefGoogle Scholar
  43. Serences, J. T., Schwarzbach, J., Courtney, S. M., Golay, X., & Yantis, S. (2004). Control of object-based attention in human cortex. Cerebral Cortex, 14, 1346–1357.PubMedCrossRefGoogle Scholar
  44. Smyth, M. M., & Pendleton, L. R. (1989). Working memory for movements. Quarterly Journal of Experimental Psychology, 41A, 235–250.Google Scholar
  45. Tresch, M. C., Sinnamon, H. M., & Seamon, J. G. (1993). Double dissociation of spatial and object visual memory: Evidence from se lective interference in intact human subject. Neuropsychologia, 31, 211–219.PubMedCrossRefGoogle Scholar
  46. Ungerleider, L. G., Galkin, T. W., & Mishkin, M. (1983). Visuotopic organization of projections from striate cortex to inferior and lateral pulvinar in rhesus monkey. Journal of Comparative Neurology, 217, 137–157.PubMedCrossRefGoogle Scholar
  47. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, & R. J. Mansfield (Eds.), Analysis of visual behavior (pp. 549–586). Cambridge, MA: MIT Press.Google Scholar
  48. Vecera, S. P., & Farah, M. J. (1994). Does visual attention select objects or locations? Journal of Experimental Psychology: General, 123, 146–160.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2008

Authors and Affiliations

  1. 1.School of PsychologyTrinity CollegeDublin 2Ireland

Personalised recommendations