Psychonomic Bulletin & Review

, Volume 21, Issue 4, pp 976–985 | Cite as

The eyes grasp, the hands see: Metric category knowledge transfers between vision and touch

  • Christian Wallraven
  • Heinrich H. Bülthoff
  • Steffen Waterkamp
  • Loes van Dam
  • Nina Gaißert
Brief Report

Abstract

Categorization of seen objects is often determined by the shapes of objects. However, shape is not exclusive to the visual modality: The haptic system also is expert at identifying shapes. Hence, an important question for understanding shape processing is whether humans store separate modality-dependent shape representations, or whether information is integrated into one multisensory representation. To answer this question, we created a metric space of computer-generated novel objects varying in shape. These objects were then printed using a 3-D printer, to generate tangible stimuli. In a categorization experiment, participants first explored the objects visually and haptically. We found that both modalities led to highly similar categorization behavior. Next, participants were trained either visually or haptically on shape categories within the metric space. As expected, visual training increased visual performance, and haptic training increased haptic performance. Importantly, however, we found that visual training also improved haptic performance, and vice versa. Two additional experiments showed that the location of the categorical boundary in the metric space also transferred across modalities, as did heightened discriminability of objects adjacent to the boundary. This observed transfer of metric category knowledge across modalities indicates that visual and haptic forms of shape information are integrated into a shared multisensory representation.

Keywords

Shape Object categorization Vision Haptics Categorization Multisensory representations 

Notes

Author note

This research was supported by a PhD stipend from the Max Planck Society; by the WCU (World Class University) program through the National Research Foundation (NRF) of Korea, funded by the Ministry of Education, Science and Technology (Grant No. R31-2008-000-10008-0); by the Basic Science Research Program through the National Research Foundation of Korea, funded by the Ministry of Science, ICT and Future Planning (Grant No. NRF-2013R1A1A1011768); and by the Brain Korea 21 PLUS Program through the National Research Foundation of Korea, funded by the Ministry of Education.

References

  1. Bornstein, M. H. (1987). Perceptual categories in vision and audition. In S. Harnad (Ed.), Categorical perception: The groundwork of cognition (pp. 287–300). New York, NY: Cambridge University Press.Google Scholar
  2. Bushnell, E. W., & Baxt, C. (1999). Children’s haptic and cross-modal recognition with familiar and unfamiliar objects. Journal of Experimental Psychology: Human Perception and Performance, 25, 1867–1881. doi: 10.1037/0096-1523.25.6.1867 PubMedGoogle Scholar
  3. Cooke, T., Jäkel, F., Wallraven, C., & Bülthoff, H. H. (2007). Multimodal similarity and categorization of novel, three-dimensional objects. Neuropsychologia, 45, 484–495. doi: 10.1016/j.neuropsychologia.2006.02.009 PubMedCrossRefGoogle Scholar
  4. Dopjans, L., Wallraven, C., & Bülthoff, H. H. (2009). Cross-modal transfer in visual and haptic face recognition. IEEE Transactions on Haptics, 200, 236–240.CrossRefGoogle Scholar
  5. Edelman, S., & Shahbazi, R. (2012). Renewing the respect for similarity. Frontiers in Computational Neuroscience, 6, 45. doi: 10.3389/fncom.2012.00045 PubMedCentralPubMedCrossRefGoogle Scholar
  6. Gaissert, N., & Wallraven, C. (2011). Categorizing natural objects—A comparison of the visual and haptic modalities. Experimental Brain Research, 216, 123–134.PubMedCrossRefGoogle Scholar
  7. Gaissert, N., Wallraven, C., & Bülthoff, H. H. (2010). Visual and haptic perceptual spaces show high similarity in humans. Journal of Vision, 10(11), 1–20. doi: 10.1167/10.11.2 CrossRefGoogle Scholar
  8. Harnad, S. (1987). Psychophysical and cognitive aspects of categorical perception: A critical overview. In S. Harnad (Ed.), Categorical perception: The groundwork of cognition (pp. 1–28). New York, NY: Cambridge University Press.Google Scholar
  9. Kikutani, M., Roberson, D., & Hanley, J. R. (2010). Categorical perception for unfamiliar faces: The effect of covert and overt face learning. Psychological Science, 21, 865–872. doi: 10.1177/0956797610371964 PubMedCrossRefGoogle Scholar
  10. Klatzky, R. L., Lederman, S. J., & Metzger, V. A. (1985). Identifying objects by touch: An “expert system. Perception & Psychophysics, 37, 299–302.CrossRefGoogle Scholar
  11. Lacey, S., Campbell, C., & Sathian, K. (2007). Vision and touch: Multiple or multisensory representations of objects? Perception, 36, 1513–1521.PubMedCrossRefGoogle Scholar
  12. Newell, F., & Bülthoff, H. H. (2002). Categorical perception of familiar objects. Cognition, 85, 113–143.PubMedCrossRefGoogle Scholar
  13. Norman, J. F., Norman, H. F., Clayton, A. M., Lianekhammy, J., & Zielke, G. (2004). The visual and haptic perception of natural object shape. Perception & Psychophysics, 66, 342–351. doi: 10.3758/BF03194883 CrossRefGoogle Scholar
  14. Pastore, R. E. (1987). Categorical perception: Some psychophysical models. In S. Harnad (Ed.), Categorical perception: The groundwork of cognition (pp. 29–52). New York, NY: Cambridge University Press.Google Scholar
  15. Phillips, F., Egan, E. J. L., & Perry, B. N. (2009). Perceptual equivalence between vision and touch is complexity dependent. Acta Psychologica, 132, 259–266. doi: 10.1016/j.actpsy.2009.07.010 PubMedCrossRefGoogle Scholar
  16. Reales, J. M., & Ballesteros, S. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 644–663. doi: 10.1037/0278-7393.25.3.644 Google Scholar
  17. Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8, 382–439. doi: 10.1016/0010-0285(76)90013-X CrossRefGoogle Scholar
  18. Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237, 1317–1323. doi: 10.1126/science.3629243 PubMedCrossRefGoogle Scholar
  19. Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293–1313. doi: 10.3758/BF03194544 CrossRefGoogle Scholar
  20. Yildirim, I., & Jacobs, R. A. (2013). Transfer of object category knowledge across visual and haptic modalities: Experimental and computational studies. Cognition, 126, 135–148.PubMedCrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  • Christian Wallraven
    • 1
  • Heinrich H. Bülthoff
    • 1
    • 2
  • Steffen Waterkamp
    • 2
  • Loes van Dam
    • 3
  • Nina Gaißert
    • 2
  1. 1.Department of Brain and Cognitive EngineeringKorea UniversitySeongbuk-guKorea
  2. 2.Max Planck Institute for Biological CyberneticsTübingenGermany
  3. 3.University of BielefeldBielefeldGermany

Personalised recommendations