Experimental Brain Research

, Volume 216, Issue 1, pp 123–134 | Cite as

Categorizing natural objects: a comparison of the visual and the haptic modalities

  • Nina Gaissert
  • Christian Wallraven
Research Article


Although the hands are the most important tool for humans to manipulate objects, only little is known about haptic processing of natural objects. Here, we selected a unique set of natural objects, namely seashells, which vary along a variety of object features, while others are shared across all stimuli. To correctly interact with objects, they have to be identified or categorized. For both processes, measuring similarities between objects is crucial. Our goal is to better understand the haptic similarity percept by comparing it to the visual similarity percept. First, direct similarity measures were analyzed using multidimensional scaling techniques to visualize the perceptual spaces of both modalities. We find that the visual and the haptic modality form almost identical perceptual spaces. Next, we performed three different categorization tasks. All tasks exhibit a highly accurate processing of complex shapes of the haptic modality. Moreover, we find that objects grouped into the same category form regions within the perceptual space. Hence, in both modalities, perceived similarity constitutes the basis for categorizing objects. Moreover, both modalities focus on shape to form categories. Taken together, our results lead to the assumption that the same cognitive processes link haptic and visual similarity perception and the resulting categorization behavior.


Vision and haptics Similarity measures Categorization Perceptual spaces 



All seashells are items of loan from the natural history museum Stuttgart, Germany (Am Löwentor, Staatliches Museum für Naturkunde Stuttgart). We thank Hans-Jörg Niederhöfer for providing the seashells and helping to select an adequate set of stimuli. This study was partially supported by the WCU (World Class University) program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (R31-2008-000-10008-0).


  1. Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosc 4(3):324–330CrossRefGoogle Scholar
  2. Borg I, Groenen P (1997) Modern multidimensional scaling. Springer, BerlinGoogle Scholar
  3. Bushnell EW, Baxt C (1999) Children’s haptic, cross-modal recognition with familiar, unfamiliar objects. J Exp Psychol Hum Percept Perf 25(6):1867–1881CrossRefGoogle Scholar
  4. Cooke T, Jäkel F, Wallraven C, Bülthoff HH (2007) Multimodal similarity and categorization of novel, three-dimensional objects. Neuropsychologia 45(3):484–495PubMedCrossRefGoogle Scholar
  5. Cooke T, Wallraven C, Bülthoff HH (2010) Multidimensional scaling analysis of haptic exploratory procedures. ACM Trans App Percept 4:1–22Google Scholar
  6. Cutzu F, Edelman S (1998) Representation of object similarity in human vision: psychophysics, a computational model. Vision Res 38(15–16):2229–2257PubMedCrossRefGoogle Scholar
  7. Edelman S (1999) Representation and recognition in vision. MIT Press, CambridgeGoogle Scholar
  8. Gaissert N, Wallraven C, Bülthoff HH (2010) Visual and haptic perceptual spaces show high similarity in humans. J Vis 10(11:2):1–20Google Scholar
  9. Goldstone RL (1994) The role of similarity in categorization: providing a groundwork. Cognition 52(2):125–157PubMedCrossRefGoogle Scholar
  10. Haag S (2011) Effects of vision and haptics on categorizing common objects. Cognit Proc 12(1):33–39CrossRefGoogle Scholar
  11. Hahn U, Ramscar M (2001) Similarity and categorization. In: Hahn U, Ramscar M (eds) Similarity and categorization. Oxford University Press, OxfordGoogle Scholar
  12. Heinrich J, Heine SJ, Norenzayan A (2010) Most people are not weird. Nature 466:29CrossRefGoogle Scholar
  13. James TW, Kim S, Fisher JS (2007) The neural basis of haptic object processing. Can J Exp Psychol 61(3):219–229PubMedCrossRefGoogle Scholar
  14. Klatzky RL, Lederman SJ, Metzger VA (1985) Identifying objects by touch: an “expert system”. Percept Psychophys 37:299–302PubMedCrossRefGoogle Scholar
  15. Lacey S, Tal N, Amedi A, Sathian K (2009a) A putative model of multisensory object representation. Brain Topogr 21(3–4):269–274PubMedCrossRefGoogle Scholar
  16. Lacey S, Pappas M, Kreps A, Lee K, Sathian K (2009b) Perceptual learning of view-independence in visuo-haptic object representations. Exp Brain Res 198(2–3):329–337PubMedCrossRefGoogle Scholar
  17. Lederman SJ, Klatzky RL (1990) Haptic classification of common objects: knowledge-driven exploration. Cogn Psychol 22(4):421–459PubMedCrossRefGoogle Scholar
  18. Lederman SJ et al (1990) Visual mediation and the haptic recognition of two-dimensional pictures of common objects. Percept Psychophys 47(1):54–64PubMedCrossRefGoogle Scholar
  19. Linnaeus C (1758) Systema Naturae per Regna Tria Naturae, Secundum Classes, Ordines, Genera, Species cum Carateribus, Differentiis, Synonymis, Locis. Editio Decima, Reformata, vol 1 Regnum Animale. Laurentii Salvii, StockholmGoogle Scholar
  20. Loomis JM, Klatzky RL, Lederman SJ (1991) Similarity of tactual and visual picture recognition with limited field of view. Perception 20(2):167–177PubMedCrossRefGoogle Scholar
  21. Mervis CB, Rosch E (1981) Categorization of natural objects. Ann Rev 32:89–115Google Scholar
  22. Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G (2004) The visual and haptic perception of natural object shape. Percept Psychophys 66(2):342–351PubMedCrossRefGoogle Scholar
  23. Norman JF, Clayton AM, Norman HF, Crabtree CE (2008) Learning to perceive differences in solid shape through vision and touch. Perception 37(2):185–196Google Scholar
  24. Op de Beeck HP, Torfs K, Wagemans J (2008) Perceived shape similarity among unfamiliar objects and the organization of the human object vision pathway. J Neurosci 28(40):10111–10123PubMedCrossRefGoogle Scholar
  25. Palmeri TJ, Gauthier I (2004) Visual object understanding. Nat Rev Neurosci 5(4):291–303PubMedCrossRefGoogle Scholar
  26. Plaisier MA, Tiest WM, Kappers AM (2009) Salient features in 3-D haptic shape perception. Atten Percept Psychophys 71(2):421–430PubMedCrossRefGoogle Scholar
  27. Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn Mem Cognit 25(3):644–663CrossRefGoogle Scholar
  28. Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8(18):3877–3881PubMedCrossRefGoogle Scholar
  29. Shepard RN (1987) Toward a universal law of generalization for psychological science. Science 237(4820):1317–1323PubMedCrossRefGoogle Scholar
  30. Shepard RN (2001) Perceptual-cognitive universals as reflections of the world. Behav Brain Sci 24(4):581–601 (discussion 652–671)Google Scholar
  31. van der Horst BJ, Kappers AM (2008) Using curvature information in haptic shape perception of 3D objects. Exp Brain Res 190(3):361–367PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Max Planck Institute for Biological CyberneticsTübingenGermany
  2. 2.Korea UniversitySeoulRepublic of Korea

Personalised recommendations