Advertisement

Experimental Brain Research

, Volume 227, Issue 3, pp 311–322 | Cite as

Isolating shape from semantics in haptic-visual priming

  • Ana PesquitaEmail author
  • Allison A. Brennan
  • James T. Enns
  • Salvador Soto-Faraco
Research Article

Abstract

The exploration of a familiar object by hand can benefit its identification by eye. What is unclear is how much this multisensory cross-talk reflects shared shape representations versus generic semantic associations. Here, we compare several simultaneous priming conditions to isolate the potential contributions of shape and semantics in haptic-to-visual priming. Participants explored a familiar object manually (haptic prime) while trying to name a visual object that was gradually revealed in increments of spatial resolution. Shape priming was isolated in a comparison of identity priming (shared semantic category and shape) with category priming (same category, but different shapes). Semantic priming was indexed by the comparisons of category priming with unrelated haptic primes. The results showed that both factors mediated priming, but that their relative weights depended on the reliability of the visual information. Semantic priming dominated in Experiment 1, when participants were free to use high-resolution visual information, but shape priming played a stronger role in Experiment 2, when participants were forced to respond with less reliable visual information. These results support the structural description hypothesis of haptic-visual priming (Reales and Ballesteros in J Exp Psychol Learn Mem Cogn 25:644–663, 1999) and are also consistent with the optimal integration theory (Ernst and Banks in Nature 415:429–433, 2002), which proposes a close coupling between the reliability of sensory signals and their weight in decision making.

Keywords

Multisensory Vision Haptic Cross-modal Priming Weighted decisions 

Notes

Acknowledgments

The authors wish to thank Angela Zhang for her assistance in data collection. This work was supported by a PhD scholarship to author AP from the Portuguese Fundação para a Ciência e Tecnologia (SFRH/BD/76087/2011), an Exchange Fellowship to author AAB from the Dr. Michael Quinn Memorial Fund, Department of Psychology, University of British Columbia, an NSERC (Canada) Discovery Grant to author JTE, and Grants to author SS-F from the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012 Grants, the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092), and the European Research Council (StG-2010 263145).

Supplementary material

221_2013_3489_MOESM1_ESM.mpg (198 kb)
An example of the progressive revelation of a target object used in Experiment 1 (mp4 file) (MPG 198 kb)
221_2013_3489_MOESM2_ESM.pptx (623 kb)
Photos of the Experimental set-up used in Experiment 1 and 2. (A) View of the participant (left) and experimenter (right). (B) Participant’s view of the visual target sequence. (C). Experimenter’s view of the haptic prime being explored by the participant. (PPTX 622 kb)

References

  1. Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330. doi: 10.1038/85201 PubMedCrossRefGoogle Scholar
  2. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559–571. doi: 10.1007/s00221-005-2396-5 PubMedCrossRefGoogle Scholar
  3. Ballesteros S, González M, Mayas J, García-Rodríguez B, Reales JM (2009) Cross-modal repetition priming in young and old adults. Eur J Cogn Psychol 21(2/3):366–387. doi: 10.1080/09541440802311956 Google Scholar
  4. Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 94(2):115–147PubMedCrossRefGoogle Scholar
  5. Biederman I, Cooper EE (1991) Evidence for complete translational and reflectional invariance in visual object priming. Perception 20:585–593PubMedCrossRefGoogle Scholar
  6. Biederman I, Cooper EE (2009) Biederman and Cooper’s 1991 paper translational and reflectional priming invariance: a retrospective. Perception 38:809–826. doi: 10.1068/ldmk-bie PubMedCrossRefGoogle Scholar
  7. Bushnell EW, Baxt C (1999) Children’s haptic and cross-modal recognition with familiar and unfamiliar objects. J Exp Psychol Hum Percept Perform 25:1867–1881PubMedCrossRefGoogle Scholar
  8. Carr TH, McCauley C, Sperber RD, Parmelee CM (1982) Words, pictures, and priming: on semantic activation, conscious identification, and the automaticity of information processing. J Exp Psychol Hum Percept Perform 8:757–777. doi: 10.1037/0096-1523.8.6.757 PubMedCrossRefGoogle Scholar
  9. Craddock M, Lawson R (2008) Repetition priming and the haptic recognition of familiar and unfamiliar objects. Percept Psychophys 70:1350–1365. doi: 10.3758/PP.70.7.1350 Google Scholar
  10. Craddock M, Lawson R (2009) Size-sensitive perceptual representations underlie visual and haptic object recognition. PLoS ONE 4(11):e8009. doi: 10.1371/journal.pone.0008009 PubMedCrossRefGoogle Scholar
  11. Easton R, Greene AJ, Srinivas K (1997a) Transfer between vision and haptics: memory for 2-D patterns and 3-D objects. Psychon Bull Rev 4(3):403–410CrossRefGoogle Scholar
  12. Easton RD, Srinivas K, Greene AJ (1997b) Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. J Exp Psychol Learn Mem Cogn 23(1):153–163PubMedCrossRefGoogle Scholar
  13. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433. doi: 10.1038/415429a PubMedCrossRefGoogle Scholar
  14. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169. doi: 10.1016/j.tics.2004.02.002 PubMedCrossRefGoogle Scholar
  15. Ernst MO, Lange C, Newell FN (2007) Multisensory recognition of actively explored objects. Can J Exp Psychol 61:242–253. doi: 10.1037/cjep2007025 PubMedCrossRefGoogle Scholar
  16. Faul F, Erdfelder E, Lang AG, Buchner A (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39:175–191PubMedCrossRefGoogle Scholar
  17. Friese U, Supp GG, Hipp JF, Engel AK, Gruber T (2012) Oscillatory MEG gamma band activity dissociates perceptual and conceptual aspects of visual object processing: a combined repetition/conceptual priming study. NeuroImage 59:86171. doi: 10.1016/j.neuroimage.2011.07.073 CrossRefGoogle Scholar
  18. Glaser WR (1992) Picture naming. Cognition 42:61–105. doi: 10.1016/0010-0277(92)90040-O PubMedCrossRefGoogle Scholar
  19. Gottfried AW, Rose SA, Bridger WH (1977) Cross-modal transfer in human infants. Child Dev 48:118–123PubMedCrossRefGoogle Scholar
  20. Grefkes C, Weiss PH, Zilles K, Fink GR (2002) Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron 35:173–184PubMedCrossRefGoogle Scholar
  21. Helbig HB, Ernst MO (2007) Optimal integration of shape information from vision and touch. Exp Brain Res 179:595–606. doi: 10.1007/s00221-006-0814-y PubMedCrossRefGoogle Scholar
  22. Hillis AE, Caramazza A (1995) Cognitive and neural mechanisms underlying visual and semantic processing: implications from “optic aphasia”. J Cognit Neurosci 7:457–478. doi: 10.1162/jocn.1995.7.4.457 CrossRefGoogle Scholar
  23. Humphreys GW, Price CJ, Riddoch MJ (1999) From objects to names: a cognitive neuroscience approach. Psychol Res 62:118–130PubMedCrossRefGoogle Scholar
  24. Iordanescu L, Guzman-Martinez E, Grabowecky M, Suzuki S (2008) Characteristic sounds facilitate visual search. Psychon Bull Rev 15(3):548–554PubMedCrossRefGoogle Scholar
  25. James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714. doi: 10.1016/S0028-3932(02)00017-9 PubMedCrossRefGoogle Scholar
  26. Johnson L, Paivio A, Clark J (1989) Spatial and verbal abilities in children’s cross-modal recognition: a dual-coding approach. Can J Psychol 43:397–412CrossRefGoogle Scholar
  27. Kahlaoui K, Baccino T, Joanette Y, Magnié MN (2007) Pictures and words: priming and category effects in object processing. Curr Psychol Lett Behav Brain Cognit 3:2–13Google Scholar
  28. Klatzky RL, Lederman SJ (1999) The haptic glance: a route to rapid object identification and manipulation. In: Gopher D, Koriat A (eds) Attention and performance XVII. The MIT Press, Cambridge, pp 164–196Google Scholar
  29. Lacey S, Campbell C (2006) Mental representation in visual/haptic crossmodal memory: evidence from interference effects. Q J Exp Psychol 59:361–376. doi: 10.1080/17470210500173232 CrossRefGoogle Scholar
  30. Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21:69–274. doi: 10.1007/s10548-009-0087-4 CrossRefGoogle Scholar
  31. Lawson R (2009) A comparison of the effects of depth rotation on visual and haptic three-dimensional object recognition. J Exp Psychol Hum Percept Perform 35(4):911–930. doi: 10.1037/a0015025 PubMedCrossRefGoogle Scholar
  32. Lewkowicz DJ (1994) Development of intersensory perception in human infants. In: Lickliter DJ, Lewkowicz R (eds) The development of inter- sensory perception: comparative perspectives. Hillsdale, Hove UK, pp 165–203Google Scholar
  33. Meltzoff AN, Borton RW (1979) Intermodal matching by human neonates. Nature 282:403–404PubMedCrossRefGoogle Scholar
  34. Neely JH (1977) Semantic priming and retrieval from lexical memory: roles of inhibitionless spreading activation and limited-capacity attention. J Exp Psychol Gen 106:226–254CrossRefGoogle Scholar
  35. Newell FN (2004) Cross-modal object recognition. In: Calvert G, Spence C, Stein BE (eds) The handbook of multisensory processes. MIT Press, Cambridge, MA, pp 123–139Google Scholar
  36. Newell FN, Ernst MO, Tjan BS, Bulthoff HH (2001) Viewpoint dependence in visual and haptic object recognition. Psychol Sci 12:37–42. doi: 10.1111/1467-9280.00307 PubMedCrossRefGoogle Scholar
  37. Orgs G, Lange K, Dombrowski JH, Heil M (2006) Conceptual priming for environmental sounds and words: an ERP study. Brain Cogn 62:267–272. doi: 10.1016/j.bandc.2006.05.003 PubMedCrossRefGoogle Scholar
  38. Paivio A (1991) Dual coding theory: retrospect and current status. Can J Psychol 45:255–287CrossRefGoogle Scholar
  39. Piai V, Roelofs A, Schriefers H (2011) Semantic interference in immediate and delayed naming and reading: attention and task decisions. J Mem Lang 64:404–423. doi: 10.1016/j.jml.2011.01.004 CrossRefGoogle Scholar
  40. Ploran EJ, Nelson SM, Velanova K, Donaldson DI, Petersen SE, Wheeler ME (2007) Evidence accumulation and the moment of recognition: dissociating perceptual recognition processes using fMRI. J Neurosci 27:11912–11924. doi: 10.1523/JNEUROSCI.3522-07.2007 PubMedCrossRefGoogle Scholar
  41. Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn Mem Cogn 25:644–663. doi: 10.1037//0278-7393.25.3.644 CrossRefGoogle Scholar
  42. Rentschler I, Juttner M, Osman E, Muller A, Caelli T (2004) Development of configural 3D object recognition. Behav Brain Res 149:107–111. doi: 10.1016/S0166-4328(03)00194-3 PubMedCrossRefGoogle Scholar
  43. Rose SA (1994) From hand to eye: findings and issues in infant cross-modal transfer. In: Lickliter (ed) The development of intersensory perception: Comparative perspectives. Psychology Press, Hove UK, pp 265–284Google Scholar
  44. Saito DN, Okada T, Morita Y, Yonekura Y, Sadato N (2003) Tactile-visual cross-modal shape matching: a functional MRI study. Cogn Brain Res 17:14–25. doi: 10.1016/S0926-6410(03)00076-4 CrossRefGoogle Scholar
  45. Schacter DL, Dobbins IG, Schnyer DM (2004) Specificity of priming: a cognitive neuroscience perspective. Nat Rev Neurosci 5:853–862. doi: 10.1038/nrn1534 PubMedCrossRefGoogle Scholar
  46. Stadthagen-Gonzalez H, Damian MF, Pérez MA, Bowers JS, Marín J (2009) Name-picture verification as a control measure for object naming: a task analysis and norms for a large set of pictures. Q J Exp Psychol 62:1581–1597. doi: 10.1080/17470210802511139 CrossRefGoogle Scholar
  47. Tipper SP, Driver J (1988) Negative priming between pictures and words in a selective attention task: evidence for semantic processing of ignored stimuli. Memory Cogn 16:64–70. doi: 10.3758/BF03197746 CrossRefGoogle Scholar
  48. Walker-Andrews AS (1994) Taxonomy for intermodal relations. In: Lewkowicz DJ, Lickliter R (eds) The development of intersensory perception: comparative perspectives. Psychology Press, Hove UK, pp 39–56Google Scholar
  49. Welch R, Warren D (1986) Handbook of perception and human performance, vol 1. Wiley-Interscience, New YorkGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ana Pesquita
    • 1
    Email author
  • Allison A. Brennan
    • 1
  • James T. Enns
    • 1
  • Salvador Soto-Faraco
    • 2
  1. 1.Department of PsychologyUniversity of British ColumbiaVancouverCanada
  2. 2.Departament de Tecnologies de la Informació i les ComunicacionsUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations