Experimental Brain Research

, Volume 195, Issue 2, pp 261–272 | Cite as

Multisensory warning signals: when spatial correspondence matters

Research Article
First Online:
Received:
Accepted:

Abstract

We report a study designed to investigate the effectiveness of task-irrelevant unimodal and bimodal audiotactile stimuli in capturing a person’s spatial attention away from a highly perceptually demanding central rapid serial visual presentation (RSVP) task. In “Experiment 1”, participants made speeded elevation discrimination responses to peripheral visual targets following the presentation of auditory stimuli that were either presented alone or else were paired with centrally presented tactile stimuli. The results showed that the unimodal auditory stimuli only captured spatial attention when participants were not performing the RSVP task, while the bimodal audiotactile stimuli did not result in any performance change in any of the conditions. In “Experiment 2”, spatial auditory stimuli were either presented alone or else were paired with a tactile stimulus presented from the same direction. In contrast to the results of “Experiment 1”, the bimodal audiotactile stimuli were especially effective in capturing participants’ spatial attention from the concurrent RSVP task. These results therefore provide support for the claim that auditory and tactile stimuli should be presented from the same direction if they are to capture attention effectively. Implications for multisensory warning signal design are discussed.

Keywords

Warning signals Multisensory integration Attentional capture Directional congruency Auditory Tactile 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This study was supported by a Clarendon Fund Bursary from Oxford University, an Overseas Research Students (ORS) Award, and a Somerville Senior Scholarship from Somerville College, Oxford to C. H.

References

  1. Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile “capture” of audition. Percept Psychophys 64:616–630PubMedGoogle Scholar
  2. Calvert GA, Spence C, Stein BE (eds) (2004) The handbook of multisensory processes. MIT Press, Cambridge, MAGoogle Scholar
  3. Chan AHS, Chan KWL (2006) Synchronous and asynchronous presentation of auditory and visual signals: implications for control console design. Appl Ergon 37:131–140PubMedCrossRefGoogle Scholar
  4. Cohen J (1973) Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educ Psychol Measur 33:107–112CrossRefGoogle Scholar
  5. Collins CC (1970) Tactile television—mechanical and electrical image projection. IEEE Trans Man Mach Syst MMS-11:65–71CrossRefGoogle Scholar
  6. Ferris T, Penfold R, Hameed S, Sarter N (2006) The implications of crossmodal links in attention for the design of multimodal interfaces: a driving simulator study. In: Proceedings of the Human Factors and Ergonomics Society 50th annual meeting, pp 406–409Google Scholar
  7. Fitch GM, Kiefer RJ, Hankey JM, Kleiner BM (2007) Toward developing an approach for alerting drivers to the direction of a crash threat. Hum Factors 49:710–720PubMedCrossRefGoogle Scholar
  8. Hennessy JR (1966) Cutaneous sensitivity communications. Hum Factors 8:463–469PubMedGoogle Scholar
  9. Ho C, Spence C (2005) Assessing the effectiveness of various auditory cues in capturing a driver’s visual attention. J Exp Psychol Appl 11:157–174PubMedCrossRefGoogle Scholar
  10. Ho C, Spence C (2007) Head orientation biases tactile localization. Brain Res 1144:136–141PubMedCrossRefGoogle Scholar
  11. Ho C, Spence C (2008) The multisensory driver: implications for ergonomic car interface design. Ashgate, AldershotGoogle Scholar
  12. Ho C, Reed N, Spence C (2007) Multisensory in-car warning signals for collision avoidance. Hum Factors 49:1107–1114PubMedCrossRefGoogle Scholar
  13. Hötting K, Rösler F, Röder B (2003) Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Exp Brain Res 148:26–37PubMedCrossRefGoogle Scholar
  14. Kitagawa N, Spence C (2006) Audiotactile multisensory interactions in human information processing. Jpn Psychol Res 48:158–173CrossRefGoogle Scholar
  15. Kitagawa N, Zampini M, Spence C (2005) Audiotactile interactions in near and far space. Exp Brain Res 166:528–537PubMedCrossRefGoogle Scholar
  16. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:405–414Google Scholar
  17. Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27:1155–1163CrossRefGoogle Scholar
  18. Lavie N (2005) Distracted and confused? Selective attention under load. Trends Cogn Sci 9:75–82PubMedCrossRefGoogle Scholar
  19. Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC et al (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963–974PubMedCrossRefGoogle Scholar
  20. Oyer J, Hardick J (1963) Response of population to optimum warning signal (Report No. SHSLR163). Office of Civil Defence, Washington, DCGoogle Scholar
  21. Poliakoff E, Miles E, Li X, Blanchette I (2007) The effect of visual threat on spatial attention to touch. Cognition 102:405–414PubMedCrossRefGoogle Scholar
  22. Santangelo V, Spence C (2007a) Assessing the automaticity of reflexive tactile attentional orienting. Perception 36:1497–1505PubMedCrossRefGoogle Scholar
  23. Santangelo V, Spence C (2007b) Multisensory cues capture spatial attention regardless of perceptual load. J Exp Psychol Hum Percept Perform 33:1311–1321PubMedCrossRefGoogle Scholar
  24. Santangelo V, Spence C (2008) Is the exogenous orienting of spatial attention truly automatic? Evidence from unimodal and multisensory studies. Conscious Cogn 17:989–1015PubMedCrossRefGoogle Scholar
  25. Santangelo V, Van der Lubbe RHJ, Berlardinelli MO, Postma A (2006) Spatial attention triggered by unimodal, crossmodal, and bimodal exogenous cues: a comparison of reflexive orienting mechanisms. Exp Brain Res 173:40–48PubMedCrossRefGoogle Scholar
  26. Santangelo V, Finoia P, Raffone A, Olivetti Belardinelli M, Spence C (2007a) Perceptual load affects exogenous spatial orienting while working memory load does not. Exp Brain Res 184:371–382PubMedCrossRefGoogle Scholar
  27. Santangelo V, Olivetti Belardinelli M, Spence C (2007b) The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. J Exp Psychol Hum Percept Perform 33:137–148PubMedCrossRefGoogle Scholar
  28. Santangelo V, Ho C, Spence C (2008) Capturing spatial attention with multisensory cues. Psychon Bull Rev 15:398–403PubMedCrossRefGoogle Scholar
  29. Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327PubMedGoogle Scholar
  30. Schürmann M, Caetano G, Jousmäki V, Hari R (2004) Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J Acoust Soc Am 115:830–832PubMedCrossRefGoogle Scholar
  31. Selcon SJ, Taylor RM, McKenna FP (1995) Integrating multiple information sources: using redundancy in the design of warnings. Ergonomics 38:2362–2370CrossRefGoogle Scholar
  32. Spence C, Driver J (1999) A new approach to the design of multimodal warning signals. In: Harris D (ed) Engineering psychology and cognitive ergonomics. Job design, product design and human-computer interaction, vol 4. Ashgate, Aldershot, pp 455–461Google Scholar
  33. Spence C, Driver J (2000) Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism. Neuroreport 11:2057–2061PubMedCrossRefGoogle Scholar
  34. Spence C, Driver J (eds) (2004) Crossmodal space and crossmodal attention. Oxford University Press, OxfordGoogle Scholar
  35. Spence C, Ho C (2008a) Multisensory warning signals for event perception and safe driving. Theor Issues Ergon Sci 9:523–554CrossRefGoogle Scholar
  36. Spence C, Ho C (2008b) Tactile & multisensory interface design for drivers. IEEE Trans Haptics 1:121–129CrossRefGoogle Scholar
  37. Spence C, Nicholls MER, Gillespie N, Driver J (1998) Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percept Psychophys 60:544–557PubMedGoogle Scholar
  38. Spence C, McDonald J, Driver J (2004) Exogenous spatial cuing studies of human crossmodal attention and multisensory integration. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, pp 277–320Google Scholar
  39. Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, MAGoogle Scholar
  40. Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9:255–267PubMedCrossRefGoogle Scholar
  41. Stein BE, Stanford TR, Wallace MT, Vaughan WJ, Jiang W (2004) Crossmodal spatial interactions in subcortical and cortical circuits. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, UK, pp 25–50Google Scholar
  42. Tajadura-Jiménez A, Kitagawa N, Väljamäe A, Zampini M, Murray MM, Spence C (2009) Auditory-somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia 47:195–203PubMedCrossRefGoogle Scholar
  43. van Erp JBF, van Veen HAHC (2004) Vibrotactile in-vehicle navigation system. Transp Res F Traffic Psychol Behav 7:247–256CrossRefGoogle Scholar
  44. Vroomen J, Bertelson P, de Gelder B (2001) Directing spatial attention towards the illusory location of a ventriloquized sound. Acta Psychol 108:21–33CrossRefGoogle Scholar
  45. Vu KL, Strybel TZ, Proctor RW (2006) Effects of displacement magnitude and direction of auditory cues on auditory spatial facilitation of visual search. Hum Factors 48:587–599PubMedCrossRefGoogle Scholar
  46. Ward LM, McDonald JJ, Golestani N (1998) Cross-modal control of attention shifts. In: Wright RD (ed) Visual attention. Oxford University Press, New York, pp 232–268Google Scholar
  47. Zampini M, Brown T, Shore DI, Maravita A, Roder B, Spence C (2005) Audiotactile temporal order judgments. Acta Psychol 118:277–291CrossRefGoogle Scholar
  48. Zampini M, Torresan D, Spence C, Murray MM (2007) Auditory-somatosensory multisensory interactions in front and rear space. Neuropsychologia 45:1869–1877PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Cristy Ho
    • 1
  • Valerio Santangelo
    • 1
    • 2
  • Charles Spence
    • 1
  1. 1.Crossmodal Research Laboratory, Department of Experimental PsychologyUniversity of OxfordOxfordUK
  2. 2.Department of Human and Educational SciencesUniversity of PerugiaPerugiaItaly

Personalised recommendations