Skip to main content
SpringerLink
Log in

Multisensory warning signals: when spatial correspondence matters

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

We report a study designed to investigate the effectiveness of task-irrelevant unimodal and bimodal audiotactile stimuli in capturing a person’s spatial attention away from a highly perceptually demanding central rapid serial visual presentation (RSVP) task. In “Experiment 1”, participants made speeded elevation discrimination responses to peripheral visual targets following the presentation of auditory stimuli that were either presented alone or else were paired with centrally presented tactile stimuli. The results showed that the unimodal auditory stimuli only captured spatial attention when participants were not performing the RSVP task, while the bimodal audiotactile stimuli did not result in any performance change in any of the conditions. In “Experiment 2”, spatial auditory stimuli were either presented alone or else were paired with a tactile stimulus presented from the same direction. In contrast to the results of “Experiment 1”, the bimodal audiotactile stimuli were especially effective in capturing participants’ spatial attention from the concurrent RSVP task. These results therefore provide support for the claim that auditory and tactile stimuli should be presented from the same direction if they are to capture attention effectively. Implications for multisensory warning signal design are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile “capture” of audition. Percept Psychophys 64:616–630

    PubMed  Google Scholar 

  • Calvert GA, Spence C, Stein BE (eds) (2004) The handbook of multisensory processes. MIT Press, Cambridge, MA

    Google Scholar 

  • Chan AHS, Chan KWL (2006) Synchronous and asynchronous presentation of auditory and visual signals: implications for control console design. Appl Ergon 37:131–140

    Article  PubMed  Google Scholar 

  • Cohen J (1973) Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educ Psychol Measur 33:107–112

    Article  Google Scholar 

  • Collins CC (1970) Tactile television—mechanical and electrical image projection. IEEE Trans Man Mach Syst MMS-11:65–71

    Article  Google Scholar 

  • Ferris T, Penfold R, Hameed S, Sarter N (2006) The implications of crossmodal links in attention for the design of multimodal interfaces: a driving simulator study. In: Proceedings of the Human Factors and Ergonomics Society 50th annual meeting, pp 406–409

  • Fitch GM, Kiefer RJ, Hankey JM, Kleiner BM (2007) Toward developing an approach for alerting drivers to the direction of a crash threat. Hum Factors 49:710–720

    Article  PubMed  Google Scholar 

  • Hennessy JR (1966) Cutaneous sensitivity communications. Hum Factors 8:463–469

    PubMed  CAS  Google Scholar 

  • Ho C, Spence C (2005) Assessing the effectiveness of various auditory cues in capturing a driver’s visual attention. J Exp Psychol Appl 11:157–174

    Article  PubMed  Google Scholar 

  • Ho C, Spence C (2007) Head orientation biases tactile localization. Brain Res 1144:136–141

    Article  PubMed  CAS  Google Scholar 

  • Ho C, Spence C (2008) The multisensory driver: implications for ergonomic car interface design. Ashgate, Aldershot

    Google Scholar 

  • Ho C, Reed N, Spence C (2007) Multisensory in-car warning signals for collision avoidance. Hum Factors 49:1107–1114

    Article  PubMed  Google Scholar 

  • Hötting K, Rösler F, Röder B (2003) Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Exp Brain Res 148:26–37

    Article  PubMed  Google Scholar 

  • Kitagawa N, Spence C (2006) Audiotactile multisensory interactions in human information processing. Jpn Psychol Res 48:158–173

    Article  Google Scholar 

  • Kitagawa N, Zampini M, Spence C (2005) Audiotactile interactions in near and far space. Exp Brain Res 166:528–537

    Article  PubMed  Google Scholar 

  • Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:405–414

    Google Scholar 

  • Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT (2006) Enhanced multisensory integration in older adults. Neurobiol Aging 27:1155–1163

    Article  Google Scholar 

  • Lavie N (2005) Distracted and confused? Selective attention under load. Trends Cogn Sci 9:75–82

    Article  PubMed  Google Scholar 

  • Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC et al (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963–974

    Article  PubMed  Google Scholar 

  • Oyer J, Hardick J (1963) Response of population to optimum warning signal (Report No. SHSLR163). Office of Civil Defence, Washington, DC

    Google Scholar 

  • Poliakoff E, Miles E, Li X, Blanchette I (2007) The effect of visual threat on spatial attention to touch. Cognition 102:405–414

    Article  PubMed  Google Scholar 

  • Santangelo V, Spence C (2007a) Assessing the automaticity of reflexive tactile attentional orienting. Perception 36:1497–1505

    Article  PubMed  Google Scholar 

  • Santangelo V, Spence C (2007b) Multisensory cues capture spatial attention regardless of perceptual load. J Exp Psychol Hum Percept Perform 33:1311–1321

    Article  PubMed  Google Scholar 

  • Santangelo V, Spence C (2008) Is the exogenous orienting of spatial attention truly automatic? Evidence from unimodal and multisensory studies. Conscious Cogn 17:989–1015

    Article  PubMed  Google Scholar 

  • Santangelo V, Van der Lubbe RHJ, Berlardinelli MO, Postma A (2006) Spatial attention triggered by unimodal, crossmodal, and bimodal exogenous cues: a comparison of reflexive orienting mechanisms. Exp Brain Res 173:40–48

    Article  PubMed  Google Scholar 

  • Santangelo V, Finoia P, Raffone A, Olivetti Belardinelli M, Spence C (2007a) Perceptual load affects exogenous spatial orienting while working memory load does not. Exp Brain Res 184:371–382

    Article  PubMed  Google Scholar 

  • Santangelo V, Olivetti Belardinelli M, Spence C (2007b) The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. J Exp Psychol Hum Percept Perform 33:137–148

    Article  PubMed  Google Scholar 

  • Santangelo V, Ho C, Spence C (2008) Capturing spatial attention with multisensory cues. Psychon Bull Rev 15:398–403

    Article  PubMed  Google Scholar 

  • Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327

    PubMed  CAS  Google Scholar 

  • Schürmann M, Caetano G, Jousmäki V, Hari R (2004) Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. J Acoust Soc Am 115:830–832

    Article  PubMed  Google Scholar 

  • Selcon SJ, Taylor RM, McKenna FP (1995) Integrating multiple information sources: using redundancy in the design of warnings. Ergonomics 38:2362–2370

    Article  Google Scholar 

  • Spence C, Driver J (1999) A new approach to the design of multimodal warning signals. In: Harris D (ed) Engineering psychology and cognitive ergonomics. Job design, product design and human-computer interaction, vol 4. Ashgate, Aldershot, pp 455–461

  • Spence C, Driver J (2000) Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism. Neuroreport 11:2057–2061

    Article  PubMed  CAS  Google Scholar 

  • Spence C, Driver J (eds) (2004) Crossmodal space and crossmodal attention. Oxford University Press, Oxford

    Google Scholar 

  • Spence C, Ho C (2008a) Multisensory warning signals for event perception and safe driving. Theor Issues Ergon Sci 9:523–554

    Article  Google Scholar 

  • Spence C, Ho C (2008b) Tactile & multisensory interface design for drivers. IEEE Trans Haptics 1:121–129

    Article  Google Scholar 

  • Spence C, Nicholls MER, Gillespie N, Driver J (1998) Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percept Psychophys 60:544–557

    PubMed  CAS  Google Scholar 

  • Spence C, McDonald J, Driver J (2004) Exogenous spatial cuing studies of human crossmodal attention and multisensory integration. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, pp 277–320

    Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, MA

    Google Scholar 

  • Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9:255–267

    Article  PubMed  CAS  Google Scholar 

  • Stein BE, Stanford TR, Wallace MT, Vaughan WJ, Jiang W (2004) Crossmodal spatial interactions in subcortical and cortical circuits. In: Spence C, Driver J (eds) Crossmodal space and crossmodal attention. Oxford University Press, Oxford, UK, pp 25–50

    Google Scholar 

  • Tajadura-Jiménez A, Kitagawa N, Väljamäe A, Zampini M, Murray MM, Spence C (2009) Auditory-somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia 47:195–203

    Article  PubMed  Google Scholar 

  • van Erp JBF, van Veen HAHC (2004) Vibrotactile in-vehicle navigation system. Transp Res F Traffic Psychol Behav 7:247–256

    Article  Google Scholar 

  • Vroomen J, Bertelson P, de Gelder B (2001) Directing spatial attention towards the illusory location of a ventriloquized sound. Acta Psychol 108:21–33

    Article  CAS  Google Scholar 

  • Vu KL, Strybel TZ, Proctor RW (2006) Effects of displacement magnitude and direction of auditory cues on auditory spatial facilitation of visual search. Hum Factors 48:587–599

    Article  PubMed  Google Scholar 

  • Ward LM, McDonald JJ, Golestani N (1998) Cross-modal control of attention shifts. In: Wright RD (ed) Visual attention. Oxford University Press, New York, pp 232–268

    Google Scholar 

  • Zampini M, Brown T, Shore DI, Maravita A, Roder B, Spence C (2005) Audiotactile temporal order judgments. Acta Psychol 118:277–291

    Article  Google Scholar 

  • Zampini M, Torresan D, Spence C, Murray MM (2007) Auditory-somatosensory multisensory interactions in front and rear space. Neuropsychologia 45:1869–1877

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This study was supported by a Clarendon Fund Bursary from Oxford University, an Overseas Research Students (ORS) Award, and a Somerville Senior Scholarship from Somerville College, Oxford to C. H.

Author information

Authors and Affiliations

  1. Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK

    Cristy Ho, Valerio Santangelo & Charles Spence

  2. Department of Human and Educational Sciences, University of Perugia, Perugia, Italy

    Valerio Santangelo

Authors
  1. Cristy Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valerio Santangelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Charles Spence
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Spence.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ho, C., Santangelo, V. & Spence, C. Multisensory warning signals: when spatial correspondence matters. Exp Brain Res 195, 261–272 (2009). https://doi.org/10.1007/s00221-009-1778-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-009-1778-5

Keywords

Search

Navigation