Advertisement

Cross-modal decoupling in temporal attention between audition and touch

  • Stefanie Mühlberg
  • Salvador Soto-Faraco
Original Article

Abstract

Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting.

Notes

Acknowledgements

This research was funded by the Spanish Ministry of Science and Innovation (PSI2016-75558-P(AEI/FEDER)), the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145) to S.S.F.

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. Aufschnaiter, S., Kiesel, A., & Thomaschke, R. (2017). Transfer of time-based task expectancy across different timing environments. Psychological Research Psychologische Forschung.  https://doi.org/10.1007/s00426-017-0895-1.PubMedGoogle Scholar
  2. Ball, F., Michels, L. E., Thiele, C., & Noesselt, T. (2018). The role of multisensory interplay in enabling temporal expectations. Cognition, 170, 130–146.  https://doi.org/10.1016/j.cognition.2017.09.015.CrossRefPubMedGoogle Scholar
  3. Bauer, M., Kennett, S., & Driver, J. (2012). Attentional selection of location and modality in vision and touch modulates low-frequency activity in associated sensory cortices. Journal of Neurophysiology, 107(9), 2342–2351.  https://doi.org/10.1152/jn.00973.2011.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bendixen, A., SanMiguel, I., & Schröger, E. (2012). Early electrophysiological indicators for predictive processing in audition: A review. International Journal of Psychophysiology, 83(2), 120–131.  https://doi.org/10.1016/j.ijpsycho.2011.08.003.CrossRefPubMedGoogle Scholar
  5. Correa, A., Lupiáñez, J., Milliken, B., & Tudela, P. (2004). Endogenous temporal orienting of attention in detection and discrimination tasks. Perception & Psychophysics, 66(2), 264–278.  https://doi.org/10.3758/BF03194878.CrossRefGoogle Scholar
  6. Correa, A., Lupiáñez, J., & Tudela, P. (2005). Attentional preparation based on temporal expectancy modulates processing at the perceptual level. Psychonomic Bulletin & Review, 12(2), 328–334.  https://doi.org/10.3758/BF03196380.CrossRefGoogle Scholar
  7. Correa, A., & Nobre, A. C. (2008). Spatial and temporal acuity of visual perception can be enhanced selectively by attentional set. Experimental Brain Research, 189(3), 339–344.  https://doi.org/10.1007/s00221-008-1429-2.CrossRefPubMedGoogle Scholar
  8. Coull, J. T., & Nobre, A. C. (1998). Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. The Journal of Neuroscience, 18(18), 7426–7435. http://www.ncbi.nlm.nih.gov/pubmed/9736662.
  9. Cravo, A. M., Rohenkohl, G., Wyart, V., & Nobre, A. C. (2013). Temporal expectation enhances contrast sensitivity by phase entrainment of low-frequency oscillations in visual cortex. The Journal of Neuroscience, 33(9), 4002–4010.  https://doi.org/10.1523/JNEUROSCI.4675-12.2013.CrossRefPubMedPubMedCentralGoogle Scholar
  10. Davranche, K., Nazarian, B., Vidal, F., & Coull, J. T. (2011). Orienting attention in time activates left intraparietal sulcus for both perceptual and motor task goals. Journal of Cognitive Neuroscience, 23(11), 3318–3330.  https://doi.org/10.1162/jocn_a_00030.CrossRefPubMedGoogle Scholar
  11. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Reviews of Neuroscience, 18, 193–222. http://www.annualreviews.org/doi/pdf/10.1146/annurev.ne.18.030195.001205.Google Scholar
  12. Driver, J., & Spence, C. (1994). Spatial synergies between auditory and visual attention. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 311–331). Cambridge: MIT Press.Google Scholar
  13. Driver, J., & Spence, C. (1998a). Attention and the crossmodal construction of space. Trends in Cognitive Sciences, 2(7), 254–262.  https://doi.org/10.1016/S1364-6613(98)01188-7.CrossRefPubMedGoogle Scholar
  14. Driver, J., & Spence, C. (1998b). Cross-modal links in spatial attention. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 353(1373), 1319–1331.  https://doi.org/10.1098/rstb.1998.0286.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Eimer, M. (1999). Can attention be directed to opposite locations in different modalities? An ERP study. Clinical Neurophysiology, 110(7), 1252–1259. http://www.ncbi.nlm.nih.gov/pubmed/10423190.
  16. Eimer, M., & Driver, J. (2000). An event-related brain potential study of cross-modal links in spatial attention between vision and touch. Psychophysiology, 37(5), 697–705. http://www.ncbi.nlm.nih.gov/pubmed/11037046.
  17. Eimer, M., van Velzen, J., & Driver, J. (2002). Cross-modal interactions between audition, touch, and vision in endogenous spatial attention: ERP evidence on preparatory states and sensory modulations. Journal of Cognitive Neuroscience, 14(2), 254–271.  https://doi.org/10.1162/089892902317236885.CrossRefPubMedGoogle Scholar
  18. Griffin, I. C., Miniussi, C., & Nobre, A. C. (2001). Orienting Attention in Time. Frontiers in Bioscience, 6, 660–671.CrossRefGoogle Scholar
  19. Jensen, O., & Mazaheri, A. (2010). Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Frontiers in Human Neuroscience, 4, 186.  https://doi.org/10.3389/fnhum.2010.00186.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Keil, J., Pomper, U., & Feuerbach, N. (2017). Temporal orienting precedes intersensory attention and has opposing effects on early evoked brain activity. NeuroImage, 148, 230–239.  https://doi.org/10.1016/j.neuroimage.2017.01.039.CrossRefPubMedGoogle Scholar
  21. Keil, J., Pomper, U., & Senkowski, D. (2016). Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction. Cortex, 74, 277–288.  https://doi.org/10.1016/j.cortex.2015.10.023.CrossRefPubMedGoogle Scholar
  22. Kennett, S., Eimer, M., Spence, C., & Driver, J. (2001). Tactile-visual links in exogenous spatial attention under different postures: convergent evidence from psychophysics and ERPs. Journal of Cognitive Neuroscience, 13(4), 462–478. http://www.ncbi.nlm.nih.gov/pubmed/11388920.
  23. Klein, R. M. (1977). Attention and visual dominance: A chronometric analysis. Journal of Experimental Psychology: Human Perception and Performance, 3(3), 365–378.PubMedGoogle Scholar
  24. Kok, P., Jehee, J. F. M., & de Lange, F. P. (2012). Less is more: expectation sharpens representations in the primary visual cortex. Neuron, 75(2), 265–270.  https://doi.org/10.1016/j.neuron.2012.04.034.CrossRefPubMedGoogle Scholar
  25. Kok, P., Rahnev, D., Jehee, J. F. M., Lau, H. C., & de Lange, F. P. (2012). Attention reverses the effect of prediction in silencing sensory signals. Cerebral Cortex, 22(9), 2197–2206.  https://doi.org/10.1093/cercor/bhr310.CrossRefPubMedGoogle Scholar
  26. Kubovy, M. (1988). Should we resist the seductiveness of the space is to time as vision is to audition analogy? Journal of Experimental Psychology: Human Perception and Performance, 14(2), 318–320.Google Scholar
  27. Lampar, A., & Lange, K. (2011). Effects of temporal trial-by-trial cuing on early and late stages of auditory processing: Evidence from event-related potentials. Attention, Perception & Psychophysics, 73, 1916–1933.  https://doi.org/10.3758/s13414-011-0149-z.CrossRefGoogle Scholar
  28. Lange, K. (2009). Brain correlates of early auditory processing are attenuated by expectations for time and pitch. Brain and Cognition, 69(1), 127–137.  https://doi.org/10.1016/j.bandc.2008.06.004.CrossRefPubMedGoogle Scholar
  29. Lange, K. (2012). The N1 effect of temporal attention is independent of sound location and intensity: implications for possible mechanisms of temporal attention. Psychophysiology, 49(11), 1468–1480.  https://doi.org/10.1111/j.1469-8986.2012.01460.x.CrossRefPubMedGoogle Scholar
  30. Lange, K. (2013). The ups and downs of temporal orienting: a review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction. Frontiers in Human Neuroscience, 7(June), 263.  https://doi.org/10.3389/fnhum.2013.00263.PubMedPubMedCentralGoogle Scholar
  31. Lange, K., Krämer, U. M., & Röder, B. (2006). Attending points in time and space. Experimental Brain Research, 173(1), 130–140.  https://doi.org/10.1007/s00221-006-0372-3.CrossRefPubMedGoogle Scholar
  32. Lange, K., & Röder, B. (2006). Orienting attention to points in time improves stimulus processing both within and across modalities. Journal of Cognitive Neuroscience, 18(5), 715–729.  https://doi.org/10.1162/jocn.2006.18.5.715.CrossRefPubMedGoogle Scholar
  33. Lange, K., Rösler, F., & Röder, B. (2003). Early processing stages are modulated when auditory stimuli are presented at an attended moment in time: an event-related potential study. Psychophysiology, 40(5), 806–817. http://www.ncbi.nlm.nih.gov/pubmed/14696734.
  34. Li, C., Chen, K., Han, H., Chui, D., & Wu, J. (2012). An FMRI study of the neural systems involved in visually cued auditory top-down spatial and temporal attention. PloS One, 7(11), e49948.  https://doi.org/10.1371/journal.pone.0049948.CrossRefPubMedPubMedCentralGoogle Scholar
  35. Liu, Y., Bengson, J., Huang, H., Mangun, G. R., & Ding, M. (2014). Top-down Modulation of Neural Activity in Anticipatory Visual Attention: Control Mechanisms Revealed by Simultaneous EEG-fMRI. Cerebral Cortex (New York, N.Y.: 1991), (February), pp. 517–529. https://doi.org/10.1093/cercor/bhu204.Google Scholar
  36. Lloyd, D. M., Merat, N., McGlone, F., & Spence, C. (2003). Crossmodal links between audition and touch in covert endogenous spatial attention. Perception & Psychophysics, 65(6), 901–924.CrossRefGoogle Scholar
  37. Los, S. A., Knol, D. L., & Boers, R. M. (2001). The foreperiod effect revisited: Conditioning as a basis for nonspecific preparation. Acta Psychologica, 106(1–2), 121–145.  https://doi.org/10.1016/S0001-6918(00)00029-9.CrossRefPubMedGoogle Scholar
  38. Macaluso, E. (2010). Orienting of spatial attention and the interplay between the senses. Cortex, 46(3), 282–297.  https://doi.org/10.1016/j.cortex.2009.05.010.CrossRefPubMedGoogle Scholar
  39. Macaluso, E., & Driver, J. (2005). Multisensory spatial interactions: a window onto functional integration in the human brain. Trends in Neurosciences, 28(5), 264–271.  https://doi.org/10.1016/j.tins.2005.03.008.CrossRefPubMedGoogle Scholar
  40. Macaluso, E., Frith, C. D., & Driver, J. (2000). Modulation of Human Visual Cortex by Crossmodal Spatial Attention. Science, 289(5482), 1206–1208.  https://doi.org/10.1126/science.289.5482.1206.CrossRefPubMedGoogle Scholar
  41. Miniussi, C., Wilding, E. L., Coull, J. T., & Nobre, A. C. (1999). Orienting attention in time. Modulation of brain potentials. Brain, 122, 1507–1518. http://www.ncbi.nlm.nih.gov/pubmed/10430834.
  42. Mühlberg, S., Oriolo, G., & Soto-Faraco, S. (2014). Cross-modal decoupling in temporal attention. European Journal of Neuroscience, 39, 2089–2097.  https://doi.org/10.1111/ejn.12563.CrossRefPubMedGoogle Scholar
  43. Nobre, A. C., & Rohenkohl, G. (2014). Time for the fourth dimension of attention. In A. C. Nobre & S. Kastner (Eds.), The Oxford Handbook of Attention (pp. 676–724). Oxford: Oxford University Press.  https://doi.org/10.1093/oxfordhb/9780199675111.013.036.CrossRefGoogle Scholar
  44. Pomper, U., Keil, J., Foxe, J. J., & Senkowski, D. (2015). Intersensory selective attention and temporal orienting operate in parallel and are instantiated in spatially distinct sensory and motor cortices. Human Brain Mapping, 3259, 3246–3259.  https://doi.org/10.1002/hbm.22845.CrossRefGoogle Scholar
  45. Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual dominance: An information-processing account of its origins and significance. Psychological Review, 83(2), 157–171.CrossRefPubMedGoogle Scholar
  46. Rohenkohl, G., Cravo, A. M., Wyart, V., & Nobre, A. C. (2012). Temporal expectation improves the quality of sensory information. The Journal of Neuroscience, 32(24), 8424–8428.  https://doi.org/10.1523/JNEUROSCI.0804-12.2012.CrossRefPubMedPubMedCentralGoogle Scholar
  47. Ruzzoli, M., & Soto-Faraco, S. (2014). Alpha stimulation of the human parietal cortex attunes tactile perception to external space. Current Biology, 24(3), 329–332.  https://doi.org/10.1016/j.cub.2013.12.029.CrossRefPubMedGoogle Scholar
  48. Sambo, C. F., & Forster, B. (2011). When far is near: ERP correlates of crossmodal spatial interactions between tactile and mirror-reflected visual stimuli. Neuroscience Letters, 500(1), 10–15.  https://doi.org/10.1016/j.neulet.2011.05.233.CrossRefPubMedGoogle Scholar
  49. Sanders, L. D., & Astheimer, L. B. (2008). Temporally selective attention modulates early perceptual processing: Event-related potential evidence. Perception & Psychophysics, 70(4), 732–742.  https://doi.org/10.3758/PP.70.4.732.CrossRefGoogle Scholar
  50. Santangelo, V., Fagioli, S., & Macaluso, E. (2010). The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space. NeuroImage, 49(3), 2717–2727.  https://doi.org/10.1016/j.neuroimage.2009.10.061.CrossRefPubMedGoogle Scholar
  51. Schroeder, C. E., & Lakatos, P. (2009). Low-frequency neuronal oscillations as instruments of sensory selection. Trends in Neurosciences, 32(1), 9–18.  https://doi.org/10.1016/j.tins.2008.09.012.CrossRefPubMedGoogle Scholar
  52. Schröter, H., Birngruber, T., Bratzke, D., Miller, J., & Ulrich, R. (2015). Task predictability influences the variable foreperiod effect: evidence of task-specific temporal preparation. Psychological Research Psychologische Forschung, 79(2), 230–237.  https://doi.org/10.1007/s00426-014-0550-z.CrossRefPubMedGoogle Scholar
  53. Schwartze, M., Farrugia, N., & Kotz, S. A. (2013). Dissociation of formal and temporal predictability in early auditory evoked potentials. Neuropsychologia, 51(2), 320–325.  https://doi.org/10.1016/j.neuropsychologia.2012.09.037.CrossRefPubMedGoogle Scholar
  54. Soto-Faraco, S., Morein-Zamir, S., & Kingstone, A. (2005). On audiovisual spatial synergy: the fragility of the phenomenon. Perception & Psychophysicssychophysics, 67(3), 444–457. http://www.ncbi.nlm.nih.gov/pubmed/16119393.
  55. Spence, C., & Driver, J. (1996). Audiovisual Links in Endogenous Covert Spatial Attention. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 1005–1031.PubMedGoogle Scholar
  56. Spence, C., & Driver, J. (1997). On measuring selective attention to an expected sensory modality. Perception & Psychophysics, 59(3), 389–403. http://www.ncbi.nlm.nih.gov/pubmed/9136269.
  57. Spence, C., Nicholls, M. E. R., & Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63(2), 330–336. http://www.ncbi.nlm.nih.gov/pubmed/11281107.
  58. Spence, C., Pavani, F., & Driver, J. (2000). Crossmodal Links between Vision and Touch in Covert Endogenous Spatial Attention. Journal of Experimental Psychology: Human Perception and Performance, 26(4), 1298–1319.PubMedGoogle Scholar
  59. Spitzer, H., Desimone, R., & Moran, J. (1988). Increased Attention Enhances Both Behavioral and Neuronal Performance. Science, 240(4850), 338–340.CrossRefPubMedGoogle Scholar
  60. Störmer, V. S., McDonald, J. J., & Hillyard, S. a. (2009). Cross-modal cueing of attention alters appearance and early cortical processing of visual stimuli. Proceedings of the National Academy of Sciences of the United States of America, 106(52), 22456–22461.  https://doi.org/10.1073/pnas.0907573106.CrossRefPubMedPubMedCentralGoogle Scholar
  61. Summerfield, C., & Egner, T. (2009). Expectation (and attention) in visual cognition. Trends in Cognitive Sciences, 13(9), 403–409.  https://doi.org/10.1016/j.tics.2009.06.003.CrossRefPubMedGoogle Scholar
  62. Tang, X., Wu, J., & Shen, Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208–224.  https://doi.org/10.1016/j.neubiorev.2015.11.002.CrossRefGoogle Scholar
  63. Teder-Sälejärvi, W. A., Münte, T. F., Sperlich, F., & Hillyard, S. A. (1999). Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. Brain Research. Cognitive Brain Research, 8(3), 327–343.CrossRefPubMedGoogle Scholar
  64. ten Oever, S., Romei, V., van Atteveldt, N., Soto-Faraco, S., Murray, M. M., & Matusz, P. J. (2016). The COGs (context, object, and goals) in multisensory processing. Experimental Brain Research, 234(5), 1307–1323.  https://doi.org/10.1007/s00221-016-4590-z.CrossRefPubMedGoogle Scholar
  65. Thomaschke, R., & Dreisbach, G. (2015). The time-event correlation effect is due to temporal expectancy, not to partial transition costs. Journal of Experimental Psychology: Human Perception and Performance, 41(1), 196–218.  https://doi.org/10.1037/a0038328.PubMedGoogle Scholar
  66. Thomaschke, R., Kunchulia, M., & Dreisbach, G. (2015). Time-based event expectations employ relative, not absolute, representations of time. Psychonomic Bulletin & Review, 22, 890–895.  https://doi.org/10.3758/s13423-014-0710-6.CrossRefGoogle Scholar
  67. Thomaschke, R., Wagener, A., Kiesel, A., & Hoffmann, J. (2011). The scope and precision of specific temporal expectancy: evidence from a variable foreperiod paradigm. Attention, Perception & Psychophysics, 73, 953–964.  https://doi.org/10.3758/s13414-010-0079-1.CrossRefGoogle Scholar
  68. Todorovic, A., Schoffelen, J. M., Van Ede, F., Maris, E., & De Lange, F. P. (2015). Temporal expectation and attention jointly modulate auditory oscillatory activity in the beta band. PLoS One, 10(3), 1–16.  https://doi.org/10.1371/journal.pone.0120288.CrossRefGoogle Scholar
  69. Trenner, M. U., Heekeren, H. R., Bauer, M., Rössner, K., Wenzel, R., Villringer, A., & Fahle, M. (2008). What happens in between? Human oscillatory brain activity related to crossmodal spatial cueing. PloS One, 3(1), e1467.  https://doi.org/10.1371/journal.pone.0001467.CrossRefPubMedPubMedCentralGoogle Scholar
  70. Treue, S. (2003). Visual attention: the where, what, how and why of saliency. Current Opinion in Neurobiology, 13(4), 428–432.  https://doi.org/10.1016/S0959-4388(03)00105-3.CrossRefPubMedGoogle Scholar
  71. van Ede, F., de Lange, F., Jensen, O., & Maris, E. (2011). Orienting attention to an upcoming tactile event involves a spatially and temporally specific modulation of sensorimotor alpha- and beta-band oscillations. Journal of Neuroscience, 31(6), 2016–2024.  https://doi.org/10.1523/JNEUROSCI.5630-10.2011.CrossRefPubMedGoogle Scholar
  72. VanRullen, R., Zoefel, B., & Ilhan, B. (2014). On the cyclic nature of perception in vision versus audition. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369, 20130214.  https://doi.org/10.1098/rstb.2013.0214.CrossRefPubMedPubMedCentralGoogle Scholar
  73. Volberg, G., & Thomaschke, R. (2017). Time-based expectations entail preparatory motor activity. Cortex, 92, 261–270.  https://doi.org/10.1016/j.cortex.2017.04.019.CrossRefPubMedGoogle Scholar
  74. Wagener, A., & Hoffmann, J. (2010). Temporal cueing of target-identity and target-location. Experimental Psychology, 57(6), 436–445.  https://doi.org/10.1027/1618-3169/a000054.CrossRefPubMedGoogle Scholar
  75. Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88(3), 638–687.  https://doi.org/10.1037/0033-2909.88.3.638.CrossRefPubMedGoogle Scholar
  76. Yang, Z., & Mayer, A. R. (2014). An event-related FMRI study of exogenous orienting across vision and audition. Human Brain Mapping, 35(3), 964–974.  https://doi.org/10.1002/hbm.22227.CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Center of Brain and CognitionUniversitat Pompeu FabraBarcelonaSpain
  2. 2.ICREA, Institució Catalana de Recerca i Estudis AvançatsBarcelonaSpain
  3. 3.Departament de Tecnologies de la Informació i les ComunicacionsUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations