Abstract
Selective attention refers to the ability to focus on goal-relevant information while filtering out irrelevant information. In a multisensory context, how do people selectively attend to multiple inputs when making categorical decisions? Here, we examined the role of selective attention in cross-modal categorization in two experiments. In a speed categorization task, participants were asked to attend to visual or auditory targets and categorize them while ignoring other irrelevant stimuli. A response-time extended multinomial processing tree (RT-MPT) model was implemented to estimate the contribution of attentional focusing on task-relevant information and attentional filtering of distractors. The results indicated that the role of selective attention was modality-specific, with differences found in attentional focusing and filtering between visual and auditory modalities. Visual information could be focused on or filtered out more effectively, whereas auditory information was more difficult to filter out, causing greater interference with task-relevant performance. The findings suggest that selective attention plays a critical and differential role across modalities, which provides a novel and promising approach to understanding multisensory processing and attentional focusing and filtering mechanisms of categorical decision-making.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Deidentified data, results, and analysis scripts are available on the OSF repository (https://osf.io/m4sy3/). Further inquiries can be directed to the first author Jianhua Li (yb97301@um.edu.mo) and the corresponding author Sophia Deng (wdeng@um.edu.mo).
References
Batchelder, W. H., & Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6(1), 57–86. https://doi.org/10.3758/bf03210812
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Beaman, C. P. (2005). Auditory distraction from low-intensity noise: A review of the consequences for learning and workplace environments. Applied Cognitive Psychology, 19(8), 1041–1064. https://doi.org/10.1002/acp.1134
Best, C. A., Yim, H., & Sloutsky, V. M. (2013). The cost of selective attention in category learning: Developmental differences between adults and infants. Journal of Experimental Child Psychology, 116(2), 105–119. https://doi.org/10.1016/j.jecp.2013.05.002
Broadbent, H. J., White, H., Mareschal, D., & Kirkham, N. Z. (2018). Incidental learning in a multisensory environment across childhood. Development Science, 21(2), e12554. https://doi.org/10.1111/desc.12554
Broadbent, H. J., Osborne, T., Mareschal, D., & Kirkham, N. Z. (2019). Withstanding the test of time: Multisensory cues improve the delayed retention of incidental learning. Development Science, 22(1), e12726. https://doi.org/10.1111/desc.12726
Broadbent, H. J., Osborne, T., Mareschal, D., & Kirkham, N. Z. (2020). Are two cues always better than one? The role of multiple intra-sensory cues compared to multi-cross-sensory cues in children’s incidental category learning. Cognition, 199, 104202. https://doi.org/10.1016/j.cognition.2020.104202
Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1(1), 9. https://doi.org/10.5334/joc.10
Butler, A. J., James, T. W., & James, K. H. (2011). Enhanced multisensory integration and motor reactivation after active motor learning of audiovisual associations. Journal of Cognitive Neuroscience, 23(11), 3515–3528. https://doi.org/10.1162/jocn_a_00015
Darby, K. P., Deng, S. W., Walther, D. B., & Sloutsky, V. M. (2021). The development of attention to objects and scenes: From object-biased to unbiased. Child Development, 92(3), 1173–1186. https://doi.org/10.1111/cdev.13469
Darwin, C. J., Turvey, M. T., & Crowder, R. G. (1972). An auditory analogue of the sperling [sic] partial report procedure: Evidence for brief auditory storage. Cognitive Psychology, 3(2), 255–267. https://doi.org/10.1016/0010-0285(72)90007-2
Deng, W. S., & Sloutsky, V. M. (2015). The development of categorization: Effects of classification and inference training on category representation. Devlopmental Psychology, 51(3), 392–405. https://doi.org/10.1037/a0038749
Deng, W. S., & Sloutsky, V. M. (2016). Selective attention, diffused attention, and the development of categorization. Cognitive Psychology, 91, 24–62. https://doi.org/10.1016/j.cogpsych.2016.09.002
Diaconescu, A. O., Alain, C., & McIntosh, A. R. (2011). The co-occurrence of multisensory facilitation and cross-modal conflict in the human brain. Journal of Neurophysiology, 106(6), 2896–2909. https://doi.org/10.1152/jn.00303.2011
Elliott, E. M., Morey, C. C., Morey, R. D., Eaves, S. D., Shelton, J. T., & Lutfi-Proctor, D. A. (2014). The role of modality: Auditory and visual distractors in stroop interference. Journal of Cognitive Psychology, 26(1), 15–26. https://doi.org/10.1080/20445911.2013.859133
Eördegh, G., Őze, A., Bodosi, B., Puszta, A., Pertich, Á., Rosu, A., ..., Nagy, A. (2019). Multisensory guided associative learning in healthy humans. PLOS ONE, 14(3), e0213094. https://doi.org/10.1371/journal.pone.0213094
Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149. https://doi.org/10.3758/BF03203267
Evans, K. K. (2020). The role of selective attention in cross-modal interactions between auditory and visual features. Cognition, 196, 104119. https://doi.org/10.1016/j.cognition.2019.104119
Franzen, L., Delis, I., De Sousa, G., Kayser, C., & Philiastides, M. G. (2020). Auditory information enhances post-sensory visual evidence during rapid multisensory decision-making. Nature Communications, 11(1), 1–14. https://doi.org/10.1038/s41467-020-19306-7
Geng, J. J., Won, B.-Y., & Carlisle, N. B. (2019). Distractor ignoring: Strategies, learning, and passive filtering. Current Directions in Psychological Science, 28(6), 600–606. https://doi.org/10.1177/0963721419867099
Griffiths, O., & Mitchell, C. J. (2008). Selective attention in human associative learning and recognition memory. Journal of Experimental Psychology: General, 137(4), 626–648. https://doi.org/10.1037/a0013685
Heck, D. W., & Erdfelder, E. (2016). Extending multinomial processing tree models to measure the relative speed of cognitive processes. Psychonomic Bulletin & Review, 23(5), 1440–1465. https://doi.org/10.3758/s13423-016-1025-6
Heck, D. W., & Erdfelder, E. (2017). Linking process and measurement models of recognition-based decisions. Psychological Review, 124(4), 442–471. https://doi.org/10.1037/rev0000063
Heck, D. W., & Erdfelder, E. (2020). Benefits of response time-extended multinomial processing tree models: A reply to Starns (2018). Psychonomic Bulletin & Review, 27(3), 571–580. https://doi.org/10.3758/s13423-019-01663-0
Heck, D. W., Arnold, N. R., & Arnold, D. (2018). TreeBUGS: An R package for hierarchical multinomial-processing-tree modeling. Behavior Research Methods, 50(1), 264–284. https://doi.org/10.3758/s13428-017-0869-7
Hoffman, A. B., & Rehder, B. (2010). The costs of supervised classification: The effect of learning task on conceptual flexibility. Journal of Experimental Psychology: General, 139, 319–340. https://doi.org/10.1037/a0019042
Horn, S. S., Bayen, U. J., & Michalkiewicz, M. (2021). The development of clustering in episodic memory: A cognitive-modeling approach. Child Development, 92(1), 239–257. https://doi.org/10.1111/cdev.13407
Kim, R. S., Seitz, A. R., & Shams, L. (2008). Benefits of stimulus congruency for multisensory facilitation of visual learning. PLOS ONE, 3(1), e1532. https://doi.org/10.1371/journal.pone.0001532
Klauer, K. C., & Kellen, D. (2018). RT-MPTs: Process models for response-time distributions based on multinomial processing trees with applications to recognition memory. Journal of Mathematical Psychology, 82, 111–130. https://doi.org/10.1016/j.jmp.2017.12.003
Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2009). Competition between auditory and visual spatial cues during visual task performance. Experimental Brain Research, 195(4), 593–602. https://doi.org/10.1007/s00221-009-1829-y
Kubovy, M., & Van Valkenburg, D. (2001). Auditory and visual objects. Cognition, 80(1), 97–126. https://doi.org/10.1016/S0010-0277(00)00155-4
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(1), 1–26. https://doi.org/10.18637/jss.v082.i13
Laurienti, P. J., Kraft, R. A., Maldjian, J. A., Burdette, J. H., & Wallace, M. T. (2004). Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research, 158(4), 405–414. https://doi.org/10.1007/s00221-004-1913-2
Lenth, R. V. (2016). Least-squares means: the R Package lsmeans. Journal of Statistical Software, 69(1), 1–33. https://doi.org/10.18637/jss.v069.i01
Li, J., & Deng, S. W. (2023). Facilitation and interference effects of the multisensory context on learning: A systematic review and meta-analysis. Psychological Research, 87, 1334–1352. https://doi.org/10.1007/s00426-022-01733-4
Marcell, M. M., Borella, D., Greene, M., Kerr, E., & Rogers, S. (2000). Confrontation naming of environmental sounds. Journal of Clinical and Experimental Neuropsychology, 22(6), 830–864. https://doi.org/10.1076/jcen.22.6.830.949
Medina, V. A., Lutfi-Proctor, D. A., & Elliott, E. M. (2021). The role of joint influence on the cross-modal stroop effect: Investigating time course and asymmetry. Auditory Perception & Cognition, 4(3/4), 186–211. https://doi.org/10.1080/25742442.2022.2034394
Moshagen, M. (2010). multiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42(1), 42–54. https://doi.org/10.3758/BRM.42.1.42
Murphy, S., Fraenkel, N., & Dalton, P. (2013). Perceptual load does not modulate auditory distractor processing. Cognition, 129(2), 345–355. https://doi.org/10.1016/j.cognition.2013.07.014
Nava, E., & Pavani, F. (2013). Changes in sensory dominance during childhood: Converging evidence from the colavita effect and the sound-induced flash illusion. Child Development, 84(2), 604–616. https://doi.org/10.1111/j.1467-8624.2012.01856.x
Parker, J. L., & Robinson, C. W. (2018). Changes in multisensory integration across the life span. Psychology and Aging, 33(3), 545–558. https://doi.org/10.1037/pag0000244
Plebanek, D. J., & Sloutsky, V. M. (2017). Costs of selective attention: When children notice what adults miss. Psychological Science, 28(6), 723–732. https://doi.org/10.1177/0956797617693005
Plebanek, D. J., & Sloutsky, V. M. (2019). Selective attention, filtering, and the development of working memory. Developmental Science, 22(1), e12727. https://doi.org/10.1111/desc.12727
Riefer, D. M., & Batchelder, W. H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339. https://doi.org/10.1037/0033-295X.95.3.318
Robinson, C. W., & Sloutsky, V. M. (2004). Auditory dominance and its change in the course of development. Child Development, 75(5), 1387–1401. https://doi.org/10.1111/j.1467-8624.2004.00747.x
Robinson, C. W., & Sloutsky, V. M. (2010). Development of cross-modal processing. Wiley Interdisciplinary Reviews: Cognitive Science, 1(1), 135–141. https://doi.org/10.1002/wcs.12
Robinson, C. W., & Sloutsky, V. M. (2013). When audition dominates vision: Evidence from cross-modal statistical learning. Experimental Psychology, 60(2), 113–121. https://doi.org/10.1027/1618-3169/a000177
Robinson, C. W., & Sloutsky, V. M. (2019). Two mechanisms underlying auditory dominance: Overshadowing and response competition. Journal of Experimental Child Psychology, 178, 317–340. https://doi.org/10.1016/j.jecp.2018.10.001
Röer, J. P., Bell, R., Körner, U., & Buchner, A. (2018). Equivalent auditory distraction in children and adults. Journal of Experimental Child Psychology, 172, 41–58. https://doi.org/10.1016/j.jecp.2018.02.005
Rueda, M. R., Posner, M. I., & Rothbart, M. K. (2005). The development of executive attention: Contributions to the emergence of self-regulation. Developmental Neuropsychology, 28(2), 573–594. https://doi.org/10.1207/s15326942dn2802_2
Santangelo, V., Olivetti, B., & Spence, C. (2007). The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. Human Perception and Performance, 33(1), 137–148. https://doi.org/10.1037/0096-1523.33.1.137
Sarmiento, B. R., Shore, D. I., Milliken, B., & Sanabria, D. (2012). Audiovisual interactions depend on context of congruency. Attention, Perception, & Psychophysics, 74(3), 563–574. https://doi.org/10.3758/s13414-011-0249-9
Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning. Trends in Cognitive Sciences, 12(11), 411–417. https://doi.org/10.1016/j.tics.2008.07.006
Sinnett, S., Soto-Faraco, S., & Spence, C. (2008). The co-occurrence of multisensory competition and facilitation. Acta Psychologica, 128(1), 153–161. https://doi.org/10.1016/j.actpsy.2007.12.002
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14(9), 400–410. https://doi.org/10.1016/j.tics.2010.06.008
Unger, L., & Sloutsky, V. (2021). Category learning is shaped by the multifaceted development of selective attention. Proceedings of the Annual Meeting of the Cognitive Science Society, 43, 2052–2058.
Van der Burg, E., Olivers, C. N., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053. https://doi.org/10.1037/0096-1523.34.5.1053
van Moorselaar, D., & Slagter, H. A. (2020). Inhibition in selective attention. Annals of the New York Academy of Sciences, 1464(1), 204–221. https://doi.org/10.1111/nyas.14304
Yamauchi, T., & Markman, A. B. (1998). Category learning by inference and classification. Journal of Memory and Language, 39(1), 124–148. https://doi.org/10.1006/jmla.1998.2566
Yim, H., Dennis, S. J., & Sloutsky, V. M. (2013). The development of episodic memory: Items, contexts, and relations. Psychological Science, 24(11), 2163–2172. https://doi.org/10.1177/0956797613487385
Funding
This research was supported by a National Natural Science Foundation of China grant to S. W. Deng. (31900769) and by a Multi-Year Research Grant of the University of Macau to S. W. Deng. (MYRG2019-00007-FSS). The funding organizations had no role in the development of the study design or collection, analysis, and interpretation of the data.
Author information
Authors and Affiliations
Contributions
Conceptualization: J. Li and S. W. Deng; Methodology: J. Li and S. W. Deng; Formal Analysis: J. Li and S. W. Deng; Investigation: J. Li; Writing—Original Draft Preparation: J. Li and S. W. Deng; Writing—Review & Editing: J. Li and S. W. Deng; Supervision: S. W. Deng; Project Administration: J. Li and S. W. Deng; Funding Acquisition: S. W. Deng.
Corresponding author
Ethics declarations
Conflicts of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Ethics approval
All procedures were approved by the Research Ethics Panel of the University of Macau.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, J., Deng, S.W. Attentional focusing and filtering in multisensory categorization. Psychon Bull Rev 31, 708–720 (2024). https://doi.org/10.3758/s13423-023-02370-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13423-023-02370-7