Abstract
Can synchrony in stimulation guide attention and aid perceptual performance? Here, in a series of three experiments, we tested the influence of visual and auditory synchrony on attentional selection during a novel human foraging task. Human foraging tasks are a recent extension of the classic visual search paradigm in which multiple targets must be located on a given trial, making it possible to capture a wide range of performance metrics. Experiment 1 was performed online, where the task was to forage for 10 (out of 20) vertical lines among 60 randomly oriented distractor lines that changed color between yellow and blue at random intervals. The targets either changed colors in visual synchrony or not. In another condition, a non-spatial sound additionally occurred synchronously with the color change of the targets. Experiment 2 was run in the laboratory (within-subjects) with the same design. When the targets changed color in visual synchrony, foraging times were significantly shorter than when they randomly changed colors, but there was no additional benefit for the sound synchrony, in contrast to predictions from the so-called “pip-and-pop” effect (Van der Burg et al., Journal of Experimental Psychology, 1053-1065, 2008). In Experiment 3, task difficulty was increased as participants foraged for as many 45° rotated lines as possible among lines of different orientations within 10 s, with the same synchrony conditions as in Experiments 1 and 2. Again, there was a large benefit of visual synchrony but no additional benefit for sound synchronization. Our results provide strong evidence that visual synchronization can guide attention during multiple target foraging. This likely reflects the local grouping of the synchronized targets. Importantly, there was no additional benefit for sound synchrony, even when the foraging task was quite difficult (Experiment 3).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alais, D., Blake, R., & Lee, S.-H. (1998). Visual features that vary together over time group together over space. Nature Neuroscience, 1(2), 160–164. https://doi.org/10.1038/414
Arnold, D. H., Johnston, A., & Nishida, S. (2005). Timing sight and sound. Vision Research, 45(10), 1275–1284. https://doi.org/10.1016/j.visres.2004.11.014
Bella-Fernández, M., Suero Suñé, M., & Gil-Gómez De Liaño, B. (2023). The time course of visual foraging in the lifespan: Spatial scanning, organization search, and target processing. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-023-02345-8
Blake, R., & Lee, S.-H. (2005). The role of temporal structure in human vision. Behavioral and Cognitive Neuroscience Reviews, 4(1), 21–42. https://doi.org/10.1177/1534582305276839
Bond, A. B. (1983). Visual search and selection of natural stimuli in the pigeon: The attention threshold hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 9(3), 292–306.
Bond, A. B., & Kamil, A. C. (2006). Spatial heterogeneity, predator cognition, and the evolution of color polymorphism in virtual prey. Proceedings of the National Academy of Sciences, 103(9), 3214–3219. https://doi.org/10.1073/pnas.0509963103
Bridges, D., Pitiot, A., MacAskill, M. R., & Peirce, J. W. (2020). The timing mega-study: Comparing a range of experiment generators, both lab-based and online. PeerJ, 8, e9414. https://doi.org/10.7717/peerj.9414
Cavanagh, P., Labianca, A. T., & Thornton, I. M. (2001). Attention-based visual routines: Sprites. Cognition, 80(1–2), 47–60. https://doi.org/10.1016/S0010-0277(00)00153-0
Chetverikov, A., Campana, G., & Kristjánsson, Á. (2016). Building ensemble representations: How the shape of preceding distractor distributions affects visual search. Cognition, 153, 196–210. https://doi.org/10.1016/j.cognition.2016.04.018
Chetverikov, A., & Upravitelev, P. (2016). Online versus offline: The Web as a medium for response time data collection. Behavior Research Methods, 48(3), 1086–1099. https://doi.org/10.3758/s13428-015-0632-x
Collignon, O., Dormal, G., Albouy, G., Vandewalle, G., Voss, P., Phillips, C., & Lepore, F. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. Brain, 136(9), 2769–2783. https://doi.org/10.1093/brain/awt176
Dawkins, M. S. (1983). Battery hens name their price: Consumer demand theory and the measurement of ethological ‘needs.’ Animal Behaviour, 31(4), 1195–1205. https://doi.org/10.1016/S0003-3472(83)80026-8
Driver, J. (1996). Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature, 381(6577), 66–68. https://doi.org/10.1038/381066a0
Driver, J., & Spence, C. (1998). Attention and the crossmodal construction of space. Trends in Cognitive Sciences, 2(7), 254–262. https://doi.org/10.1016/S1364-6613(98)01188-7
Dukas, R., & Elner, S. (1993). Information processing and prey detection. Ecology, 74(5), 1337–1346. https://doi.org/10.2307/1940064
Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96(3), 433–458. https://doi.org/10.1037/0033-295x.96.3.433
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429–433. https://doi.org/10.1038/415429a
Fahle, M. (1993). Figure–ground discrimination from temporal information. Proceedings of the Royal Society of London. Series B: Biological Sciences, 254(1341), 199–203. https://doi.org/10.1098/rspb.1993.0146
Farid, H. (2002). Temporal synchrony in perceptual grouping: A critique. Trends in Cognitive Sciences, 6(7), 284–288. https://doi.org/10.1016/S1364-6613(02)01927-7
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
Fujisaki, W., Koene, A., Arnold, D., Johnston, A., & Nishida, S. (2006). Visual search for a target changing in synchrony with an auditory signal. Proceedings of the Royal Society B: Biological Sciences, 273(1588), 865–874. https://doi.org/10.1098/rspb.2005.3327
Fujisaki, W., & Nishida, S. (2007). Feature-based processing of audio-visual synchrony perception revealed by random pulse trains. Vision Research, 47(8), 1075–1093. https://doi.org/10.1016/j.visres.2007.01.021
Gao, M., Chang, R., Wang, A., Zhang, M., Cheng, Z., Li, Q., & Tang, X. (2021). Which can explain the pip-and-pop effect during a visual search: Multisensory integration or the oddball effect? Journal of Experimental Psychology: Human Perception and Performance, 47(5), 689–703. https://doi.org/10.1037/xhp0000905
Gil-Gómez De Liaño, B., & Wolfe, J. M. (2022). The FORAGEKID Game: Hybrid-Foraging as a new way to study aspects of executive function in development. Cognitive Development, 64, 101233. https://doi.org/10.1016/j.cogdev.2022.101233
Gilliam, J. F., & Fraser, D. F. (1987). Habitat selection under predation hazard: Test of a model with foraging minnows. Ecology, 68(6), 1856–1862. https://doi.org/10.2307/1939877
Gori, M., Mazzilli, G., Sandini, G., & Burr, D. (2011). Cross-sensory facilitation reveals neural interactions between visual and tactile motion in humans. Frontiers in Psychology, 2. https://doi.org/10.3389/fpsyg.2011.00055
Guttman, S. E., Gilroy, L. A., & Blake, R. (2007). Spatial grouping in human vision: Temporal structure trumps temporal synchrony. Vision Research, 47(2), 219–230. https://doi.org/10.1016/j.visres.2006.09.012
Ho, C., & Spence, C. (2005). Assessing the effectiveness of various auditory cues in capturing a driver’s visual attention. Journal of Experimental Psychology: Applied, 11(3), 157–174. https://doi.org/10.1037/1076-898X.11.3.157
Ho, C., Spence, C., & Gray, R. (2013). Looming auditory and vibrotactile collision warning for safe driving. Driving Assessment Conference, 7. https://doi.org/10.17077/drivingassessment.1540
Horowitz, T. S., & Thornton, I. M. (2008). Objects or locations in vision for action? Evidence from the MILO task. Visual Cognition, 16(4), 486–513. https://doi.org/10.1080/13506280601087356
Jeffreys, H. (1998). Theory of Probability. Clarendon Press. https://books.google.is/books?id=_PuRmAEACAAJ
Jóhannesson, Ó. I., Kristjánsson, Á., & Thornton, I. M. (2017). Are foraging patterns in humans related to working memory and inhibitory control?: Foraging patterns, working memory, and inhibitory control. Japanese Psychological Research, 59(2), 152–166. https://doi.org/10.1111/jpr.12152
Jóhannesson, Ó. I., Thornton, I. M., Smith, I. J., Chetverikov, A., & Kristjánsson, Á. (2016). Visual foraging with fingers and eye gaze. I-Perception, 7(2), 204166951663727. https://doi.org/10.1177/2041669516637279
Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14(2), 201–211. https://doi.org/10.3758/BF03212378
Kassambara A. (2021). rstatix: Pipe-Friendly Framework for Basic Statistical Tests [Computer software]. https://cran.r-project.org/web/packages/rstatix/index.html
Klapetek, A., Ngo, M. K., & Spence, C. (2012). Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? Attention, Perception, & Psychophysics, 74(6), 1154–1167. https://doi.org/10.3758/s13414-012-0317-9
Kong, G., Alais, D., & Van der Burg, E. (2017). Orientation categories used in guidance of attention in visual search can differ in strength. Attention, Perception, & Psychophysics, 79(8), 2246–2256. https://doi.org/10.3758/s13414-017-1387-5
Kristjánsson, Á., Björnsson, A. S., & Kristjánsson, T. (2020). Foraging with Anne Treisman: Features versus conjunctions, patch leaving and memory for foraged locations. Attention, Perception, & Psychophysics, 82(2), 818–831. https://doi.org/10.3758/s13414-019-01941-y
Kristjánsson, Á., & Draschkow, D. (2021). Keeping it real: Looking beyond capacity limits in visual cognition. Attention, Perception, & Psychophysics, 83(4), 1375–1390. https://doi.org/10.3758/s13414-021-02256-7
Kristjánsson, Á., & Egeth, H. (2020). How feature integration theory integrated cognitive psychology, neurophysiology, and psychophysics. Attention, Perception, & Psychophysics, 82(1), 7–23. https://doi.org/10.3758/s13414-019-01803-7
Kristjánsson, Á., Jóhannesson, Ó. I., & Thornton, I. M. (2014). Common attentional constraints in visual foraging. PLoS ONE, 9(6), e100752. https://doi.org/10.1371/journal.pone.0100752
Kristjánsson, Á., Ólafsdóttir, I. M., & Kristjánsson, T. (2019). Visual foraging tasks provide new insights into the orienting of visual attention: Methodological considerations. In S. Pollmann (Ed.), Spatial Learning and Attention Guidance (Vol. 151, pp. 3–21). Springer US. https://doi.org/10.1007/7657_2019_21
Kristjánsson, T., Draschkow, D., Pálsson, Á., Haraldsson, D., Jónsson, P. Ö., & Kristjánsson, Á. (2022). Moving foraging into three dimensions: Feature- versus conjunction-based foraging in virtual reality. Quarterly Journal of Experimental Psychology, 75(2), 313–327. https://doi.org/10.1177/1747021820937020
Kristjánsson, T., Páll Thorvaldsson, T., & Kristjánsson, Á. (2014). Divided multimodal attention: Sensory trace and context coding strategies in spatially congruent auditory and visual presentation. Multisensory Research, 27(2), 91–110. https://doi.org/10.1163/22134808-00002448
Kristjánsson, T., Thornton, I. M., & Kristjánsson, Á. (2018). Time limits during visual foraging reveal flexible working memory templates. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 827–835. https://doi.org/10.1037/xhp0000517
Lawrence MA. (2016). _ez: Easy Analysis and Visualization of Factorial Experiments_. R package version 4.4-0 [Computer software]. <https://CRAN.R-project.org/package=ez>
Lee, S.-H., & Blake, R. (1999). Visual form created solely from temporal structure. Science, 284(5417), 1165–1168. https://doi.org/10.1126/science.284.5417.1165
Leonards, U., Singer, W., & Fahle, M. (1996). The influence of temporal phase differences on texture segmentation. Vision Research, 36(17), 2689–2697. https://doi.org/10.1016/0042-6989(96)86829-5
Li, H., Bao, Y., Pöppel, E., & Su, Y.-H. (2014). A unique visual rhythm does not pop out. Cognitive Processing, 15(1), 93–97. https://doi.org/10.1007/s10339-013-0581-1
Mark, V. W., Woods, A. J., Ball, K. K., Roth, D. L., & Mennemeier, M. (2004). Disorganized search on cancellation is not a consequence of neglect. Neurology, 63(1), 78–84. https://doi.org/10.1212/01.WNL.0000131947.08670.D4
Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43(3), 679–690. https://doi.org/10.3758/s13428-010-0049-5
Ngo, M. K., & Spence, C. (2010). Crossmodal facilitation of masked visual target identification. Attention, Perception & Psychophysics, 72(7), 1938–1947. https://doi.org/10.3758/APP.72.7.1938
Ólafsdóttir, I. M., Gestsdóttir, S., & Kristjánsson, Á. (2020). Age differences in foraging and executive functions: A cross-sectional study. Journal of Experimental Child Psychology, 198, 104910. https://doi.org/10.1016/j.jecp.2020.104910
Ólafsdóttir, I. M., Gestsdóttir, S., & Kristjánsson, Á. (2021). The development of foraging organization. Attention, Perception, & Psychophysics, 83(7), 2891–2904. https://doi.org/10.3758/s13414-021-02328-8
Prpic, V., Kniestedt, I., Camilleri, E., Maureira, M. G., Kristjánsson, Á., & Thornton, I. M. (2019). A serious game to explore human foraging in a 3D environment. PLoS ONE, 14(7), e0219827. https://doi.org/10.1371/journal.pone.0219827
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111. https://doi.org/10.2307/271063
Reisberg, D. (1978). Looking where you listen: Visual cues and auditory attention. Acta Psychologica, 42(4), 331–341. https://doi.org/10.1016/0001-6918(78)90007-0
RStudio Team. (2020). RStudio: Integrated Development for R [Computer software]. http://www.rstudio.com/
Sternberg, S. (1969). The discovery of processing stages: Extensions of Donders’ method. Acta Psychologica, 30, 276–315. https://doi.org/10.1016/0001-6918(69)90055-9
Tagu, J., & Kristjánsson, Á. (2022). Dynamics of attentional and oculomotor orienting in visual foraging tasks. Quarterly Journal of Experimental Psychology, 75(2), 260–276. https://doi.org/10.1177/1747021820919351
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14(9), 400–410. https://doi.org/10.1016/j.tics.2010.06.008
Thornton, I. M., de’Sperati, C., & Kristjánsson, Á. (2019). The influence of selection modality, display dynamics and error feedback on patterns of human foraging. Visual Cognition, 27(5–8), 626–648. https://doi.org/10.1080/13506285.2019.1658001
Thornton, I. M., & Horowitz, T. S. (2004). The multi-item localization (MILO) task: Measuring the spatiotemporal context of vision for action. Perception & Psychophysics, 66(1), 38–50. https://doi.org/10.3758/BF03194859
Thornton, I. M., & Horowitz, T. S. (2020a). MILO Mobile: An iPad app to measure search performance in multi-target sequences. I-Perception, 11(3), 204166952093258. https://doi.org/10.1177/2041669520932587
Thornton, I. M., & Horowitz, T. S. (2020b). Searching through alternating sequences: Working memory and inhibitory tagging mechanisms revealed using the MILO task. I-Perception, 11(5), 204166952095801. https://doi.org/10.1177/2041669520958018
Thornton, I. M., Nguyen, T. T., & Kristjánsson, Á. (2020). Foraging tempo: Human run patterns in multiple-target search are constrained by the rate of successive responses. Quarterly Journal of Experimental Psychology, 75(2), 297–312. https://doi.org/10.1177/1747021820961640
Thornton, I. M., Nguyen, T. T., & Kristjánsson, Á. (2022). Foraging tempo: Human run patterns in multiple-target search are constrained by the rate of successive responses. Quarterly Journal of Experimental Psychology, 75(2), 297–312. https://doi.org/10.1177/1747021820961640
Thornton, I. M., Tagu, J., Zdravković, S., & Kristjánsson, Á. (2021a). The Predation Game: Does dividing attention affect patterns of human foraging? Cognitive Research: Principles and Implications, 6(1), 35. https://doi.org/10.1186/s41235-021-00299-w
Thornton, I. M., Vuong, Q. C., & Pilz, K. S. (2021b). A search advantage for horizontal targets in dynamic displays. I-Perception, 12(2), 204166952110046. https://doi.org/10.1177/20416695211004616
Treisman, A. (1999). Solutions to the Binding Problem. Neuron, 24(1), 105–125. https://doi.org/10.1016/s0896-6273(00)80826-0
Treisman, A., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136. https://doi.org/10.1016/0010-0285(80)90005-5
Van der Burg, E., Awh, E., & Olivers, C. N. L. (2013). The Capacity of Audiovisual Integration Is Limited to One Item. Psychological Science, 24(3), 345–351. https://doi.org/10.1177/0956797612452865
Van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053–1065. https://doi.org/10.1037/0096-1523.34.5.1053
Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M., & von der Heydt, R. (2012). A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychological Bulletin, 138(6), 1172–1217. https://doi.org/10.1037/a0029333
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100(3), 426–432. https://doi.org/10.1037/a0022790
Wertheimer, M. (1923). Untersuchungen zur Lehre von der Gestalt. II. Psychologische Forschung, 4(1), 301–350. https://doi.org/10.1007/BF00410640
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L., François, R., … Yutani, H. (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686
Wolfe, J. M., Aizenman, A. M., Boettcher, S. E. P., & Cain, M. S. (2016). Hybrid foraging search: Searching for multiple instances of multiple types of target. Vision Research, 119, 50–59. https://doi.org/10.1016/j.visres.2015.12.006
Wolfe, J. M., Friedman-Hill, S. R., Stewart, M. I., & O’Connell, K. M. (1982). The role of categorization in visual search for orientation. Journal of Experimental Psychology: Human Perception and Performance, 18(1), 34–49. https://doi.org/10.1037/0096-1523.18.1.34
Wolfe, J. M., & Horowitz, T. S. (2017). Five factors that guide attention in visual search. Nature Human Behaviour, 1(3), 0058. https://doi.org/10.1038/s41562-017-0058
Woods, A. J., Göksun, T., Chatterjee, A., Zelonis, S., Mehta, A., & Smith, S. E. (2013). The development of organized visual search. Acta Psychologica, 143(2), 191–199. https://doi.org/10.1016/j.actpsy.2013.03.008
Acknowledgement
The work of the first author was funded by Technology Development Fund of the Icelandic Research fund (project no. 1910271).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Makarov, I., Unnthorsson, R., Kristjánsson, Á. et al. The effects of visual and auditory synchrony on human foraging. Atten Percept Psychophys 86, 909–930 (2024). https://doi.org/10.3758/s13414-023-02840-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-023-02840-z