Advertisement

Studying the dynamics of visual search behavior using RT hazard and micro-level speed–accuracy tradeoff functions: A role for recurrent object recognition and cognitive control processes

  • Sven PanisEmail author
  • Rani Moran
  • Maximilian P. Wolkersdorfer
  • Thomas Schmidt
40 Years of Feature Integration: Special Issue in Memory of Anne Treisman

Abstract

Thanks to the work of Anne Treisman and many others, the visual search paradigm has become one of the most popular paradigms in the study of visual attention. However, statistics like mean correct response time (RT) and percent error do not usually suffice to decide between the different search models that have been developed. Recently, to move beyond mean performance measures in visual search, RT histograms have been plotted, theoretical waiting time distributions have been fitted, and whole RT and error distributions have been simulated. Here we promote and illustrate the general application of discrete-time hazard analysis to response times, and of micro-level speed–accuracy tradeoff analysis to timed response accuracies. An exploratory analysis of published benchmark search data from feature, conjunction, and spatial configuration search tasks reveals new features of visual search behavior, such as a relatively flat hazard function in the right tail of the RT distributions for all tasks, a clear effect of set size on the shape of the RT distribution for the feature search task, and individual differences in the presence of a systematic pattern of early errors. Our findings suggest that the temporal dynamics of visual search behavior results from a decision process that is temporally modulated by concurrently active recurrent object recognition, learning, and cognitive control processes, next to attentional selection processes.

Keywords

Visual search Response times Discrete-time hazard analysis Individual differences Speed–accuracy tradeoff Event history analysis 

Notes

Acknowledgements

We wish to thank Heinrich René Liesefeld, Hermann J. Müller, and an anonymous reviewer for their helpful comments on previous drafts. This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Projektnummer PA 2947/1-1 (to S.P.).

Open practices statement

This article presents a reanalysis of publicly available data. The R code for the descriptive and inferential event-history analyses is available from the first author upon request.

References

  1. Allison, P. D. (1982). Discrete-time methods for the analysis of event histories. Sociological Methodology, 13, 61–98.CrossRefGoogle Scholar
  2. Allison, P. D. (2010). Survival analysis using SAS: A practical guide, Second Edition. SAS Institute Inc., Cary, NC, USA.Google Scholar
  3. Balota, D. A., & Yap, M. J. (2011). Moving beyond the mean in studies of mental chronometry: The power of response time distributional analyses. Current Directions in Psychological Science, 20 (3), 160–166.CrossRefGoogle Scholar
  4. Bloxom, B. (1984). Estimating response time hazard functions: An exposition and extension. Journal of Mathematical Psychology, 28, 401–420.CrossRefGoogle Scholar
  5. Braver, T. S. (2012). The variable nature of cognitive control: A dual mechanisms framework. Trends in Cognitive Sciences, 16 (2), 106–113.PubMedPubMedCentralCrossRefGoogle Scholar
  6. Burle, B., Vidal, F., Tandonnet, C., & Hasbroucq, T. (2004). Physiological evidence for response inhibition in choice reaction time tasks. Brain and Cognition, 56, 153–164.PubMedCrossRefGoogle Scholar
  7. Cheal, M., & Lyon, D. R. (1992). Attention in visual search: Multiple search classes. Perception & Psychophysics, 52 (2), 113–138.CrossRefGoogle Scholar
  8. Chechile, R. A. (2003). Mathematical tools for hazard function analysis. Journal of Mathematical Psychology, 47, 478–494.CrossRefGoogle Scholar
  9. Cisek, P., & Kalaska, J. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annual Review of Neuroscience, 33, 269–298.PubMedCrossRefGoogle Scholar
  10. Deco, G., & Zihl, J. (2006). The neurodynamics of visual search. Visual Cognition, 14, 1006–1024.CrossRefGoogle Scholar
  11. Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96 (3), 433–458.PubMedCrossRefGoogle Scholar
  12. Dutilh, G., et al. (2018). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review, published online:  https://doi.org/10.3758/s13423-017-1417-2 CrossRefGoogle Scholar
  13. Eckstein, M. P. (1998). The lower visual search efficiency for conjunctions is due to noise and not serial attentional processing. Psychological Science, 9 (2), 111–118.CrossRefGoogle Scholar
  14. Eckstein, M. P. (2011). Visual search: A retrospective. Journal of Vision, 11(5):14, 1–36.PubMedCrossRefGoogle Scholar
  15. Engel, F. L. (1977) Visual conspicuity, visual search and fixation tendencies of the eye. Vision Research, 17, 95–108.  https://doi.org/10.1016/0042-6989(77)90207-3.CrossRefPubMedGoogle Scholar
  16. Eriksen, C. W., Coles, M. G. H., Morris, L. R., & O’hara, W. P. (1985). An electromyographic examination of response competition. Bulletin of the Psychonomic Society, 23 (3), 165–168.CrossRefGoogle Scholar
  17. Fix, J., Rougier, N., & Alexandre, F. (2011). A dynamic neural field approach to the covert and overt deployment of spatial attention. Cognitive Computation, 3, 279–293.CrossRefGoogle Scholar
  18. Gabroi, D., & Lisman, J. (2003). Recognition by top-down and bottum-up processing in cortex: The control of selective attention. Journal of Neurophysiology, 90, 798–810.CrossRefGoogle Scholar
  19. Geisler, W. S. & Chou, K. L. (1995) Separation of low-level and high-level factors in complex tasks: Visual search. Psychological Review, 102, 356–78.  https://doi.org/10.1037/0033-295X.102.2.356.CrossRefPubMedGoogle Scholar
  20. Grieben, R., Tekülve, J., Zibner, S. K. U., Schneegans, S., & Schöner, G. (2018). Sequences of discrete attentional shifts emerge from a neural dynamic architecture for conjunctive visual search that operates in continuous time. In T. T. Rogers, Rau, M., Zhu, X., & Kalish, C. W. (Eds.), Proceedings of the 40thAnnual Conference of the Cognitive Science Society (pp. 429–434). Downloaded from http://mindmodeling.org/cogsci2018/papers/0099/index.html
  21. Haslam, N., Porter, M., & Rothschild, L. (2001). Visual search: Efficiency continuum or distinct processes? Psychonomic Bulletin & Review, 8, 742–746.  https://doi.org/10.3758/BF03196212 CrossRefGoogle Scholar
  22. Heinke, D., & Backhaus, A. (2011). Modelling visual search with the selective attention for identification model (VS-SAIM): A novel explanation for visual search asymmetries. Cognitive Computation, 3, 185–205.PubMedCrossRefPubMedCentralGoogle Scholar
  23. Heinke, D., & Humphreys, G. W. (2003). Attention, spatial representation and visual neglect: Simulating emergent attention and spatial memory in the Selective Attention for Identification model (SAIM). Psychological Review, 110 (1), 29–87.PubMedCrossRefPubMedCentralGoogle Scholar
  24. Hochstein, S., & Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791–804.PubMedCrossRefPubMedCentralGoogle Scholar
  25. Holden, J. G., Van Orden, G. C., & Turvey, M. T. (2009). Dispersion of response times reveals cognitive dynamics. Psychological Review, 116 (2), 318–342.PubMedPubMedCentralCrossRefGoogle Scholar
  26. Hulleman, J., & Olivers, C. N. L. (2017). The impending demise of the item in visual search. The Behavioral and Brain Sciences, 40, e132.  https://doi.org/10.1017/S0140525X15002794 CrossRefPubMedPubMedCentralGoogle Scholar
  27. Humphreys, G. W., & Müller, H. J. (1993). Search via Recursive Rejection (SERR): A connectionist model of visual search. Cognitive Psychology, 25, 43–110.CrossRefGoogle Scholar
  28. Humphreys, G. W. (2016). Feature confirmation in object perception: Feature integration theory 26 years on from the Treisman Bartlett lecture. The Quarterly Journal of Experimental Psychology, 69 (10), 1910–1940.PubMedCrossRefPubMedCentralGoogle Scholar
  29. Kazanovich, Y., & Borisyuk, R. (2017). Reaction times in visual search can be explained by a simple model of neural synchronization. Neural Networks, 87, 1–7.PubMedCrossRefPubMedCentralGoogle Scholar
  30. Kiss, M., Grubert, A., & Eimer, M. (2012). Top-down task sets for combined features: Behavioral and electrophysiological evidence for two stages in attentional object selection. Attention, Perception, & Psychophysics, 75 (2), 216–228.CrossRefGoogle Scholar
  31. Lee, J., Leonard, C. J., Luck, S. J., & Geng, J. J. (2018). Dynamics of feature-based attentional selection during color-shape conjunction search. Journal of Cognitive Neuroscience, 30 (12), 1773–1787.CrossRefGoogle Scholar
  32. Li, K., Kadohisa, M., Kusunoki, M., Duncan, J., Bundesen, C., & Ditlevsen, S. (2018). Distinguishing between parallel and serial processing in visual attention from neurobiological data. bioRxiv preprint first posted online Aug. 2, 2018; 10.1101/383596.Google Scholar
  33. Liesefeld, H. R. (2018). Estimating the timing of cognitive operations with MEG/EEG latency measures: A primer, a brief tutorial, and an implementation of various methods. Frontiers in Neuroscience, 12, Article 765.Google Scholar
  34. Liesefeld, H. R., Moran, R., Usher, M., Müller, H. J., & Zehetleitner, M. (2016). Search efficiency as a function of target saliency: The transition from inefficient to efficient search and beyond. Journal of Experimental Psychology: Human Perception and Performance, 42 (6), 821–836.PubMedGoogle Scholar
  35. Liesefeld, H.R., & Müller, H.J. (2019). A theoretical attempt to revive the serial/parallel-search dichotomy. Attention, Perception, & Psychophysics. Advance online publication.  https://doi.org/10.3758/s13414-019-01819-z
  36. Luce, R. D. (1986). Response times. Their role in inferring elementary mental organization. New York: Oxford University Press Inc.Google Scholar
  37. McElree, B., & Carrasco, M. (1999). The temporal dynamics of visual search: Evidence for parallel processing in feature and conjunction searches. JEP:HPP, 25 (6), 1517–1539.Google Scholar
  38. Meyer, D. E., Osman, A. M., Irwin, D. E., & Yantis, S. (1988). Modern mental chronometry. Biological Psychology, 26, 3–67.PubMedCrossRefGoogle Scholar
  39. Moran, R., Zehetleitner, M., Müller, H. J., & Usher, M. (2013). Competitive guided search: Meeting the challenge of benchmark RT distributions. Journal of Vision, 13(8):24, 1–31.PubMedCrossRefPubMedCentralGoogle Scholar
  40. Moran, R., Zehetleitner, M., Liesefeld, H. R., Müller, H. J., & Usher, M. (2016). Serial vs. parallel models of attention in visual search: accounting for benchmark RT-distributions. Psychonomic Bulletin & Review, 23, 1300–1315.CrossRefGoogle Scholar
  41. Mordkoff, J. T., Yantis, S., & Egeth, H. E. (1990). Detecting conjunctions of color and form in parallel. Perception & Psychophysics, 48 (2), 157–568.CrossRefGoogle Scholar
  42. Müller, H. J., Humphreys, G. W., & Donnelly, N. (1994). Search via Recursive Rejection (SERR): Visual search for single and dual form-conjunction targets. Journal of Experimental Psychology: Human Perception and Performance, 20 (2), 235–258.PubMedGoogle Scholar
  43. Nakayama, K., & Martini, P. (2011). Situating visual search. Vision Research, 51, 1526–1537.PubMedCrossRefPubMedCentralGoogle Scholar
  44. Narbutas, V., Lin, Y.-S., Kristan, M., & Heinke, D. (2017). Serial versus parallel search: A model comparison approach based on reaction time distributions. Visual Cognition, 25 (1–3), 306–325.CrossRefGoogle Scholar
  45. Pachella, R. G. (1974). The interpretation of reaction time in information processing research. In: B. Kantowitz (Ed.), Human information processing, 41–82. Potomac, MD: Erlbaum.Google Scholar
  46. Palmer, J. (1995). Attention in visual search: Distinguishing four causes of a set-size effect. Current Directions in Psychological Science, 4 (4), 118–123.CrossRefGoogle Scholar
  47. Palmer, E. M., Horowitz, T. S., Torralba, A., & Wolfe, J. M. (2011). What are the shapes of response time distributions in visual search? Journal of Experimental Psychology: Human Perception and Performance, 37 (1), 58–71.PubMedGoogle Scholar
  48. Panis, S., & Hermens, F. (2014). Time course of spatial contextual interference: Event history analyses of simultaneous masking by nonoverlapping patterns. Journal of Experimental Psychology: Human Perception & Performance, 40 (1), 129–144.  https://doi.org/10.1037/a0032949 CrossRefGoogle Scholar
  49. Panis, S., & Schmidt, T. (2016). What is shaping RT and accuracy distributions? Active and selective response inhibition causes the negative compatibility effect. Journal of Cognitive Neuroscience, 28 (11), 1651–1671.PubMedCrossRefGoogle Scholar
  50. Panis, S., Torfs, K., Gillebert, C. R., Wagemans, J., & Humphreys, G. W. (2017). Neuropsychological evidence for the temporal dynamics of category-specific naming. Visual Cognition, 25 (1–3), 79–99.  https://doi.org/10.1080/13506285.2017.1330790 CrossRefPubMedPubMedCentralGoogle Scholar
  51. Panis, S., & Wagemans, J. (2009). Time-course contingencies in perceptual organization and object identification of fragmented object outlines. Journal of Experimental Psychology: Human Perception and Performance, 35, 661–687.PubMedPubMedCentralGoogle Scholar
  52. Pashler, H. (1987). Detecting conjunctions of color and form: Reassessing the serial search hypothesis. Perception & Psychophysics, 41 (3), 191–201.CrossRefGoogle Scholar
  53. Praamstra, P., & Seiss, E. (2005). The neurophysiology of response competition: Motor cortex activation and inhibition following subliminal response priming. Journal of Cognitive Neuroscience, 17 (3), 483–493.PubMedCrossRefPubMedCentralGoogle Scholar
  54. R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.
  55. Shenoy, K. V., Sahani, M., & Churchland, M. M. (2013). Cortical control of arm movements: A dynamical systems perspective. Annual Review of Neuroscience, 36, 337–359.PubMedCrossRefPubMedCentralGoogle Scholar
  56. Singer, J. D., & Willett, J. B. (1991). Modelling the days of our lives: Using survival analysis when designing and analyzing longitudinal studies of duration and the timing of events. Psychological Bulletin, 110 (2), 268–290.CrossRefGoogle Scholar
  57. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modelling change and event occurrence. New York: Oxford University Press Inc.CrossRefGoogle Scholar
  58. Smith, P. L., & Little, D. R. (2018). Small is beautiful: In defense of the small-N design. Psychonomic Bulletin and Review, 25, 2083–2101.PubMedCrossRefGoogle Scholar
  59. Song, J.-H., & Nakayama, K. (2009). Hidden cognitive states revealed in choice reaching tasks. Trends in Cognitive Sciences, 13 (8), 360–366.PubMedCrossRefGoogle Scholar
  60. Sung K. (2008). Serial and parallel attentive visual searches: Evidence from cumulative distribution functions of response times. Journal of Experimental Psychology: Human Perception and Performance, 34 (6), 1372–1388.PubMedPubMedCentralGoogle Scholar
  61. Theeuwes, J. (1994). Endogenous and exogenous control of visual selection. Perception, 23, 429–440.  https://doi.org/10.1068/p230429 CrossRefPubMedPubMedCentralGoogle Scholar
  62. Thompson, K. G., Hanes, D. P., Bichot, N. P., & Schall, J. D. (1996). Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. Journal of Neurophysiology, 76 (6), 4040–4055.PubMedCrossRefPubMedCentralGoogle Scholar
  63. Torfs, K., Panis, S., & Wagemans, J. (2010). Identification of fragmented object outlines: A dynamic interplay between different component processes. Visual Cognition, 18 (8), 1133–1164.CrossRefGoogle Scholar
  64. Townsend, J. T. (1990a). Serial vs. parallel processing: Sometimes they look like Tweedledum and Tweedledee but they can (and should) be distinguished. Psychological Science, 1 (1), 46–54.CrossRefGoogle Scholar
  65. Townsend, J. T. (1990b). Truth and consequences of ordinal differences in statistical distributions: Toward a theory of hierarchical inference. Psychological Bulletin, 108 (3), 551–567.PubMedCrossRefPubMedCentralGoogle Scholar
  66. Treisman, A., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97–136.PubMedCrossRefPubMedCentralGoogle Scholar
  67. Treisman, A., & Sato, S. (1990). Conjunction Search Revisited. Journal of Experimental Psychology: Human Perception and Performance, 16 (3), 459–478.PubMedPubMedCentralGoogle Scholar
  68. Treisman, A., & Souther, J. (1985). Search asymmetry: A diagnostic for preattentive processing of separable features. Journal of Experimental Psychology: General, 114, 285–310.  https://doi.org/10.1037/0096-3445.114.3.285 CrossRefGoogle Scholar
  69. VanRullen, R., Reddy, L., & Koch, C. (2004). Visual search and dual tasks reveal two distinct attentional resources. Journal of Cognitive Neuroscience, 16 (1), 4–14.PubMedCrossRefPubMedCentralGoogle Scholar
  70. Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin & Review, 7 (3), 424–465.CrossRefGoogle Scholar
  71. Wenger, M. J., & Gibson, B. S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cuing paradigm. JEP:HPP, 30 (4), 708–719.Google Scholar
  72. Whelan, R. (2008). Effective analysis of reaction time data. The Psychological Record, 58, 475–482.CrossRefGoogle Scholar
  73. Wickelgren, W. A. (1977). Speed–accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85.CrossRefGoogle Scholar
  74. Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Psychonomic Bulletin & Review, 1 (2), 202–238.CrossRefGoogle Scholar
  75. Wolfe, J. M. (1998). What can 1 million trials tell us about visual search? Psychological Science, 9, 33–39.  https://doi.org/10.1111/1467-9280.00006 CrossRefGoogle Scholar
  76. Wolfe, J. M. (2007). Guided search 4.0: Current progress with a model of visual search. In: W. D. Grey (Ed.), Integrated Models of Cognitive Systems, 99–119. New York, Oxford University Press, Inc.Google Scholar
  77. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. JEP:HPP, 15 (3), 419–433.Google Scholar
  78. Wolfe, J. M., Palmer, E. M., & Horowitz, T. S. (2010). Reaction time distributions constrain models of visual search. Vision Research, 50, 1304–1311.PubMedCrossRefGoogle Scholar
  79. Willett, J. B., & Singer, J. D. (1993). Investigating onset, cessation, relapse, and recovery: Why you should, and how you can, use discrete-time survival analysis to examine event occurrence. Journal of Consulting and Clinical Psychology, 61 (6), 952-965.PubMedCrossRefGoogle Scholar
  80. Willett, J. B., & Singer, J. D. (1995). It’s déjà vu all over again: Using multiple-spell discrete-time survival analysis. Journal of Educational and Behavioral Statistics, 20, 41–67.Google Scholar

Copyright information

© The Psychonomic Society, Inc. 2020

Authors and Affiliations

  1. 1.Experimental Psychology Unit, Faculty of Social SciencesTechnische Universität KaiserslauternKaiserslauternGermany
  2. 2.Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchUniversity College LondonLondonUK
  3. 3.Wellcome Centre for Human NeuroimagingUniversity College LondonLondonUK

Personalised recommendations