Advertisement

Memory & Cognition

, Volume 46, Issue 2, pp 230–243 | Cite as

Covert shifts of attention can account for the functional role of “eye movements to nothing”

  • Agnes Scholz
  • Anja Klichowicz
  • Josef F. Krems
Article
  • 325 Downloads

Abstract

When trying to remember verbal information from memory, people look at spatial locations that have been associated with visual stimuli during encoding, even when the visual stimuli are no longer present. It has been shown that such “eye movements to nothing” can influence retrieval performance for verbal information, but the mechanism underlying this functional relationship is unclear. More precisely, covert in comparison to overt shifts of attention could be sufficient to elicit the observed differences in retrieval performance. To test if covert shifts of attention explain the functional role of the looking-at-nothing phenomenon, we asked participants to remember verbal information that had been associated with a spatial location during an encoding phase. Additionally, during the retrieval phase, all participants solved an unrelated visual tracking task that appeared in either an associated (congruent) or an incongruent spatial location. Half the participants were instructed to look at the tracking task, half to shift their attention covertly (while keeping the eyes fixed). In two experiments, we found that memory retrieval depended on the location to which participants shifted their attention covertly. Thus, covert shifts of attention seem to be sufficient to cause differences in retrieval performance. The results extend the literature on the relationship between visuospatial attention, eye movements, and verbal memory retrieval and provide deep insights into the nature of the looking-at-nothing phenomenon.

Keywords

Verbal memory retrieval Eye movements Looking at nothing Covert attention 

Notes

Acknowledgements

Parts of the data of Experiment 1 were presented at the 37th annual meeting of the Cognitive Science Society (July 2015, Pasadena, CA). Agnes Scholz gratefully acknowledges the support of the Swiss National Science Foundation (grant PP00P1_157432). Furthermore, the authors thank Helene Kreysa, Elke Lange, Corinna Martarelli, and Alessandra Souza for helpful comments on a previous version of the manuscript, Anita Todd for editing the manuscript and Daniela Eileen Lippoldt and Marisa Müller for their help in collecting the data.

Author note

Agnes Scholz, Department of Psychology, University of Zurich, Switzerland; Anja Klichowicz and Josef F. Krems, Department of Psychology, Chemnitz University of Technology, Germany.

References

  1. Abrahamse, E., Majerus, S., Fias, W., & van Dijck, J.-P. (2015). Editorial: Turning the mind’s eye inward: The interplay between selective attention and working memory. Frontiers in Human Neuroscience, 9, 1–3. doi: https://doi.org/10.3389/fnhum.2015.00616 CrossRefGoogle Scholar
  2. Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: The “blank screen paradigm.” Cognition, 93, 79–87. doi: https://doi.org/10.1016/j.cognition.2004.02.005 CrossRefGoogle Scholar
  3. Awh, E., Jonides, J., & Reuter-Lorenz, P. A. (1998). Rehearsal in spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 24, 780–790. doi: https://doi.org/10.1037/0096-1523.24.3.780 PubMedGoogle Scholar
  4. Ballard, D. H., Hayhoe, M. M., Pook, P. K., & Rao, R. P. (1997). Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences, 20, 723–742.PubMedGoogle Scholar
  5. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi: https://doi.org/10.1146/annurev.psych.59.103006.093639 CrossRefPubMedGoogle Scholar
  6. Belopolsky, A. V., & Theeuwes, J. (2009). When are attention and saccade preparation dissociated? Psychological Science, 20, 1340–1347. doi: https://doi.org/10.1111/j.1467-9280.2009.02445.x CrossRefPubMedGoogle Scholar
  7. Bochynska, A., & Laeng, B. (2015). Tracking down the path of memory: Eye scanpaths facilitate retrieval of visuospatial information. Cognitive Processing, 16, 159–163. doi: https://doi.org/10.1007/s10339-015-0690-0 CrossRefPubMedPubMedCentralGoogle Scholar
  8. Brandt, S. A., & Stark, L. W. (1997). Spontaneous eye movements during visual imagery reflect the content of the visual scene. Journal of Cognitive Neuroscience, 9, 27–38. doi: https://doi.org/10.1162/jocn.1997.9.1.27 CrossRefPubMedGoogle Scholar
  9. Deubel, H., & Schneider, W. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36, 1827–1837.CrossRefPubMedGoogle Scholar
  10. Ferreira, F., Apel, J., & Henderson, J. M. (2008). Taking a new look at looking at nothing. Trends in Cognitive Sciences, 12, 405–410. doi: https://doi.org/10.1016/j.tics.2008.07.007 CrossRefPubMedGoogle Scholar
  11. Godijn, R., & Theeuwes, J. (2012). Overt is no better than covert when rehearsing visuo-spatial information in working memory. Memory & Cognition, 40, 52–61. doi: https://doi.org/10.3758/s13421-011-0132-x CrossRefGoogle Scholar
  12. Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15, 1176–1194. doi: https://doi.org/10.1162/089892903322598139 CrossRefPubMedGoogle Scholar
  13. Guérard, K., Tremblay, S., & Saint-Aubin, J. (2009). The processing of spatial information in short-term memory: Insights from eye tracking the path length effect. Acta Psychologica, 132, 136–144. doi: https://doi.org/10.1016/j.actpsy.2009.01.003 CrossRefPubMedGoogle Scholar
  14. Hayhoe, M., & Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Sciences, 9, 188–194. doi: https://doi.org/10.1016/j.tics.2005.02.009 CrossRefPubMedGoogle Scholar
  15. Hoffman, J. E., & Subramaniam, B. (1995). The role of visual attention in saccadic eye movements. Perception & Psychophysics, 57, 787–795.CrossRefGoogle Scholar
  16. Hoover, M. A., & Richardson, D. C. (2008). When facts go down the rabbit hole: Contrasting features and objecthood as indexes to memory. Cognition, 108, 533–542. doi: https://doi.org/10.1016/j.cognition.2008.02.011 CrossRefPubMedGoogle Scholar
  17. Huettig, F., Olivers, C. N. L., & Hartsuiker, R. J. (2011). Looking, language, and memory: Bridging research from the visual world and visual search paradigms. Acta Psychologica, 137, 138–150. doi: https://doi.org/10.1016/j.actpsy.2010.07.013 CrossRefPubMedGoogle Scholar
  18. Hulleman, J., & Olivers, C. N. L. (2017). The impending demise of the item in visual search. Behavioral and Brain Sciences, 40. doi: https://doi.org/10.1017/S0140525X15002794
  19. Irwin, D. E. (2004). Fixation location and fixation duration as indices of cognitive processing. In J. M. Henderson & F. Ferreira (Eds.), The interface of language, vision, and action: Eye movements and the visual world (pp. 105–133). New York, NY: Psychology Press.Google Scholar
  20. Jahn, G., & Braatz, J. (2014). Memory indexing of sequential symptom processing in diagnostic reasoning. Cognitive Psychology, 68, 59–97. doi: https://doi.org/10.1016/j.cogpsych.2013.11.002 CrossRefPubMedGoogle Scholar
  21. Johansson, R., Holsanova, J., Dewhurst, R., & Holmqvist, K. (2012). Eye movements during scene recollection have a functional role, but they are not reinstatements of those produced during encoding. Journal of Experimental Psychology: Human Perception and Performance, 38, 1289–1314. doi: https://doi.org/10.1037/a0026585 PubMedGoogle Scholar
  22. Johansson, R., Holsanova, J., & Holmqvist, K. (2006). Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cognitive Science, 30, 1053–1079. doi: https://doi.org/10.1207/s15516709cog0000 CrossRefPubMedGoogle Scholar
  23. Johansson, R., & Johansson, M. (2014). Look here, eye movements play a functional role in memory retrieval. Psychological Science, 25, 236–242. doi: https://doi.org/10.1177/0956797613498260 CrossRefPubMedGoogle Scholar
  24. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. doi: https://doi.org/10.1080/01621459.1995.10476572 CrossRefGoogle Scholar
  25. Kent, C., & Lamberts, K. (2008). The encoding-retrieval relationship: Retrieval as mental simulation. Trends in Cognitive Sciences, 12, 92–98. doi: https://doi.org/10.1016/j.tics.2007.12.004 CrossRefPubMedGoogle Scholar
  26. Kosslyn, S. M. (1994). Image and brain. Cambridge, MA: MIT Press.Google Scholar
  27. Kowler, E., Anderson, E., Dosher, B., & Blaser, E. (1995). The role of attention in the programming of saccades. Vision Research, 35, 1897–1916.CrossRefPubMedGoogle Scholar
  28. Laeng, B., Bloem, I. M., D’Ascenzo, S., & Tommasi, L. (2014). Scrutinizing visual images: The role of gaze in mental imagery and memory. Cognition, 131, 263–283. doi: https://doi.org/10.1016/j.cognition.2014.01.003 CrossRefPubMedGoogle Scholar
  29. Laeng, B., & Teodorescu, D. (2002). Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cognitive Science, 26, 207–231. doi: https://doi.org/10.1207/s15516709cog2602 CrossRefGoogle Scholar
  30. Lawrence, B. M., Myerson, J., & Abrams, R. A. (2004). Interference with spatial working memory: An eye movement is more than a shift of attention. Psychonomic Bulletin & Review, 11, 488–494. doi: https://doi.org/10.3758/BF03196600 CrossRefGoogle Scholar
  31. Lepsien, J., Griffin, I. C., Devlin, J. T., & Nobre, A. C. (2005). Directing spatial attention in mental representations: Interactions between attentional orienting and working-memory load. NeuroImage, 26, 733–743. doi: https://doi.org/10.1016/j.neuroimage.2005.02.026 CrossRefPubMedGoogle Scholar
  32. Martarelli, C. S., Chiquet, S., Laeng, B., & Mast, F.W. (2017). Using space to represent categories: Insight from gaze position. Psychological Research, 81, 721–729. doi: https://doi.org/10.1007/s00426-016-0781-2 CrossRefPubMedGoogle Scholar
  33. Martarelli, C. S., & Mast, F. W. (2011). Preschool children’s eye-movements during pictorial recall. British Journal of Developmental Psychology, 29, 425–436. doi: https://doi.org/10.1348/026151010X495844 CrossRefPubMedGoogle Scholar
  34. Martarelli, C. S., & Mast, F. W. (2013). Eye movements during long-term pictorial recall. Psychological Research, 77, 303–309. doi: https://doi.org/10.1007/s00426-012-0439-7 CrossRefPubMedGoogle Scholar
  35. Morey, R. D. (2008). Confidence intervals from normalized data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4, 61–64.CrossRefGoogle Scholar
  36. Mulckhuyse, M., & Theeuwes, J. (2010). Unconscious attentional orienting to exogenous cues: A review of the literature. Acta Psychologica, 134, 299–309. doi: https://doi.org/10.1016/j.actpsy.2010.03.002 CrossRefPubMedGoogle Scholar
  37. Olivers, C. N. L., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception and Performance, 32, 1243–1265. doi: https://doi.org/10.1037/0096-1523.32.5.1243 PubMedGoogle Scholar
  38. Pearson, D., & Sahraie, A. (2003). Oculomotor control and the maintenance of spatially and temporally distributed events in visuo-spatial working memory. Quarterly Journal of Experimental Psychology, 56A, 1089–1111. doi: https://doi.org/10.1080/02724980343000044 CrossRefGoogle Scholar
  39. Platzer, C., Bröder, A. & Heck, D. (2014). Deciding with the eye: How the visually manipulated accessibility of information in memory influences decision behavior. Memory & Cognition, 42, 595–608. doi: https://doi.org/10.3758/s13421-013-0380-z CrossRefGoogle Scholar
  40. Postle, B. R., Idzikowski, C., Sala, S. D., Logie, R. H., & Baddeley, A. D. (2006). The selective disruption of spatial working memory by eye movements. Quarterly Journal of Experimental Psychology, 59, 100–120. doi: https://doi.org/10.1080/17470210500151410 CrossRefGoogle Scholar
  41. Renkewitz, F., & Jahn, G. (2012). Memory indexing: A novel method for tracing memory processes in complex cognitive tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 1622–1639. doi: https://doi.org/10.1037/a0028073 PubMedGoogle Scholar
  42. Richardson, D. C., Altmann, G. T. M., Spivey, M. J., & Hoover, M. A. (2009). Much ado about eye movements to nothing. Trends in Cognitive Sciences, 13, 235–236. doi: https://doi.org/10.1016/j.tics.2009.02.006 CrossRefPubMedGoogle Scholar
  43. Richardson, D. C., & Kirkham, N. Z. (2004). Multimodal events and moving locations: Eye movements of adults and 6-month-olds reveal dynamic spatial indexing. Journal of Experimental Psychology: General, 133, 46–62. doi: https://doi.org/10.1037/0096-3445.133.1.46 CrossRefGoogle Scholar
  44. Richardson, D. C., & Spivey, M. J. (2000). Representation, space and Hollywood Squares: Looking at things that aren’t there anymore. Cognition, 76, 269–295. doi: https://doi.org/10.1016/S0010-0277(00)00084-6 CrossRefPubMedGoogle Scholar
  45. Rizzolatti, G., Riggio, L., & Sheliga, B. M. (1987). Space and selective attention. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 231–265). Cambridge, MA: MIT Press/Bradford Books.Google Scholar
  46. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237. doi: https://doi.org/10.3758/PBR.16.2.225 CrossRefGoogle Scholar
  47. Scholz, A., Krems, J. F., & Jahn, G. (2017). Watching diagnoses develop: Eye movements reveal symptom processing during diagnostic reasoning. Psychonomic Bulletin & Review. doi: https://doi.org/10.3758/s13423-017-1294-8
  48. Scholz, A., Mehlhorn, K., Bocklisch, F., & Krems, J. F. (2011). Looking at nothing diminishes with practice. In L. Carlson, C. Hoelscher, & T. F. Shipley (Eds.), Proceedings of the 33rd annual conference of the Cognitive Science Society (pp. 1070–1075). Austin, TX: Cognitive Science Society.Google Scholar
  49. Scholz, A., Mehlhorn, K., & Krems, J. F. (2016). Listen up, eye movements play a role in verbal memory retrieval. Psychological Research, 80, 149–158. doi: https://doi.org/10.1007/s00426-014-0639-4 CrossRefPubMedGoogle Scholar
  50. Scholz, A., von Helversen, B., & Rieskamp, J. (2015). Eye movements reveal memory processes during similarity- and rule-based decision making. Cognition, 136, 228–246. doi: https://doi.org/10.1016/j.cognition.2014.11.019 CrossRefPubMedGoogle Scholar
  51. Smyth, M. M. (1996). Interference with rehearsal in spatial working memory in the absence of eye movements. Quarterly Journal of Experimental Psychology, 49A, 940–949. doi: https://doi.org/10.1080/713755669 CrossRefGoogle Scholar
  52. Smyth, M. M., & Scholey, K. A. (1994). Interference in immediate spatial memory. Memory & Cognition, 22, 1–13.CrossRefGoogle Scholar
  53. Souza, A. S., & Oberauer, K. (2016). In search of the focus of attention in working memory: 13 years of the retro-cue effect. Attention, Perception, & Psychophysics, 78, 1839–1860. doi: https://doi.org/10.3758/s13421-013-0392-8 CrossRefGoogle Scholar
  54. Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65, 235–241. doi: https://doi.org/10.1007/s004260100059 CrossRefPubMedGoogle Scholar
  55. Staudte, M., & Altmann, G. T. M. (2016). Recalling what was where when seeing nothing there. Psychonomic Bulletin & Review, 24, 400–407. doi: https://doi.org/10.3758/s13423-016-1104-8 CrossRefGoogle Scholar
  56. Tas, A. C., Luck, S. J., & Hollingworth, A. (2016). The relationship between visual attention and visual working memory encoding: A dissociation between covert and overt orienting. Journal of Experimental Psychology: Human Perception and Performance, 42, 1121–1138. doi: https://doi.org/10.1037/xhp0000212.PubMedPubMedCentralGoogle Scholar
  57. Theeuwes, J., Belopolsky, A. V., & Olivers, C. N. L. (2009). Interactions between working memory, attention and eye movements. Acta Psychologica, 132, 106–114. doi: https://doi.org/10.1016/j.actpsy.2009.01.005 CrossRefPubMedGoogle Scholar
  58. Theeuwes, J., Kramer, A. F., & Irwin, D. E. (2011). Attention on our mind: The role of spatial attention in visual working memory. Acta Psychologica, 137, 248–251. doi: https://doi.org/10.1016/j.actpsy.2010.06.011 CrossRefPubMedGoogle Scholar
  59. Thomas, L. E., & Lleras, A. (2009). Covert shifts of attention function as an implicit aid to insight. Cognition, 111, 168–174. doi: https://doi.org/10.1016/j.cognition.2009.01.005 CrossRefPubMedGoogle Scholar
  60. Thomas, N. J. T. (1999). Are theories of imagery theories of imagination? An active perception approach to conscious mental content. Cognitive Science, 23, 207–245.CrossRefGoogle Scholar
  61. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804. doi: https://doi.org/10.3758/BF03194105 CrossRefGoogle Scholar
  62. Wantz, A. L., Martarelli, C. S., Cazzoli, D., Kalla, R., Müri, R., & Mast, F. W. (2016). Disrupting frontal eye-field activity impairs memory recall. NeuroReport, 27, 374–378. doi: https://doi.org/10.1097/WNR.0000000000000544 CrossRefPubMedGoogle Scholar
  63. Wantz, A. L., Martarelli, C. S., & Mast, F. W. (2016). When looking back to nothing goes back to nothing. Cognitive Processing, 17, 105–114. doi: https://doi.org/10.1007/s10339-015-0741-6 CrossRefPubMedGoogle Scholar
  64. Yantis, S., & Jonides, J. (1981). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance, 7, 937–947.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Agnes Scholz
    • 1
  • Anja Klichowicz
    • 2
  • Josef F. Krems
    • 2
  1. 1.Cognitive Decision PsychologyUniversity of ZurichZurichSwitzerland
  2. 2.Cognitive and Engineering PsychologyChemnitz University of TechnologyChemnitzGermany

Personalised recommendations