Advertisement

Psychological Research

, Volume 80, Issue 1, pp 149–158 | Cite as

Listen up, eye movements play a role in verbal memory retrieval

  • Agnes ScholzEmail author
  • Katja Mehlhorn
  • Josef F. Krems
Original Article

Abstract

People fixate on blank spaces if visual stimuli previously occupied these regions of space. This so-called “looking at nothing” (LAN) phenomenon is said to be a part of information retrieval from internal memory representations, but the exact nature of the relationship between LAN and memory retrieval is unclear. While evidence exists for an influence of LAN on memory retrieval for visuospatial stimuli, evidence for verbal information is mixed. Here, we tested the relationship between LAN behavior and memory retrieval in an episodic retrieval task where verbal information was presented auditorily during encoding. When participants were allowed to gaze freely during subsequent memory retrieval, LAN occurred, and it was stronger for correct than for incorrect responses. When eye movements were manipulated during memory retrieval, retrieval performance was higher when participants fixated on the area associated with to-be-retrieved information than when fixating on another area. Our results provide evidence for a functional relationship between LAN and memory retrieval that extends to verbal information.

Keywords

Incorrect Response Memory Retrieval Response Accuracy Verbal Information Retrieval Performance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

The authors would like to thank Claudia Dietzel and Lars Eberspach for collecting the data that was reported here, Denise Schneider for recording auditory materials and Anja Prittmann for helpful comments.

References

  1. Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439. doi: 10.1006/jmla.1997.2558.CrossRefGoogle Scholar
  2. Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: the “blank screen paradigm”. Cognition, 93, 79–87. doi: 10.1016/j.cognition.2004.02.005.CrossRefGoogle Scholar
  3. Altmann, G. T. M., & Kamide, Y. (2007). The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language, 57, 502–518. doi: 10.1016/j.jml.2006.12.004.CrossRefGoogle Scholar
  4. Altmann, G. T. M., & Kamide, Y. (2009). Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation. Cognition, 111, 55–71. doi: 10.1016/j.cognition.2008.12.005.PubMedPubMedCentralCrossRefGoogle Scholar
  5. Anderson, S. E., & Spivey, M. J. (2009). The enactment of language: decades of interactions between linguistic and motor processes. Language and Cognition, 1, 87–111. doi: 10.1515/LANGCOG.2009.005.CrossRefGoogle Scholar
  6. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi: 10.1146/annurev.psych.59.103006.093639.PubMedCrossRefGoogle Scholar
  7. Barsalou, L. W., Kyle Simmons, W., Barbey, A. K., & Wilson, C. D. (2003). Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Sciences, 7, 84–91.PubMedCrossRefGoogle Scholar
  8. Brandt, S. A., & Stark, L. W. (1997). Spontaneous eye movements during visual imagery reflect the content of the visual scene. Journal of Cognitive Neuroscience, 9, 27–38.PubMedCrossRefGoogle Scholar
  9. Danker, J. F., & Anderson, J. R. (2010). The ghosts of brain states past: remembering reactivates the brain regions engaged during encoding. Psychological Bulletin, 136, 87–102. doi: 10.1037/a0017937.PubMedPubMedCentralCrossRefGoogle Scholar
  10. Ferreira, F., Apel, J., & Henderson, J. M. (2008). Taking a new look at looking at nothing. Trends in Cognitive Sciences, 12, 405–410. doi: 10.1016/j.tics.2008.07.007.PubMedCrossRefGoogle Scholar
  11. Foulsham, T., & Kingstone, A. (2013). Fixation-dependent memory for natural scenes: An experimental test of scanpath theory. Journal of Experimental Psychology: General, 142, 41–56. doi: 10.1037/a0028227.CrossRefGoogle Scholar
  12. Godijn, R., & Theeuwes, J. (2012). Overt is no better than covert when rehearsing visuo-spatial information in working memory. Memory & Cognition, 40, 52–61. doi: 10.3758/s13421-011-0132-x.CrossRefGoogle Scholar
  13. Grant, E. R., & Spivey, M. J. (2003). Eye movements and problem solving: Guiding attention guides thought. Psychological Science, 14, 462–466.PubMedCrossRefGoogle Scholar
  14. Holm, L., & Mäntylä, T. (2007). Memory for scenes: Refixations reflect retrieval. Memory & Cognition, 35, 1664–1674.CrossRefGoogle Scholar
  15. Hommel, B. (1998). Event files: Evidence for automatic integration of stimulus-response episodes. Visual Cognition, 5, 183–216. doi: 10.1080/713756773.CrossRefGoogle Scholar
  16. Hommel, B. (2004). Event files: Feature binding in and across perception and action. Trends in Cognitive Sciences, 8, 494–500. doi: 10.1016/j.tics.2004.08.007.PubMedCrossRefGoogle Scholar
  17. Hoover, M. A., & Richardson, D. C. (2008). When facts go down the rabbit hole: Contrasting features and objecthood as indexes to memory. Cognition, 108, 533–542. doi: 10.1016/j.cognition.2008.02.011.PubMedCrossRefGoogle Scholar
  18. Huettig, F., Mishra, R. K., & Olivers, C. N. L. (2012). Mechanisms and representations of language-mediated visual attention. Frontiers in Psychology, 2, 1–11. doi: 10.3389/fpsyg.2011.00394.CrossRefGoogle Scholar
  19. Huettig, F., Olivers, C. N. L., & Hartsuiker, R. J. (2011). Looking, language, and memory: Bridging research from the visual world and visual search paradigms. Acta Psychologica, 137, 138–150. doi: 10.1016/j.actpsy.2010.07.013.PubMedCrossRefGoogle Scholar
  20. Jahn, G., & Braatz, J. (2014). Memory indexing of sequential symptom processing in diagnostic reasoning. Cognitive Psychology, 68, 59–97. doi: 10.1016/j.cogpsych.2013.11.002.PubMedCrossRefGoogle Scholar
  21. Johansson, R., Holsanova, J., Dewhurst, R., & Holmqvist, K. (2012). Eye movements during scene recollection have a functional role, but they are not reinstatements of those produced during encoding. Journal of Experimental Psychology: Human Perception and Performance, 38, 1289–1314. doi: 10.1037/a0026585.PubMedGoogle Scholar
  22. Johansson, R., Holsanova, J., & Holmqvist, K. (2006). Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cognitive Science, 30, 1053–1079. doi: 10.1207/s15516709cog0000.PubMedCrossRefGoogle Scholar
  23. Johansson, R., & Johansson, M. (2014). Look here, eye movements play a functional role in memory retrieval. Psychological Science, 25, 236–242. doi: 10.1177/0956797613498260.PubMedCrossRefGoogle Scholar
  24. Kent, C., & Lamberts, K. (2008). The encoding-retrieval relationship: Retrieval as mental simulation. Trends in Cognitive Sciences, 12, 92–98. doi: 10.1016/j.tics.2007.12.004.PubMedCrossRefGoogle Scholar
  25. Laeng, B., Bloem, I. M., D’Ascenzo, S., & Tommasi, L. (2014). Scrutinizing visual images: The role of gaze in mental imagery and memory. Cognition, 131, 263–283. doi: 10.1016/j.cognition.2014.01.003.PubMedCrossRefGoogle Scholar
  26. Laeng, B., & Teodorescu, D. (2002). Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cognitive Science, 26, 207–231. doi: 10.1207/s15516709cog2602.CrossRefGoogle Scholar
  27. Mäntylä, T., & Holm, L. (2006). Gaze control and recollective experience in face recognition. Visual Cognition, 14, 365–386. doi: 10.1080/13506280500347992.CrossRefGoogle Scholar
  28. Martarelli, C. S., & Mast, F. W. (2010). Preschool children’s eye-movements during pictorial recall. The British Journal of Developmental Psychology, 29, 425–436. doi: 10.1348/026151010X495844.CrossRefGoogle Scholar
  29. Martarelli, C. S., & Mast, F. W. (2013). Eye movements during long-term pictorial recall. Psychological Research, 77, 303–309. doi: 10.1007/s00426-012-0439-7.PubMedCrossRefGoogle Scholar
  30. Mast, F. W., & Kosslyn, S. M. (2002). Eye movements during visual mental imagery. Trends in Cognitive Sciences, 6, 271–272.PubMedCrossRefGoogle Scholar
  31. Mayberry, M. R., Crocker, M. W., & Knoeferle, P. (2009). Learning to attend: a connectionist model of situated language comprehension. Cognitive Science, 33, 449–496. doi: 10.1111/j.1551-6709.2009.01019.x.PubMedCrossRefGoogle Scholar
  32. Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16, 519–533. doi: 10.1016/S0022-5371(77)80016-9.CrossRefGoogle Scholar
  33. Mulckhuyse, M., & Theeuwes, J. (2010). Unconscious attentional orienting to exogenous cues: A review of the literature. Acta Psychologica, 134, 299–309. doi: 10.1016/j.actpsy.2010.03.002.PubMedCrossRefGoogle Scholar
  34. Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. San Francisco, CA: WH Freeman.Google Scholar
  35. Platzer, C., Bröder, A., & Heck, D. W. (2014). Deciding with the eye: How the visually manipulated accessibility of information in memory influences decision behavior. Memory & Cognition, 42, 595–608. doi: 10.3758/s13421-013-0380-z.CrossRefGoogle Scholar
  36. Postle, B. R., Idzikowski, C., Sala, S. Della, Logie, R. H., & Baddeley, A. D. (2006). The selective disruption of spatial working memory by eye movements. Quarterly Journal of Experimental Psychology, 59, 100–120. doi: 10.1080/17470210500151410.
  37. Pylyshyn, Z. W. (2001). Visual indexes, preconceptual objects, and situated vision. Cognition, 80, 127–158.PubMedCrossRefGoogle Scholar
  38. Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. Quarterly Journal of Experimental Psychology, 62, 1457–1506. doi: 10.1080/17470210902816461.CrossRefGoogle Scholar
  39. Renkewitz, F., & Jahn, G. (2012). Memory Indexing: A novel method for tracing memory processes in complex cognitive tasks. Journal of Experimental Psychology. Learning, Memory, and Cognition, 38, 1622–1639. doi: 10.1037/a0028073.PubMedCrossRefGoogle Scholar
  40. Richardson, D. C., Altmann, G. T. M., Spivey, M. J., & Hoover, M. A. (2009). Much ado about eye movements to nothing: A response to Ferreira et al.: Taking a new look at looking at nothing. Trends in Cognitive Sciences, 13, 235–236. doi: 10.1016/j.tics.2009.02.006.PubMedCrossRefGoogle Scholar
  41. Richardson, D. C., & Kirkham, N. Z. (2004). Multimodal events and moving locations: eye movements of adults and 6-month-olds reveal dynamic spatial indexing. Journal of Experimental Psychology: General, 133, 46–62. doi: 10.1037/0096-3445.133.1.46.CrossRefGoogle Scholar
  42. Richardson, D. C., & Spivey, M. J. (2000). Representation, space and Hollywood Squares: Looking at things that aren’t there anymore. Cognition, 76, 269–295. doi: 10.1016/S0010-0277(00)00084-6.PubMedCrossRefGoogle Scholar
  43. Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. New York, NY: Cambridge University Press.Google Scholar
  44. Rugg, M. D., Johnson, J. D., Park, H., & Uncapher, M. R. (2008). Encoding-retrieval overlap in human episodic memory: A functional neuroimaging perspective. Progress in Brain Research, 169, 339–352. doi: 10.1016/S0079-6123(07)00021-0.PubMedCrossRefGoogle Scholar
  45. Rugg, M. D., & Wilding, E. (2000). Retrieval processing and episodic memory. Trends in Cognitive Sciences, 4, 108–115.PubMedCrossRefGoogle Scholar
  46. Scholz, A., von Helversen, B., & Rieskamp, J. (2015). Eye movements reveal memory processes during similarity-and rule-based decision making. Cognition, 136, 228–246. doi: 10.1016/j.cognition.2014.11.019.
  47. Spivey, M. J. (2007). The Continuity of Mind. New York, NY: Oxford University Press.Google Scholar
  48. Spivey, M. J., & Dale, R. (2011). Eye movements both reveal and influence problem solving. In S. P. Liversedge, I. D. Gilchrist, & S. Everling (Eds.), The Oxford handbook of eye movements (pp. 551–562). Oxford: Oxford University Press.Google Scholar
  49. Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65, 235–241. doi: 10.1007/s004260100059.PubMedCrossRefGoogle Scholar
  50. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.PubMedCrossRefGoogle Scholar
  51. Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77–99. doi: 10.1016/j.actpsy.2010.02.006.PubMedCrossRefGoogle Scholar
  52. Theeuwes, J., Belopolsky, A. V., & Olivers, C. N. L. (2009). Interactions between working memory, attention and eye movements. Acta Psychologica, 132, 106–114. doi: 10.1016/j.actpsy.2009.01.005.PubMedCrossRefGoogle Scholar
  53. Thomas, L. E., & Lleras, A. (2009). Covert shifts of attention function as an implicit aid to insight. Cognition, 111, 168–174. doi: 10.1016/j.cognition.2009.01.005.PubMedCrossRefGoogle Scholar
  54. Tulving, E. (1983). Elements of episodic memory. Oxford, England: Clarendon Press.Google Scholar
  55. Tulving, E., & Thomson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, 352–373.CrossRefGoogle Scholar
  56. Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9, 625–636.CrossRefGoogle Scholar
  57. Yantis, S., & Jonides, J. (1981). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance, 7, 937–947.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of PsychologyTechnische Universität ChemnitzChemnitzGermany
  2. 2.Department of Artificial IntelligenceUniversity of Groningen9747 AG GroningenThe Netherlands

Personalised recommendations