How well do word recognition measures correlate? Effects of language context and repeated presentations

  • Nicolas DirixEmail author
  • Marc Brysbaert
  • Wouter Duyck


In the present study we assessed the extent to which different word recognition time measures converge, using large databases of lexical decision times and eyetracking measures. We observed a low proportion of shared variance between these measures, which limits the validity of lexical decision times to real-life reading. We further investigated and compared the role of word frequency and length, two important predictors of word-processing latencies in these paradigms, and found that they influenced the measures to different extents. A second analysis of two different eyetracking corpora compared the eyetracking reading times for short paragraphs with those from reading of an entire book. Our results revealed that the correlations between eyetracking reading times of identical words in two different corpora are also low, suggesting that the higher-order language context in which words are presented plays a crucial role. Finally, our findings indicate that lexical decision times better resemble the average processing time of multiple presentations of the same word, across different language contexts.


Visual word recognition Eyetracking Big data studies 



  1. Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H., & Yap, M. J. (2004). Visual word recognition of single-syllable words. Journal of Experimental Psychology: General, 133, 283–316. CrossRefGoogle Scholar
  2. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., . . . Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39, 445–459. CrossRefGoogle Scholar
  3. Boston, M. F., Hale, J. T., Kliegl, R., Patil, U., & Vasishth, S. (2008). Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. Journal of Eye Movement Research, 2, 1–12.
  4. Brysbaert, M., & Cortese, M. J. (2011). Do the effects of subjective frequency and age of acquisition survive better word frequency norms? Quarterly Journal of Experimental Psychology, 64, 545–559. CrossRefGoogle Scholar
  5. Brysbaert, M., Lagrou, E., & Stevens, M. (2017). Visual word recognition in a second language: A test of the lexical entrenchment hypothesis with lexical decision times. Bilingualism: Language and Cognition, 20, 530–548. CrossRefGoogle Scholar
  6. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41, 977–990. CrossRefPubMedGoogle Scholar
  7. Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1, 1–20. CrossRefGoogle Scholar
  8. Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). The impact of word prevalence on lexical decision times: Evidence from the Dutch Lexicon Project 2. Journal of Experimental Psychology: Human Perception and Performance, 42, 441–458. CrossRefPubMedGoogle Scholar
  9. Butler, B., & Hains, S. (1979). Individual differences in word recognition latency. Memory & Cognition, 7, 68–76. CrossRefGoogle Scholar
  10. Carter, B. T., & Luke, S. G. (2018). Individuals’ eye movements in reading are highly consistent across time and trial. Journal of Experimental Psychology: Human Perception and Performance, 44, 482–492. CrossRefPubMedGoogle Scholar
  11. Cop, U., Dirix, N., Drieghe, D., & Duyck, W. (2017). Presenting GECO: An eyetracking corpus of monolingual and bilingual sentence reading. Behavior Research Methods, 49, 602–615. CrossRefPubMedGoogle Scholar
  12. Cop, U., Drieghe, D., & Duyck, W. (2015a). Eye movement patterns in natural reading: A comparison of monolingual and bilingual reading of a novel. PLOS ONE, 10, e0134008. CrossRefPubMedPubMedCentralGoogle Scholar
  13. Cop, U., Keuleers, E., Drieghe, D., & Duyck, W. (2015b). Frequency effects in monolingual and bilingual natural reading. Psychonomic Bulletin & Review, 22, 1216–1234. CrossRefGoogle Scholar
  14. Courrieu, P., Brand-D’Abrescia, M., Peereman, R., Spieler, D., & Rey, A. (2011). Validated intraclass correlation statistics to test item performance models. Behavior Research Methods, 43, 37–55. CrossRefPubMedGoogle Scholar
  15. Courrieu, P., & Rey, A. (2011). Missing data imputation and corrected statistics for large-scale behavioral databases. Behavior Research Methods, 43, 310–330. CrossRefPubMedGoogle Scholar
  16. Dirix, N., Cop, U., Drieghe, D., & Duyck, W. (2017). Cross-lingual neighborhood effects in generalized lexical decision and natural reading. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43, 887–915. CrossRefPubMedGoogle Scholar
  17. Dirix, N., & Duyck, W. (2017). The first- and second-language age of acquisition effect in first- and second-language book reading. Journal of Memory and Language, 97, 103–120. CrossRefGoogle Scholar
  18. Duyck, W., Van Assche, E., Drieghe, D., & Hartsuiker, R. J. (2007). Visual word recognition by bilinguals in a sentence context: Evidence for nonselective lexical access. Journal of Experimental Psychology. Learning, Memory, and Cognition, 33, 663–679. CrossRefPubMedGoogle Scholar
  19. Duyck, W., Vanderelst, D., Desmet, T., & Hartsuiker, R. J. (2008). The frequency effect in second-language visual word recognition. Psychonomic Bulletin & Review, 15, 850–855. CrossRefGoogle Scholar
  20. Engbert, R., Nuthmann, A., Richter, E. M., & Kliegl, R. (2005). SWIFT: A dynamical model of saccade generation during reading. Psychological Review, 112, 777–813. CrossRefPubMedGoogle Scholar
  21. Gollan, T. H., Slattery, T. J., Goldenberg, D., Van Assche, E., Duyck, W., & Rayner, K. (2011). Frequency drives lexical access in reading but not in speaking: The frequency-lag hypothesis. Journal of Experimental Psychology. General, 140, 186–209. CrossRefPubMedPubMedCentralGoogle Scholar
  22. Hoedemaker, R. S., & Gordon, P. C. (2014). It takes time to prime: Semantic priming in the ocular lexical decision task. Journal of Experimental Psychology: Human Perception and Performance, 40, 2179–2197. CrossRefPubMedGoogle Scholar
  23. Hoedemaker, R. S., & Gordon, P. C. (2017). The onset and time course of semantic priming during rapid recognition of visual words. Journal of Experimental Psychology: Human Perception and Performance, 43, 881–902. CrossRefPubMedGoogle Scholar
  24. Hudson, P. T. W., & Bergman, M. W. (1985). Lexical knowledge in word recognition: Word length and word frequency in naming and lexical decision tasks. Journal of Memory and Language, 24, 46–58. CrossRefGoogle Scholar
  25. Kennedy, A., & Pynte, J. (2005). Parafoveal-on-foveal effects in normal reading. Vision Research, 45, 153–168. CrossRefPubMedGoogle Scholar
  26. Keuleers, E., & Brysbaert, M. (2010). Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42, 627–633. CrossRefPubMedGoogle Scholar
  27. Keuleers, E., & Brysbaert, M. (2011). Detecting inherent bias in lexical decision experiments with the LD1NN algorithm. Mental Lexicon, 6, 34–52. CrossRefGoogle Scholar
  28. Keuleers, E., Brysbaert, M., & New, B. (2010a). SUBTLEX-NL: A new measure for Dutch word frequency based on film subtitles. Behavior Research Methods, 42, 643–50. CrossRefPubMedGoogle Scholar
  29. Keuleers, E., Diependaele, K., & Brysbaert, M. (2010b). Practice effects in large-scale visual word recognition studies: A lexical decision study on 14,000 Dutch mono- and disyllabic words and nonwords. Frontiers in Psychology, 1, 174. CrossRefPubMedPubMedCentralGoogle Scholar
  30. Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M. (2012). The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44, 287–304. CrossRefPubMedGoogle Scholar
  31. Kliegl, R., Grabner, E., Rolfs, M., & Engbert, R. (2004). Length, frequency, and predictability effects of words on eye movements in reading. European Journal of Cognitive Psychology, 16, 262–284. CrossRefGoogle Scholar
  32. Kliegl, R., Nuthmann, A., & Engbert, R. (2006). Tracking the mind during reading: The influence of past, present, and future words on fixation durations. Journal of Experimental Psychology: General, 135, 12–35. CrossRefGoogle Scholar
  33. Kuperman, V., Dambacher, M., Nuthmann, A., & Kliegl, R. (2010). The effect of word position on eye-movements in sentence and paragraph reading. Quarterly Journal of Experimental Psychology, 63, 1838–1857. CrossRefGoogle Scholar
  34. Kuperman, V., Drieghe, D., Keuleers, E., & Brysbaert, M. (2013). How strongly do word reading times and lexical decision times correlate ? Combining data from eye movement corpora and megastudies. Quarterly Journal of Experimental Psychology, 66, 563–580. CrossRefGoogle Scholar
  35. Kuperman, V., & Van Dyke, J. A. (2011). Effects of individual differences in verbal skills on eye-movement patterns during sentence reading. Journal of Memory and Language, 65, 42–73. CrossRefPubMedPubMedCentralGoogle Scholar
  36. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. CrossRefGoogle Scholar
  37. New, B., Ferrand, L., Pallier, C., & Brysbaert, M. (2006). Reexamining the word length effect in visual word recognition: New evidence from the English Lexicon Project. Psychonomic Bulletin & Review, 13, 45–52. CrossRefGoogle Scholar
  38. Radach, R., Huestegge, L., & Reilly, R. (2008). The role of global top-down factors in local eye-movement control in reading. Psychological Research, 72, 675–688. CrossRefPubMedGoogle Scholar
  39. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422. CrossRefPubMedGoogle Scholar
  40. Rayner, K., & Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & Cognition, 14, 191–201. CrossRefGoogle Scholar
  41. Rayner, K., Pollatsek, A., Drieghe, D., Slattery, T. J., & Reichle, E. D. (2007). Tracking the mind during reading via eye movements: Comments on Kliegl, Nuthmann, and Engbert (2006). Journal of Experimental Psychology: General, 136, 520–529. CrossRefGoogle Scholar
  42. R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from Google Scholar
  43. Revelle, W. (2018). Package “psych.” Retrieved from
  44. Rubenstein, H., Garfield, L., & Millikan, J. A. (1970). Homographic entries in the internal lexicon. Journal of Verbal Learning and Verbal Behavior, 9, 487–494. CrossRefGoogle Scholar
  45. Schilling, H. E. H., Rayner, K., & Chumbley, J. I. (1998). Comparing naming, lexical decision, and eye fixation times: Word frequency effects and individual differences. Memory & Cognition, 26, 1270–1281. CrossRefGoogle Scholar
  46. Schuster, S., Hawelka, S., Hutzler, F., Kronbichler, M., & Richlan, F. (2016). Words in context: The effects of length, frequency, and predictability on brain responses during natural reading. Cerebral Cortex, 26, 3889–3904. CrossRefPubMedGoogle Scholar
  47. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlation: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. CrossRefGoogle Scholar
  48. Teng, D. W., Wallot, S., & Kelty-Stephen, D. G. (2016). Single-word recognition need not depend on single-word features: Narrative coherence counteracts effects of single-word features that lexical decision emphasizes. Journal of Psycholinguistic Research, 45, 1451–1472. CrossRefPubMedGoogle Scholar
  49. van Heuven, W. J. B., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, 39, 458–483. CrossRefGoogle Scholar
  50. van Heuven, W. J. B., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology, 67, 1176–1190. CrossRefGoogle Scholar
  51. Vitu, F., O’Regan, J. K., & Mittau, M. (1990). Optimal landing position in reading isolated words and continuous text. Perception & Psychophysics, 47, 583–600. CrossRefGoogle Scholar
  52. Whitford, V., & Titone, D. (2017). Lexical entrenchment and cross-language activation: Two sides of the same coin for bilingual reading across the adult lifespan. Bilingualism: Language and Cognition, in press.
  53. Wochna, K. L., & Juhasz, B. J. (2013). Context length and reading novel words: An eye-movement investigation. British Journal of Psychology, 104, 347–363. CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Department of Experimental PsychologyGhent UniversityGhentBelgium

Personalised recommendations