Advertisement

How well do word recognition measures correlate? Effects of language context and repeated presentations

  • Nicolas DirixEmail author
  • Marc Brysbaert
  • Wouter Duyck
Article

Abstract

In the present study we assessed the extent to which different word recognition time measures converge, using large databases of lexical decision times and eyetracking measures. We observed a low proportion of shared variance between these measures, which limits the validity of lexical decision times to real-life reading. We further investigated and compared the role of word frequency and length, two important predictors of word-processing latencies in these paradigms, and found that they influenced the measures to different extents. A second analysis of two different eyetracking corpora compared the eyetracking reading times for short paragraphs with those from reading of an entire book. Our results revealed that the correlations between eyetracking reading times of identical words in two different corpora are also low, suggesting that the higher-order language context in which words are presented plays a crucial role. Finally, our findings indicate that lexical decision times better resemble the average processing time of multiple presentations of the same word, across different language contexts.

Keywords

Visual word recognition Eyetracking Big data studies 

Notes

References

  1. Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H., & Yap, M. J. (2004). Visual word recognition of single-syllable words. Journal of Experimental Psychology: General, 133, 283–316.  https://doi.org/10.1037/0096-3445.133.2.283 CrossRefGoogle Scholar
  2. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., . . . Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39, 445–459.  https://doi.org/10.3758/BF03193014 CrossRefGoogle Scholar
  3. Boston, M. F., Hale, J. T., Kliegl, R., Patil, U., & Vasishth, S. (2008). Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus. Journal of Eye Movement Research, 2, 1–12.  https://doi.org/10.16910/jemr.2.1.1
  4. Brysbaert, M., & Cortese, M. J. (2011). Do the effects of subjective frequency and age of acquisition survive better word frequency norms? Quarterly Journal of Experimental Psychology, 64, 545–559.  https://doi.org/10.1080/17470218.2010.503374 CrossRefGoogle Scholar
  5. Brysbaert, M., Lagrou, E., & Stevens, M. (2017). Visual word recognition in a second language: A test of the lexical entrenchment hypothesis with lexical decision times. Bilingualism: Language and Cognition, 20, 530–548.  https://doi.org/10.1017/S1366728916000353 CrossRefGoogle Scholar
  6. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41, 977–990.  https://doi.org/10.3758/BRM.41.4.977 CrossRefPubMedGoogle Scholar
  7. Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1, 1–20.  https://doi.org/10.5334/joc.10 CrossRefGoogle Scholar
  8. Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). The impact of word prevalence on lexical decision times: Evidence from the Dutch Lexicon Project 2. Journal of Experimental Psychology: Human Perception and Performance, 42, 441–458.  https://doi.org/10.1037/xhp0000159 CrossRefPubMedGoogle Scholar
  9. Butler, B., & Hains, S. (1979). Individual differences in word recognition latency. Memory & Cognition, 7, 68–76.  https://doi.org/10.3758/BF03197587 CrossRefGoogle Scholar
  10. Carter, B. T., & Luke, S. G. (2018). Individuals’ eye movements in reading are highly consistent across time and trial. Journal of Experimental Psychology: Human Perception and Performance, 44, 482–492.  https://doi.org/10.1037/xhp0000471 CrossRefPubMedGoogle Scholar
  11. Cop, U., Dirix, N., Drieghe, D., & Duyck, W. (2017). Presenting GECO: An eyetracking corpus of monolingual and bilingual sentence reading. Behavior Research Methods, 49, 602–615.  https://doi.org/10.3758/s13428-016-0734-0 CrossRefPubMedGoogle Scholar
  12. Cop, U., Drieghe, D., & Duyck, W. (2015a). Eye movement patterns in natural reading: A comparison of monolingual and bilingual reading of a novel. PLOS ONE, 10, e0134008.  https://doi.org/10.1371/journal.pone.0134008 CrossRefPubMedPubMedCentralGoogle Scholar
  13. Cop, U., Keuleers, E., Drieghe, D., & Duyck, W. (2015b). Frequency effects in monolingual and bilingual natural reading. Psychonomic Bulletin & Review, 22, 1216–1234.  https://doi.org/10.3758/s13423-015-0819-2 CrossRefGoogle Scholar
  14. Courrieu, P., Brand-D’Abrescia, M., Peereman, R., Spieler, D., & Rey, A. (2011). Validated intraclass correlation statistics to test item performance models. Behavior Research Methods, 43, 37–55.  https://doi.org/10.3758/s13428-010-0020-5 CrossRefPubMedGoogle Scholar
  15. Courrieu, P., & Rey, A. (2011). Missing data imputation and corrected statistics for large-scale behavioral databases. Behavior Research Methods, 43, 310–330.  https://doi.org/10.3758/s13428-011-0071-2 CrossRefPubMedGoogle Scholar
  16. Dirix, N., Cop, U., Drieghe, D., & Duyck, W. (2017). Cross-lingual neighborhood effects in generalized lexical decision and natural reading. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43, 887–915.  https://doi.org/10.1037/xlm0000352 CrossRefPubMedGoogle Scholar
  17. Dirix, N., & Duyck, W. (2017). The first- and second-language age of acquisition effect in first- and second-language book reading. Journal of Memory and Language, 97, 103–120.  https://doi.org/10.1016/j.jml.2017.07.012 CrossRefGoogle Scholar
  18. Duyck, W., Van Assche, E., Drieghe, D., & Hartsuiker, R. J. (2007). Visual word recognition by bilinguals in a sentence context: Evidence for nonselective lexical access. Journal of Experimental Psychology. Learning, Memory, and Cognition, 33, 663–679.  https://doi.org/10.1037/0278-7393.33.4.663 CrossRefPubMedGoogle Scholar
  19. Duyck, W., Vanderelst, D., Desmet, T., & Hartsuiker, R. J. (2008). The frequency effect in second-language visual word recognition. Psychonomic Bulletin & Review, 15, 850–855.  https://doi.org/10.3758/PBR.15.4.850 CrossRefGoogle Scholar
  20. Engbert, R., Nuthmann, A., Richter, E. M., & Kliegl, R. (2005). SWIFT: A dynamical model of saccade generation during reading. Psychological Review, 112, 777–813.  https://doi.org/10.1037/0033-295X.112.4.777 CrossRefPubMedGoogle Scholar
  21. Gollan, T. H., Slattery, T. J., Goldenberg, D., Van Assche, E., Duyck, W., & Rayner, K. (2011). Frequency drives lexical access in reading but not in speaking: The frequency-lag hypothesis. Journal of Experimental Psychology. General, 140, 186–209.  https://doi.org/10.1037/a0022256 CrossRefPubMedPubMedCentralGoogle Scholar
  22. Hoedemaker, R. S., & Gordon, P. C. (2014). It takes time to prime: Semantic priming in the ocular lexical decision task. Journal of Experimental Psychology: Human Perception and Performance, 40, 2179–2197.  https://doi.org/10.1037/a0037677 CrossRefPubMedGoogle Scholar
  23. Hoedemaker, R. S., & Gordon, P. C. (2017). The onset and time course of semantic priming during rapid recognition of visual words. Journal of Experimental Psychology: Human Perception and Performance, 43, 881–902.  https://doi.org/10.1037/xhp0000377 CrossRefPubMedGoogle Scholar
  24. Hudson, P. T. W., & Bergman, M. W. (1985). Lexical knowledge in word recognition: Word length and word frequency in naming and lexical decision tasks. Journal of Memory and Language, 24, 46–58.  https://doi.org/10.1016/0749-596X(85)90015-4 CrossRefGoogle Scholar
  25. Kennedy, A., & Pynte, J. (2005). Parafoveal-on-foveal effects in normal reading. Vision Research, 45, 153–168.  https://doi.org/10.1016/j.visres.2004.07.037 CrossRefPubMedGoogle Scholar
  26. Keuleers, E., & Brysbaert, M. (2010). Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42, 627–633.  https://doi.org/10.3758/BRM.42.3.627 CrossRefPubMedGoogle Scholar
  27. Keuleers, E., & Brysbaert, M. (2011). Detecting inherent bias in lexical decision experiments with the LD1NN algorithm. Mental Lexicon, 6, 34–52.  https://doi.org/10.1075/ml.6.1.02keu CrossRefGoogle Scholar
  28. Keuleers, E., Brysbaert, M., & New, B. (2010a). SUBTLEX-NL: A new measure for Dutch word frequency based on film subtitles. Behavior Research Methods, 42, 643–50.  https://doi.org/10.3758/BRM.42.3.643 CrossRefPubMedGoogle Scholar
  29. Keuleers, E., Diependaele, K., & Brysbaert, M. (2010b). Practice effects in large-scale visual word recognition studies: A lexical decision study on 14,000 Dutch mono- and disyllabic words and nonwords. Frontiers in Psychology, 1, 174.  https://doi.org/10.3389/fpsyg.2010.00174 CrossRefPubMedPubMedCentralGoogle Scholar
  30. Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M. (2012). The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44, 287–304.  https://doi.org/10.3758/s13428-011-0118-4 CrossRefPubMedGoogle Scholar
  31. Kliegl, R., Grabner, E., Rolfs, M., & Engbert, R. (2004). Length, frequency, and predictability effects of words on eye movements in reading. European Journal of Cognitive Psychology, 16, 262–284.  https://doi.org/10.1080/09541440340000213 CrossRefGoogle Scholar
  32. Kliegl, R., Nuthmann, A., & Engbert, R. (2006). Tracking the mind during reading: The influence of past, present, and future words on fixation durations. Journal of Experimental Psychology: General, 135, 12–35.  https://doi.org/10.1037/0096-3445.135.1.12 CrossRefGoogle Scholar
  33. Kuperman, V., Dambacher, M., Nuthmann, A., & Kliegl, R. (2010). The effect of word position on eye-movements in sentence and paragraph reading. Quarterly Journal of Experimental Psychology, 63, 1838–1857.  https://doi.org/10.1080/17470211003602412 CrossRefGoogle Scholar
  34. Kuperman, V., Drieghe, D., Keuleers, E., & Brysbaert, M. (2013). How strongly do word reading times and lexical decision times correlate ? Combining data from eye movement corpora and megastudies. Quarterly Journal of Experimental Psychology, 66, 563–580.  https://doi.org/10.1080/17470218.2012.658820 CrossRefGoogle Scholar
  35. Kuperman, V., & Van Dyke, J. A. (2011). Effects of individual differences in verbal skills on eye-movement patterns during sentence reading. Journal of Memory and Language, 65, 42–73.  https://doi.org/10.1016/j.jml.2011.03.002 CrossRefPubMedPubMedCentralGoogle Scholar
  36. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.  https://doi.org/10.1037/1082-989X.1.1.30 CrossRefGoogle Scholar
  37. New, B., Ferrand, L., Pallier, C., & Brysbaert, M. (2006). Reexamining the word length effect in visual word recognition: New evidence from the English Lexicon Project. Psychonomic Bulletin & Review, 13, 45–52.  https://doi.org/10.3758/BF03193811 CrossRefGoogle Scholar
  38. Radach, R., Huestegge, L., & Reilly, R. (2008). The role of global top-down factors in local eye-movement control in reading. Psychological Research, 72, 675–688.  https://doi.org/10.1007/s00426-008-0173-3 CrossRefPubMedGoogle Scholar
  39. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422.  https://doi.org/10.1037/0033-2909.124.3.372 CrossRefPubMedGoogle Scholar
  40. Rayner, K., & Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & Cognition, 14, 191–201.  https://doi.org/10.3758/BF03197692 CrossRefGoogle Scholar
  41. Rayner, K., Pollatsek, A., Drieghe, D., Slattery, T. J., & Reichle, E. D. (2007). Tracking the mind during reading via eye movements: Comments on Kliegl, Nuthmann, and Engbert (2006). Journal of Experimental Psychology: General, 136, 520–529.  https://doi.org/10.1037/0096-3445.136.3.520 CrossRefGoogle Scholar
  42. R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org Google Scholar
  43. Revelle, W. (2018). Package “psych.” Retrieved from http://personality-project.org/r/psych/psych-manual.pdf
  44. Rubenstein, H., Garfield, L., & Millikan, J. A. (1970). Homographic entries in the internal lexicon. Journal of Verbal Learning and Verbal Behavior, 9, 487–494.  https://doi.org/10.1016/S0022-5371(70)80091-3 CrossRefGoogle Scholar
  45. Schilling, H. E. H., Rayner, K., & Chumbley, J. I. (1998). Comparing naming, lexical decision, and eye fixation times: Word frequency effects and individual differences. Memory & Cognition, 26, 1270–1281.  https://doi.org/10.3758/BF03201199 CrossRefGoogle Scholar
  46. Schuster, S., Hawelka, S., Hutzler, F., Kronbichler, M., & Richlan, F. (2016). Words in context: The effects of length, frequency, and predictability on brain responses during natural reading. Cerebral Cortex, 26, 3889–3904.  https://doi.org/10.1093/cercor/bhw184 CrossRefPubMedGoogle Scholar
  47. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlation: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.  https://doi.org/10.1037/0033-2909.86.2.420 CrossRefGoogle Scholar
  48. Teng, D. W., Wallot, S., & Kelty-Stephen, D. G. (2016). Single-word recognition need not depend on single-word features: Narrative coherence counteracts effects of single-word features that lexical decision emphasizes. Journal of Psycholinguistic Research, 45, 1451–1472.  https://doi.org/10.1007/s10936-016-9416-4 CrossRefPubMedGoogle Scholar
  49. van Heuven, W. J. B., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, 39, 458–483.  https://doi.org/10.1006/jmla.1998.2584 CrossRefGoogle Scholar
  50. van Heuven, W. J. B., Mandera, P., Keuleers, E., & Brysbaert, M. (2014). SUBTLEX-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology, 67, 1176–1190.  https://doi.org/10.1080/17470218.2013.850521 CrossRefGoogle Scholar
  51. Vitu, F., O’Regan, J. K., & Mittau, M. (1990). Optimal landing position in reading isolated words and continuous text. Perception & Psychophysics, 47, 583–600.  https://doi.org/10.3758/BF03203111 CrossRefGoogle Scholar
  52. Whitford, V., & Titone, D. (2017). Lexical entrenchment and cross-language activation: Two sides of the same coin for bilingual reading across the adult lifespan. Bilingualism: Language and Cognition, in press.  https://doi.org/10.1017/S1366728917000554
  53. Wochna, K. L., & Juhasz, B. J. (2013). Context length and reading novel words: An eye-movement investigation. British Journal of Psychology, 104, 347–363.  https://doi.org/10.1111/j.2044-8295.2012.02127.x CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Department of Experimental PsychologyGhent UniversityGhentBelgium

Personalised recommendations