LAB: Linguistic Annotated Bibliography – a searchable portal for normed database information

  • Erin M. BuchananEmail author
  • K. D. Valentine
  • Nicholas P. Maxwell


This article presents the Linguistic Annotated Bibliography (LAB) as a searchable Web portal to quickly and easily access reliable database norms, related programs, and variable calculations. These publications were coded by language, number of stimuli, stimuli type (i.e., words, pictures, symbols), keywords (i.e., frequency, semantics, valence), and other useful information. This tool not only allows researchers to search for the specific type of stimuli needed for experiments but also permits the exploration of publication trends across 100 years of research. Details about the portal creation and use are outlined, as well as various analyses of change in publication rates and keywords. In general, advances in computational power have allowed for the increase in dataset size in the recent decades, in addition to an increase in the number of linguistic variables provided in each publication.


Database Stimuli Online portal Megastudy Trends 



Erin M. Buchanan is an Associate Professor of Quantitative Psychology at Missouri State University. K. D. Valentine is a Ph.D. candidate at the University of Missouri. Nicholas P. Maxwell received his master’s degree from Missouri State University and is now a Ph.D. candidate at the University of Southern Mississippi. We thank Michael T. Carr, Farren E. Bankovich, Samantha D. Saxton, and Emmanuel Segui for their help with the original data processing, Bodo Winter and an anonymous reviewer for their comments on the manuscript, and William Padfield, Abigial Van Nuland, and Addie Wikowsky for their help with the application development for the Web site.


  1. Adelman, J. S., Brown, G. D., & Quesada, J. F. (2006). Contextual diversity, not word frequency, determines word-naming and lexical decision times. Psychological Science, 17(9), 814–823. CrossRefGoogle Scholar
  2. Aust, F., & Barth, M. (2017). papaja: Create APA manuscripts with R Markdown. Retrieved from
  3. Baayen, R. H., Piepenbrock, R., Gulikers, L., & Linguistic Data Consortium (1995). The CELEX lexical database (CD-ROM). Philadelphia.Google Scholar
  4. Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H., & Yap, M. J. (2004). Visual word recognition of single-syllable words. Journal of Experimental Psychology: General, 133(2), 283–316. CrossRefGoogle Scholar
  5. Balota, D. A., Yap, M. J., Hutchison, K. A., Cortese, M. J., Kessler, B., Loftis, B., & Treiman, R. (2007). The English lexicon project. Behavior Research Methods, 39(3), 445–459. CrossRefGoogle Scholar
  6. Barca, L., Burani, C., & Arduino, L. S. (2002). Word naming times and psycholinguistic norms for Italian nouns. Behavior Research Methods, Instruments, & Computers, 34(3), 424–434. CrossRefGoogle Scholar
  7. Boudelaa, S., & Marslen-Wilson, W. D. (2010). Aralex: A lexical database for modern standard Arabic. Behavior Research Methods, 42(2), 481–487. CrossRefGoogle Scholar
  8. Bradshaw, J. L. (1984). A guide to norms, ratings, and lists. Memory & Cognition, 12(2), 202–206. CrossRefGoogle Scholar
  9. Brodeur, M. B., Dionne-Dostie, E., Montreuil, T., & Lepage, M. (2010). The bank of standardized stimuli (BOSS), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research. PLoS ONE, 5(5), e10773. CrossRefGoogle Scholar
  10. Brysbaert, M., & New, B. (2009). Moving beyond Kuč,era and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977–990. CrossRefGoogle Scholar
  11. Brysbaert, M., Buchmeier, M., Conrad, M., Jacobs, A. M., Bölte, J., & Böhl, A. (2011). The word frequency effect: A review of recent developments and implications for the choice of frequency estimates in German. Experimental Psychology, 58(5), 412–424. CrossRefGoogle Scholar
  12. Brysbaert, M., Warriner, A. B., & Kuperman, . V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904–911. CrossRefGoogle Scholar
  13. Buchanan, E. M., Holmes, J. L., Teasley, M. L., & Hutchison, K. A. (2013). English semantic word-pair norms and a searchable Web portal for experimental stimulus creation. Behavior Research Methods, 45(3), 746–757. CrossRefGoogle Scholar
  14. Buchanan, E. M., & Scofield, J. E. (2018). Methods to detect low-quality data and its implication for psychological research. Behavior Research Methods.
  15. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. CrossRefGoogle Scholar
  16. Burgess, C., & Livesay, K. (1998). The effect of corpus size in predicting reaction time in a basic word recognition task: Moving on from Kuč,era and Francis. Behavior Research Methods, Instruments, and Computers, 30(2), 272–277. CrossRefGoogle Scholar
  17. Cai, Q., & Brysbaert, M. (2010). SUBTLEX-CH: Chinese word and character frequencies based on film subtitles. PLoS ONE, 5(6), e10729. CrossRefGoogle Scholar
  18. Chang, W., Cheng, J., Allaire, J., Xie, Y., & McPherson, J. (2017). Shiny: Web application framework for R. Retrieved from
  19. Cohen-Shikora, E. R., Balota, D. A., Kapuria, A., & Yap, M. J. (2013). The past tense inflection project (PTIP): Speeded past tense inflections, imageability ratings, and past tense consistency measures for 2,200 verbs. Behavior Research Methods, 45(1), 151–159. CrossRefGoogle Scholar
  20. Cree, G. S., & McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). Journal of Experimental Psychology: General, 132(2), 163–201. CrossRefGoogle Scholar
  21. Cree, G. S., McRae, K., & McNorgan, C. (1999). An attractor model of lexical conceptual processing: Simulating semantic priming. Cognitive Science, 23, 371–414. CrossRefGoogle Scholar
  22. Cuetos, F., Glez-Nosti, M., Barbon, A., & Brysbaert, M. (2011). SUBTLEX-ESP: Spanish word frequencies based on film subtitles. Psicologica, 32, 133–143.Google Scholar
  23. De Deyne, S., Navarro, D. J., & Storms, G. (2013). Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations. Behavior Research Methods, 45(2), 480–498. CrossRefGoogle Scholar
  24. Dimitropoulou, M., Duñabeitia, J. A., Avilés, A., Corral, J., & Carreiras, M. (2010). Subtitle-based word frequencies as the best estimate of reading behavior: The case of Greek. Frontiers in Psychology, 1(DEC), 1–12. Google Scholar
  25. Dodds, P. S., Harris, K. D., Kloumann, I. M., Bliss, C. A., & Danforth, C. M. (2011). Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PLoS ONE, 6(12), e26752. CrossRefGoogle Scholar
  26. Dufau, S., Duñabeitia, J. A., Moret-Tatay, C., McGonigal, A., Peeters, D., Alario, F. X., & Grainger, J. (2011). Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PLoS ONE, 6(9), e24974. CrossRefGoogle Scholar
  27. Guasch, M., Boada, R., Ferré, P., & Sánchez-Casas, R. (2013). NIM: A Web-based Swiss army knife to select stimuli for psycholinguistic studies. Behavior Research Methods, 45(3), 765–771. CrossRefGoogle Scholar
  28. Hammarstrom, F. H. (n.d.) Glottolog 3.3. Retrieved from
  29. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61–83. CrossRefGoogle Scholar
  30. Heuven, W. J. B., van Mandera, P., Keuleers, E., & Brysbaert, M. (2014). Subtlex-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology, 67(6), 1176–1190. CrossRefGoogle Scholar
  31. Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse, C. -S., & Buchanan, E. M. (2013). The semantic priming project. Behavior Research Methods, 45(4), 1099–1114. CrossRefGoogle Scholar
  32. Kent, G. H., & Rosanoff, A. J. (1910). A study of association in insanity. American Journal of Insanity, 67, 37–96. Google Scholar
  33. Keuleers, E., Brysbaert, M., & New, B. (2010). SUBTLEX-NL: A new measure for Dutch word frequency based on film subtitles. Behavior Research Methods, 42(3), 643–650. CrossRefGoogle Scholar
  34. Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M. (2012). The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44(1), 287–304. CrossRefGoogle Scholar
  35. Kloumann, I. M., Danforth, C. M., Harris, K. D., Bliss, C. A., & Dodds, P. S. (2012). Positivity of the English language. PLoS ONE, 7(1), e29484. CrossRefGoogle Scholar
  36. Kučera, H., & Francis, W. N. (1967) Computational analysis of present-day American English. Providence: Brown University Press.Google Scholar
  37. Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012). Age-of-acquisition ratings for 30,000 English words. Behavior Research Methods, 44(4), 978–990. CrossRefGoogle Scholar
  38. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211–240. CrossRefGoogle Scholar
  39. Lété, B., Sprenger-Charolles, L., & Colé, P. (2004). MANULEX: A grade-level lexical database from French elementary school readers. Behavior Research Methods, Instruments, & Computers, 36(1), 156–166. CrossRefGoogle Scholar
  40. List, J.-M., Winter, B., & Wedel, A. (n.d.) The Language Goldmine. Retrieved from
  41. Maki, W. S., McKinley, L. N., & Thompson, A. G. (2004). Semantic distance norms computed from an electronic dictionary (WordNet). Behavior Research Methods, Instruments, & Computers, 36(3), 421–431. CrossRefGoogle Scholar
  42. Mandera, P., Keuleers, E., Wodniecka, Z., & Brysbaert, M. (2015). Subtlex-pl: Subtitle-based word frequency estimates for Polish. Behavior Research Methods, 47(2), 471–483. CrossRefGoogle Scholar
  43. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44(1), 1–23. CrossRefGoogle Scholar
  44. McRae, K., Sa, V. R., & Seidenberg, M. S. (1997). On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology: General, 126(2), 99–130. CrossRefGoogle Scholar
  45. Miller, G. A. (2003). The cognitive revolution: a historical perspective. Trends in Cognitive Sciences, 7, 141–144. CrossRefGoogle Scholar
  46. Moss, H. E., Tyler, L. K., Devlin, J. T, & Devlin, J. T. (2002). The emergence of category-specific deficits in a distributed semantic system E. Forde, G. Humphreys, H. E. Moss, & L. K. Tyler (Eds.) In Forde, E., Humphreys, G., Moss, H. E., & Tyler, L. K. (Eds.) Category-specificity in mind and brain (pp. 115–145). CRC Press.Google Scholar
  47. Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The University of South Florida free association, rhyme, and word fragment norms. Behavior Research Methods, Instruments, & Computers, 36(3), 402–407. CrossRefGoogle Scholar
  48. New, B., Brysbaert, M., Veronis, J., & Pallier, C. (2007). The use of film subtitles to estimate word frequencies. Applied Psycholinguistics, 28(4), 661–677. Scholar
  49. Pexman, P. M., Holyk, G. G., & Monfils, M. -H. (2003). Number-of-features effects and semantic processing. Memory & Cognition, 31(6), 842–855. CrossRefGoogle Scholar
  50. Postman, L., & Keppel, G. (1970) Norms of word association. New York: Academic Press.Google Scholar
  51. Proctor, R. W., & Vu, K. -P. L (1999). Index of norms and ratings published in the Psychonomic Society journals. Behavior Research Methods, Instruments, & Computers, 31(4), 659–667. CrossRefGoogle Scholar
  52. Rayner, K., & Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & Cognition, 14(3), 191–201. CrossRefGoogle Scholar
  53. Rogers, T. T., & McClelland, J. L. (2004) Semantic cognition: A parallel distributed processing approach. Cambridge: MIT Press.Google Scholar
  54. Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6 (2), 174–215. Google Scholar
  55. Soares, A. P., Medeiros, J. C., Simões, A., Machado, J., Costa, A., Iriarte, Á., & Scomesaña, M. (2014). ESCOLEX: A grade-level lexical database from European Portuguese elementary to middle school textbooks. Behavior Research Methods, 46(1), 240–253. CrossRefGoogle Scholar
  56. Sze, W. P., Rickard Liow, S. J., & Yap, M. J. (2014). The Chinese Lexicon Project: A repository of lexical decision behavioral responses for 2,500 Chinese characters. Behavior Research Methods, 46(1), 263–273. CrossRefGoogle Scholar
  57. Tse, C.-S., Yap, M. J., Chan, Y.-L., Sze, W. P., Shaoul, C., & Lin, D. (2017). The Chinese Lexicon Project: A megastudy of lexical decision performance for 25,000+ traditional Chinese two-character compound words. Behavior Research Methods, 49(4), 1503–1519. CrossRefGoogle Scholar
  58. Vaughan, J. (2004). A web-based archive of norms, stimuli, and data. Behavior Research Methods, Instruments, & Computers, 36(3), 363–370. CrossRefGoogle Scholar
  59. Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. F. (2004). Representing the meanings of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology, 48(4), 422–488. CrossRefGoogle Scholar
  60. Vinson, D. P., Vigliocco, G., Cappa, S., & Siri, S. (2003). The breakdown of semantic knowledge: Insights from a statistical model of meaning representation. Brain and Language, 86(3), 347–365. CrossRefGoogle Scholar
  61. Vo, M. L. H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M. J., & Jacobs, A. M. (2009). The Berlin Affective Word List Reloaded (BAWL-R). Behavior Research Methods, 41(2), 534–538. CrossRefGoogle Scholar
  62. Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45(4), 1191–1207. CrossRefGoogle Scholar
  63. Yap, M. J., Rickard Liow, S. J., Jalil, S. B., & Faizal, S. S. B. (2010). The Malay Lexicon Project: A database of lexical statistics for 9,592 words. Behavior Research Methods, 42(4), 992–1003. CrossRefGoogle Scholar
  64. Yap, M. J., Tan, S. E., Pexman, P. M., & Hargreaves, I. S. (2011). Is more always better? Effects of semantic richness on lexical decision, speeded pronunciation, and semantic classification. Psychonomic Bulletin and Review, 18(4), 742–750. CrossRefGoogle Scholar
  65. Zevin, J., & Seidenberg, M. (2002). Age of acquisition effects in word reading and other tasks. Journal of Memory and Language, 47(1), 1–29. CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Erin M. Buchanan
    • 1
    Email author
  • K. D. Valentine
    • 2
  • Nicholas P. Maxwell
    • 1
  1. 1.Missouri State UniversitySpringfieldUSA
  2. 2.University of MissouriColumbiaUSA

Personalised recommendations