Advertisement

The Massive Auditory Lexical Decision (MALD) database

  • Benjamin V. TuckerEmail author
  • Daniel Brenner
  • D. Kyle Danielson
  • Matthew C. Kelley
  • Filip Nenadić
  • Michelle Sims
Article

Abstract

The Massive Auditory Lexical Decision (MALD) database is an end-to-end, freely available auditory and production data set for speech and psycholinguistic research, providing time-aligned stimulus recordings for 26,793 words and 9592 pseudowords, and response data for 227,179 auditory lexical decisions from 231 unique monolingual English listeners. In addition to the experimental data, we provide many precompiled listener- and item-level descriptor variables. This data set makes it easy to explore responses, build and test theories, and compare a wide range of models. We present summary statistics and analyses.

Keywords

Megastudy Auditory lexical decision Spoken word recognition 

Notes

Acknowledgements

This research was funded by SSHRC Grant #435-2014-0678 and by a University of Alberta Killam Research Grant, both to the first author. It also benefited greatly from planning consultation with R. Harald Baayen, and organizational, subject-running, coding, and markup contributions by Kara Hawthorne, Danielle Fonseca, Catherine Ford, Pearl Lorentzen, and Katelynn Pawlenchuk. Thanks also to Emmanuel Keuleers for adapting Wuggy for our pseudoword creation. Correspondence may be addressed to Benjamin V. Tucker, 4-32 Assiniboia Hall, Department of Linguistics, University of Alberta, Edmonton, Alberta, T6G2E7, Canada (e-mail: bvtucker@ualberta.ca).

References

  1. Abdi, H. (2007). Signal detection theory (SDT). Encyclopedia of measurement and statistics (pp. 886–889).Google Scholar
  2. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Czaki. Budapest: Akademiai Kiado.Google Scholar
  3. Andrews, S. (1997). The effect of orthographic similarity on lexical retrieval: resolving neighborhood conflicts. Psychonomic Bulletin & Review, 4(4), 439–461. https://link.springer.com/article/10.3758/BF03214334 CrossRefGoogle Scholar
  4. Antworth, E. L. (1994). Morphological parsing with a unification-based word grammar. In: Proceedings of the North Texas Natural Language Processing Workshop (pp. 24–32).Google Scholar
  5. Antworth, E.L. (1995). User’s guide to pc-kimmo version 2. Página web]. Disponible en http://www.sil.org/pckimmo/v2/doc/guide.html
  6. Baayen, H., Vasishth, S., Kliegl, R., & Bates, D. (2017). The cave of shadows: addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. http://www.sciencedirect.com/science/article/pii/S0749596X16302467 CrossRefGoogle Scholar
  7. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.CrossRefGoogle Scholar
  8. Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995) The CELEX lexical database (CD-ROM). Linguistic Data Consortium. Philadelphia: University of Pennsylvania.Google Scholar
  9. Balota, D. A., Yap, M. J., & Cortese, M. J. (2006). Handbook of psycholinguistics (pp. 285–375). Academic Press, ch. Visual Word Recognition (Ch. 9).Google Scholar
  10. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. I., Kessler, B., Loftis, B., ..., Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459.CrossRefPubMedGoogle Scholar
  11. Balota, D. A., Yap, M. J., Hutchison, K. A., & Cortese, M. J. (2012). Megastudies: What do millions (or so) of trials tell us about lexical processing. Visual word recognition volume 1: Models and methods, orthography and phonology, 90. https://books.google.ca/books?hl=en&lr=&id=uco5lasTR2oC&oi=fnd&pg=PA90&dq=Megastudies:+What+do+millions+(or+so)+of+trials+tell+us+about+lexical+processing&ots=azU5QTJobe&sig=CFZNClqnhGkjOgPTXsYpZPTiKYc
  12. Boersma, P., & Weenink, D. (2011). Praat, a system for doing phonetics by computer. www.praat.org
  13. Bradley, D. C., & Forster, K. I. (1987). A reader’s view of listening. Cognition, 25, 103–134.CrossRefPubMedGoogle Scholar
  14. Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: a critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977–990.CrossRefPubMedGoogle Scholar
  15. Brysbaert, M., New, B., & Keuleers, E. (2012). Adding part-of-speech information to the SUBTLEX-US word frequencies. Behavior Research Methods, 44(4), 991–997.CrossRefPubMedGoogle Scholar
  16. Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). The impact of word prevalence on lexical decision times: evidence from the Dutch Lexicon Project 2. Journal of Experimental Psychology: Human Perception and Performance, 42(3), 441.PubMedGoogle Scholar
  17. Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904–911.CrossRefPubMedGoogle Scholar
  18. Campbell, D. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.CrossRefPubMedGoogle Scholar
  19. Chee, M. W., O’Craven, K. M., Bergida, R., Rosen, B. R., & Savoy, R. L. (1999). Auditory and visual word processing studied with fMRI. Human Brain Mapping, 7(1), 15–28.CrossRefPubMedGoogle Scholar
  20. Cohen, J. (1962). The statistical power of abnormal-social psychological research: a review. The Journal of Abnormal and Social Psychology, 65(3), 145.CrossRefPubMedGoogle Scholar
  21. Coltheart, M., Davelaar, E., Jonasson, J., & Besner, D. (1977). Access to the internal lexicon. In Dornic, S. (Ed.) Attention and Performance VI (pp. 535–555). Hillsdale: Lawrence Erlbaum Associates. http://www.maccs.mq.edu.au/max/cv/#four.
  22. Cutler, A. (1981). Making up materials is a confounded nuisance, or: will we able to run any psycholinguistic experiments at all in 1990? Cognition, 10 (1-3), 65–70. http://pubman.mpdl.mpg.de/pubman/faces/viewItemFullPage.jsp?itemId=escidoc:68678 CrossRefPubMedGoogle Scholar
  23. Cutler, A. (2012) Native listening: language experience and the recognition of spoken words. Cambridge: MIT Press.Google Scholar
  24. Davies, M. (2009). The 385+ million word Corpus of Contemporary American English (19902008+): design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14(2), 159–190.CrossRefGoogle Scholar
  25. Dufau, S., Grainger, J., & Ziegler, J. C. (2012). How to say “no” to a nonword: A leaky competing accumulator model of lexical decision. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(4), 1117.PubMedGoogle Scholar
  26. Ernestus, M., & Cutler, A. (2015). BALDEY: A database of auditory lexical decisions. The Quarterly Journal of Experimental Psychology, 68(8), 1469–1488.  https://doi.org/10.1080/17470218.2014.984730 CrossRefPubMedGoogle Scholar
  27. Ferrand, L., Mot, A., Spinelli, E., New, B., Pallier, C., Bonin, P., ..., Grainger, J. (2017). MEGALEX: A megastudy of visual and auditory word recognition. Behavior Research Methods, 1–23. https://link.springer.com/article/10.3758/s13428-017-0943-1
  28. Ferrand, L., New, B., Brysbaert, M., Keuleers, E., Bonin, P., Méot, A., ..., Pallier, C. (2010). The French Lexicon Project: lexical decision data for 38,840 French words and 38,840 pseudowords. Behavior Research Methods, 42(2), 488–496.CrossRefPubMedGoogle Scholar
  29. Forster, K., Mohan, K., & Hector, J. (2003). Masked priming: State of the art (pp. 3–37). New York: Psychology Press Ch, The Mechanics of Masked Priming (Ch. 1).Google Scholar
  30. Forster, K. I. (1976). Accessing the mental lexicon. In R. J. Wales, & E Walker (Eds.) New approaches to language mechanisms (pp. 257–287). Amsterdam: A collection of psycholinguistic studies.Google Scholar
  31. Forster, K. I. (2000). The potential for experimenter bias effects in word recognition experiments. Memory & Cognition, 28, 1109–1115.CrossRefGoogle Scholar
  32. Goh, W. D., Yap, M. J., Lau, M. C., Ng, M. M. R., & Tan, L.-C. (2016). Semantic Richness Effects in Spoken Word Recognition: A Lexical Decision and Semantic Categorization Megastudy. Frontiers in Psychology 7. https://www.frontiersin.org/articles/10.3389/fpsyg.2016.00976/full
  33. Hoffman, P., Ralph, M. A. L., & Rogers, T. T. (2013). Semantic diversity: a measure of semantic ambiguity based on variability in the contextual usage of words. Behavior Research Methods, 45(3), 718–730.CrossRefPubMedGoogle Scholar
  34. Ioannidis, J. P. (2005). Why most published research findings are false. PLos Med, 2(8), 0696–0701.CrossRefGoogle Scholar
  35. Jarema, G., Libben, G., & Tucker, B.V. (2015). The integration of phonological and phonetic processing: a matter of sound judgment G. Jarema, & G. Libben (Eds.) Benjamins Current Topics (Vol. 80, pp. 1–14). Amsterdam: John Benjamins Publishing Company.  https://doi.org/10.1075/bct.80.002int. https://benjamins.com/catalog/bct.80.002int
  36. Jusezyk, P. W., & Luce, P. A. (2002). Speech perception and spoken word recognition: past and present. Ear and Hearing, 23(1), 2–40.CrossRefPubMedGoogle Scholar
  37. Keuleers, E., & Balota, D.A. (2015). Megastudies, crowdsourcing, and large datasets in psycholinguistics: An overview of recent developments. The Quarterly Journal of Experimental Psychology, 68(8), 1457–1468.  https://doi.org/10.1080/17470218.2015.1051065 CrossRefPubMedGoogle Scholar
  38. Keuleers, E., & Brysbaert, M. (2010). Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42(3), 627–633. http://link.springer.com/article/10.3758/BRM.42.3.627 CrossRefPubMedGoogle Scholar
  39. Keuleers, E., Diependaele, K., & Brysbaert, M. (2010). Practice effects in large-scale visual word recognition studies: a lexical decision study on 14,000 Dutch mono- and disyllabic words and nonwords. Frontiers in Language Sciences 1, 174. http://www.frontiersin.org/language_sciences/10.3389/fpsyg.2010.00174/abstract
  40. Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M. (2012). The British Lexicon Project: lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44(1), 287–304.CrossRefPubMedGoogle Scholar
  41. Keuleers, E., Stevens, M., Mandera, P., & Brysbaert, M. (2015). Word knowledge in the crowd: measuring vocabulary size and word prevalence in a massive online experiment. The Quarterly Journal of Experimental Psychology, 68(8), 1665–1692.  https://doi.org/10.1080/17470218.2015.1022560 CrossRefPubMedGoogle Scholar
  42. Kuperman, V. (2015). Virtual experiments in megastudies: a case study of language and emotion. The Quarterly Journal of Experimental Psychology, 68(8), 1693–1710.  https://doi.org/10.1080/17470218.2014.989865 CrossRefPubMedPubMedCentralGoogle Scholar
  43. Levenshtein, V. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics – Doklady, 10, 707–710.Google Scholar
  44. Luce, P.A., & Pisoni, D.B. (1998). Recognizing spoken words: the neighborhood activation model. Ear and Hearing, 19(1), 1–36. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3467695/ CrossRefPubMedPubMedCentralGoogle Scholar
  45. Mandera, P., Keuleers, E., & Brysbaert, M. (2017). Explaining human performance in psycholinguistic tasks with models of semantic similarity based on prediction and counting: a review and empirical validation. Journal of Memory and Language, 92, 57–78. http://www.sciencedirect.com/science/article/pii/S0749596X16300079 CrossRefGoogle Scholar
  46. Mattys, S.L. (1997). The use of time during lexical processing and segmentation: a review. Psychonomic Bulletin & Review, 4(3), 310–329. http://link.springer.com/article/10.3758/BF03210789 CrossRefGoogle Scholar
  47. Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? what does ”failure to replicate” really mean? American Psychologist, 70(6), 487.CrossRefPubMedGoogle Scholar
  48. McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37(4), 547–559.CrossRefPubMedGoogle Scholar
  49. Michel, J. -B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M., The Google Books Team, ..., Aiden, E.L. (2011). Quantitative analysis of culture using millions of digitized books. Science, 331(6014), 176–182.CrossRefPubMedGoogle Scholar
  50. New, B. et al., (2006). Reexamining the word length effect in visual word recognition: new evidence from the English Lexicon Project. Psychonomic Bulletin & Review, 13(1), 45–52.CrossRefGoogle Scholar
  51. Norris, D. (2013). Models of visual word recognition. Trends in Cognitive Sciences, 17(10), 517–524.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Norris, D., & McQueen, J.M. (2008). Shortlist B: a Bayesian model of continuous speech recognition. Psychological Review, 115(2), 357–395. http://www.ncbi.nlm.nih.gov/pubmed/18426294 CrossRefPubMedGoogle Scholar
  53. Pastore, R., & Scheirer, C. (1974). Signal detection theory: considerations for general application. Psychological Bulletin, 81(12), 945–958.CrossRefGoogle Scholar
  54. Pitt, M. A., Dilley, L., Johnson, K., Kiesling, S., Raymond, W., Hume, E., & Fosler-Lussier, E. (2007). Buckeye Corpus of Conversational Speech (2nd release) [www.buckeyecorpus.osu.edu] Columbus, OH: Department of Psychology. Ohio State University (Distributor).
  55. Radach, R., & Kennedy, A. (2013). Eye movements in reading: some theoretical context. The Quarterly Journal of Experimental Psychology, 66(3), 429–452.  https://doi.org/10.1080/17470218.2012.750676 CrossRefPubMedGoogle Scholar
  56. Rayner, K., Chace, K.H., Slattery, T. J., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading, 10(3), 241–255.  https://doi.org/10.1207/s1532799xssr1003_3 CrossRefGoogle Scholar
  57. Rayner, K., & Clifton, C. (2009). Language processing in reading and speech perception is fast and incremental: implications for event-related potential research. Biological Psychology, 80(1), 4–9.CrossRefPubMedGoogle Scholar
  58. Schmidtke, D., Gagn, C. L., Kuperman, V., Spalding, T. L., & Tucker, B. V. (2018). Conceptual relations compete during auditory and visual compound word recognition. Language, Cognition and Neuroscience. http://www.tandfonline.com/doi/abs/10.1080/23273798.2018.1437192
  59. Schneider, W., Eschman, A., & Zuccolotto, A. (2012) E-Prime Reference guide. Pittsburgh: Psychology Software Tools Inc.Google Scholar
  60. Schröter, P., & Schroeder, S. (2017). The developmental lexicon project: a behavioral database to investigate visual word recognition across the lifespan. Behavior Research Methods, 1–21.  https://doi.org/10.3758/s13428-016-0851-9
  61. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105(2), 309.CrossRefGoogle Scholar
  62. Seidenberg, M. S., & Waters, G. S. (1989). Reading words aloud—a mega study. Bulletin of the Psychonomic Society, 27(6), 489–489.Google Scholar
  63. Shadish, W. R. (1993). Critical multiplism: a research strategy and its attendant tactics. New Directions for Evaluation, 1993(60), 13–57.CrossRefGoogle Scholar
  64. Shaoul, C., & Westbury, C. (2010). Exploring lexical co-occurrence space using HiDEx. Behavior Research Methods, 42(2), 393–413.CrossRefPubMedGoogle Scholar
  65. Smits, R., Warner, N., McQueen, J.M., & Cutler, A. (2003). Unfolding of phonetic information over time: a database of Dutch diphone perception. The Journal of the Acoustical Society of America, 113(1), 563–574.CrossRefPubMedGoogle Scholar
  66. Taft, M. (1986). Lexical access codes in visual and auditory word recognition. Language and Cognitive Processes, 1(4), 297–308.CrossRefGoogle Scholar
  67. ten Bosch, L., Boves, L., & Ernestus, M. (2013). Towards an end-to-end computational model of speech comprehension: simulating a lexical decision task. In: INTERSPEECH 2013: 14th Annual Conference of the International Speech Communication Association. pp. 2822–2826. http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:1835410
  68. ten Bosch, L., Boves, L., & Ernestus, M. (2015a). DIANA, an end-to-end computational model of human word comprehension. In: 18th International Congress of Phonetic Sciences (ICPhS 2015). University of Glasgow. http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:2181015
  69. ten Bosch, L., Boves, L., Tucker, B., & Ernestus, M. (2015b). DIANA: towards computational modeling reaction times in lexical decision in North American English. In: Interspeech 2015: 16th Annual Conference of the International Speech Communication Association. pp. 1576–1580. http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:2230858
  70. ten Bosch, L., Ernestus, M., & Boves, L. (2014). Comparing reaction time sequences from human participants and computational models. In: Interspeech 2014: 15th Annual Conference of the International Speech Communication Association. pp. 462–466. http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:2058455
  71. Tse, C.-S., Yap, M. J., Chan, Y.-L., Sze, W. P., Shaoul, C., & Lin, D. (2016). The Chinese Lexicon Project: a megastudy of lexical decision performance for 25,000+ traditional Chinese two-character compound words. Behavior Research Methods, 1–17.  https://doi.org/10.3758/s13428-016-0810-5
  72. Tucker, B. V., & Ernestus, M. (2016). Why we need to investigate casual speech to truly understand language production, processing and the mental lexicon. The Mental Lexicon, 11 (3), 375–400. http://www.jbe-platform.com/content/journals/10.1075/ml.11.3.03tuc CrossRefGoogle Scholar
  73. Vitevitch, M. S., & Luce, P. A. (1998). When words compete: levels of processing in perception of spoken words. Psychological Science, 9(4), 325–329. http://www.jstor.org/stable/40063346 CrossRefGoogle Scholar
  74. Warner, N., Clayton, I. D., Carnie, A., Fisher, M., Brenner, D., & Hammond, M. (2014). The effect of Gaelic initial consonant mutation on spoken word recognition. In: Celtic linguistics conference (Vol. 8). Edinburgh, UK, poster presentation.Google Scholar
  75. Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45(4), 1191–1207.CrossRefPubMedGoogle Scholar
  76. Weide, R. (2005). The Carnegie Mellon Pronouncing Dictionary [cmudict. 0.6]. Carnegie Mellon University: http://www.speech.cs.cmu.edu/cgi-bin/cmudict. Accessed.
  77. Yap, M. J., Balota, D. A., Sibley, D. E., & Ratcliff, R. (2012). Individual differences in visual word recognition: insights from the English Lexicon Project. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 53.PubMedGoogle Scholar
  78. Yap, M. J., Sibley, D. E., Balota, D. A., Ratcliff, R., & Rueckl, J. (2015). Responding to nonwords in the lexical decision task: insights from the English Lexicon Project. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 597.PubMedGoogle Scholar
  79. Yarkoni, T., Balota, D., & Yap, M. (2008). Moving beyond Coltheart’s N: A new measure of orthographic similarity. Psychonomic Bulletin & Review, 15(5), 971–979.CrossRefGoogle Scholar
  80. Yates, M., Locker, L., & Simpson, G. B. (2004). The influence of phonological neighborhood on visual word perception. Psychonomic Bulletin & Review, 11(3), 452–457. https://link.springer.com/article/10.3758/BF03196594 CrossRefGoogle Scholar
  81. Yuan, J., & Liberman, M. (2008). Speaker identification on the SCOTUS corpus. Proceedings of Acoustics.Google Scholar
  82. Ziegler, J. C., Muneaux, M., & Grainger, J. (2003). Neighborhood effects in auditory word recognition: phonological competition and orthographic facilitation. Journal of Memory and Language, 48(4), 779–793. http://www.sciencedirect.com/science/article/pii/S0749596X03000068 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Benjamin V. Tucker
    • 1
    Email author
  • Daniel Brenner
    • 1
  • D. Kyle Danielson
    • 2
  • Matthew C. Kelley
    • 1
  • Filip Nenadić
    • 1
  • Michelle Sims
    • 1
  1. 1.Department of LinguisticsUniversity of AlbertaEdmontonCanada
  2. 2.University of TorontoTorontoCanada

Personalised recommendations