Behavior Research Methods

, Volume 41, Issue 1, pp 13–19 | Cite as

Using the World-Wide Web to obtain large-scale word norms: 190,212 ratings on a set of 2,654 German nouns

  • Olaf Lahl
  • Anja S. Göritz
  • Reinhard Pietrowsky
  • Jessica Rosenberg


This article presents a new database of 2,654 German nouns rated by a sample of 3,907 subjects on three psycholinguistic attributes: concreteness, valence, and arousal. As a new means of data collection in the field of psycholinguistic research, all ratings were obtained via the Internet, using a tailored Web application. Analysis of the obtained word norms showed good agreement with two existing norm sets. A cluster analysis revealed a plausible set of four classes of nouns: abstract concepts, aversive events, pleasant activities, and physical objects. In an additional application example, we demonstrate the usefulness of the database for creating parallel word lists whose elements match as closely as possible. The complete database is available for free from Moreover, the Web application used for data collection is inherently capable of collecting word norms in any language and is going to be released for public use as well.


Behavior Research Method Ambiguous Word Internet Protocol Address Word Norm Word Pool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Algarabel, S., Ruiz, J. C., & Sanmartin, J. (1988). The University of Valencia’s computerized word pool. Behavior Research Methods, Instruments, & Computers, 20, 398–403.CrossRefGoogle Scholar
  2. Altarriba, J., Bauer, L. M., & Benvenuto, C. (1999). Concreteness, context availability, and imageability ratings and word associations for abstract, concrete, and emotion words. Behavior Research Methods, Instruments, & Computers, 31, 578–602.CrossRefGoogle Scholar
  3. Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX lexical database (Release 2) [CD-ROM]. Philadelphia: University of Pennsylvania, Linguistic Data Consortium.Google Scholar
  4. Balota, D. A., Pilotti, M., & Cortese, M. J. (2001). Subjective frequency estimates for 2,938 monosyllabic words. Memory & Cognition, 29, 639–647.CrossRefGoogle Scholar
  5. Barca, L., Burani, C., & Arduino, L. S. (2002). Word naming times and psycholinguistic norms for Italian nouns. Behavior Research Methods, Instruments, & Computers, 34, 424–434.CrossRefGoogle Scholar
  6. Birnbaum, M. H. (2004). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803–832.CrossRefPubMedGoogle Scholar
  7. Brown, G. D. A. (1984). A frequency count of 190,000 words in the London-Lund Corpus of English Conversation. Behavior Research Methods, Instruments, & Computers, 16, 502–532.CrossRefGoogle Scholar
  8. Brown, W. P., & Ure, D. M. J. (1969). Five rated characteristics of 650 word association stimuli. British Journal of Psychology, 60, 233–249.CrossRefPubMedGoogle Scholar
  9. Clark, J. M., & Paivio, A. (2004). Extensions of the Paivio, Yuille, and Madigan (1968) norms. Behavior Research Methods, Instruments, & Computers, 36, 371–383.CrossRefGoogle Scholar
  10. Cortese, M. J., & Fugett, A. (2004). Imageability ratings for 3,000 monosyllabic words. Behavior Research Methods, Instruments, & Computers, 36, 384–387.CrossRefGoogle Scholar
  11. Friendly, M., Franklin, P. E., Hoffman, D., & Rubin, D. C. (1982). The Toronto Word Pool: Norms for imagery, concreteness, orthographic variables, and grammatical usage for 1,080 words. Behavior Research Methods & Instrumentation, 14, 375–399.CrossRefGoogle Scholar
  12. Gilhooly, K. J., & Logie, R. H. (1980). Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words. Behavior Research Methods & Instrumentation, 12, 395–427.CrossRefGoogle Scholar
  13. Hager, W., & Hasselhorn, M. (1994). Handbuch deutschsprachiger Wortnormen. Göttingen: Hogrefe.Google Scholar
  14. Izura, C., Hernández-Muñoz, N., & Ellis, A. W. (2005). Category norms for 500 Spanish words in five semantic categories. Behavior Research Methods, 37, 385–397.CrossRefPubMedGoogle Scholar
  15. Kerr, N. H., & Johnson, T. H. (1991). Word norms for blind and sighted subjects: Familiarity, concreteness, meaningfulness, imageability, imagery modality, and word associations. Behavior Research Methods, Instruments, & Computers, 23, 461–485.CrossRefGoogle Scholar
  16. Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness. Behavior Research Methods, Instruments, & Computers, 29, 264–269.CrossRefGoogle Scholar
  17. Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35–60). San Diego: Academic Press.CrossRefGoogle Scholar
  18. Lahl, O., & Pietrowsky, R. (2006). EQUIWORD: A software application for the automatic creation of truly equivalent word lists. Behavior Research Methods, 38, 146–152.CrossRefPubMedGoogle Scholar
  19. Lahl, O., & Pietrowsky, R. (2008). Tracer: A general purpose software library for logging events in computerized experiments. Behavior Research Methods, 40, 1163–1169.CrossRefPubMedGoogle Scholar
  20. Lahl, O., Wispel, C., Willigens, B., & Pietrowsky, R. (2008). An ultra short episode of sleep is sufficient to promote declarative memory performance. Journal of Sleep Research, 17, 3–10.CrossRefPubMedGoogle Scholar
  21. Liu, Y., Shu, H., & Li, P. (2007). Word naming and psycholinguistic norms: Chinese. Behavior Research Methods, 39, 192–198.CrossRefPubMedGoogle Scholar
  22. Marques, J. F., Fonseca, F. L., Morais, A. S., & Pinto, I. A. (2007). Estimated age of acquisition norms for 834 Portuguese nouns and their relation with other psycholinguistic variables. Behavior Research Methods, 39, 439–444.CrossRefPubMedGoogle Scholar
  23. Osgood, C. E., & Suci, G. J. (1955). Factor analysis of meaning. Journal of Experimental Psychology, 50, 325–338.CrossRefPubMedGoogle Scholar
  24. Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 words. Journal of Experimental Psychology Monograph Supplement, 76(1, Pt. 2).Google Scholar
  25. Pérez, M. A., & Navalón, C. (2005). Objective-AoA norms for 175 names in Spanish: Relationships with other psycholinguistic variables, estimated AoA, and data from other languages. European Journal of Cognitive Psychology, 17, 179–206.CrossRefGoogle Scholar
  26. Reips, U. D. (2000). The Web experiment method: Advantages, disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89–117). San Diego: Academic Press.CrossRefGoogle Scholar
  27. Stadthagen-Gonzalez, H., & Davis, C. J. (2006). The Bristol norms for age of acquisition, imageability, and familiarity. Behavior Research Methods, 38, 598–605.CrossRefPubMedGoogle Scholar
  28. Stevenson, R. A., Mikels, J. A., & James, T. W. (2007). Characterization of the Affective Norms for English Words by discrete emotional categories. Behavior Research Methods, 39, 1020–1024.CrossRefPubMedGoogle Scholar
  29. Toglia, M. P., & Battig, W. F. (1978). Handbook of semantic word norms. Hillsdale, NJ: Erlbaum.Google Scholar
  30. van Casteren, M., & Davis, M. H. (2007). Match: A program to assist in matching the conditions of factorial experiments. Behavior Research Methods, 39, 973–978.CrossRefPubMedGoogle Scholar
  31. Võ, M. L.-H., Jacobs, A. M., & Conrad, M. (2006). Cross-validating the Berlin Affective Word List. Behavior Research Methods, 38, 606–609.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2009

Authors and Affiliations

  • Olaf Lahl
    • 3
    • 1
  • Anja S. Göritz
    • 1
  • Reinhard Pietrowsky
    • 3
  • Jessica Rosenberg
    • 2
  1. 1.Universität Erlangen-NürnbergNürnbergGermany
  2. 2.Universität DüsseldorfDüsseldorfGermany
  3. 3.Institut für Experimentelle PsychologieHeinrich-Heine-Universität DüsseldorfDüsseldorfGermany

Personalised recommendations