Behavior Research Methods

, Volume 39, Issue 4, pp 723–730 | Cite as

On the use of multilevel modeling as an alternative to items analysis in psycholinguistic research

  • Lawrence LockerEmail author
  • Lesa Hoffman
  • James A. Bovaird


The use of multilevel modeling is presented as an alternative to separate item and subject ANOVAs (F 1 ×F 2) in psycholinguistic research. Multilevel modeling is commonly utilized to model variability arising from the nesting of lower level observations within higher level units (e.g., students within schools, repeated measures within individuals). However, multilevel models can also be used when two random factors are crossed at the same level, rather than nested. The current work illustrates the use of the multilevel model for crossed random effects within the context of a psycholinguistic experimental study, in which both subjects and items are modeled as random effects within the same analysis, thus avoiding some of the problems plaguing current approaches.


Multilevel Model Math Achievement Neighborhood Frequency Psycholinguistic Research Electronic Appendix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Baayen, R. H., Tweedie, F. J., &Schreuder, R. (2002). The subjects as a simple random effect fallacy: Subject variability and morphological family effects in the mental lexicon.Brain & Language,81, 55–65.CrossRefGoogle Scholar
  2. Buchanan, L., & Westbury, C. (2000).Wordmine database: Probabilistic values for all four to seven letter words in the English Language. Scholar
  3. Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research.Journal of Verbal Learning & Verbal Behavior,12, 335–359.CrossRefGoogle Scholar
  4. Forster, K. I., &Dickinson, R. G. (1976). More on the language-asfixed-effect fallacy: Monte Carlo estimates of error rates forF1,F2,F′, minF′.Journal of Verbal Learning & Verbal Behavior,15, 135–142.CrossRefGoogle Scholar
  5. Ghisletta, P., & Renaud, O. (2005).Multilevel models for cross-factors data to generalize across both subjects and items. Paper presented at the 58th annual scientific meeting of the Gerontological Society of America, Orlando, FL.Google Scholar
  6. Hoffman, L., &Rovine, M. J. (2007). Multilevel models for experimental psychologists: Foundations and illustrative examples.Behavior Research Methods,39, 107–117.Google Scholar
  7. Locker, L., Simpson, G. B., &Yates, M. (2003). Semantic neighborhood effects on the recognition of ambiguous words.Memory & Cognition,31, 505–515.CrossRefGoogle Scholar
  8. Locker, L., Yates, M., & Simpson, G. B. (2003, November).The influence of phonological neighborhood frequency in visual lexical decision. Poster presented at the 44th Annual Meeting of the Psychonomic Society, Vancouver, BC.Google Scholar
  9. Nelson, D. L., McEvoy, C. L., & Schrieber, T. A. (1998).The University of South Florida word association, rhyme, and word fragment norms. Available at Scholar
  10. Pexman, P. M., &Lupker, S. J. (1999). Ambiguity and visual word recognition: Can feedback explain both homophone and polysemy effects?Canadian Journal of Experimental Psychology,53, 323–334.PubMedGoogle Scholar
  11. Quené, H., &van den Bergh, H. (2004). On multi-level modeling of data from repeated measures designs: A tutorial.Speech Communication,43, 103–121.CrossRefGoogle Scholar
  12. Rubenstein, H., Garfield, L., &Millikan, J. A. (1970). Homographic entries in the internal lexicon.Journal of Verbal Learning & Verbal Behavior,9, 487–494.CrossRefGoogle Scholar
  13. Raaijmakers, J. G. W. (2003). A further look at the “language-as-fixedeffect fallacy.”Canadian Journal of Experimental Psychology,57, 141–151.PubMedGoogle Scholar
  14. Raaijmakers, J. G. W., Schrijnemakers, J.M.C., &Gremmen, F. (1999). How to deal with the “language-as-fixed-effect fallacy”: Common misconceptions and alternative solutions.Journal of Memory & Language,41, 416–426.CrossRefGoogle Scholar
  15. Raudenbush, S. W. (1993). A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research.Journal of Educational & Behavioral Statistics, 18(4), 321–349.CrossRefGoogle Scholar
  16. Raudenbush, S. W., &Bryk, A. S. (2002).Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks CA: Sage.Google Scholar
  17. Strain, E., Patterson, K., &Seidenberg, M. S. (1995). Semantic effects in single-word naming.Journal of Experimental Psychology: Learning, Memory, & Cognition,21, 1140–1154.CrossRefGoogle Scholar
  18. Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models and individual growth models.Journal of Educational & Behavioral Statistics,24, 323–355.Google Scholar
  19. Snijders, T. A. B., &Bosker, R. (1999). Multilevel analysis. Thousand Oaks, CA: Sage.Google Scholar
  20. Schneider, W., Eschman, A., &Zuccolotto, A. (2002).E-Prime user’s guide. Pittsburgh, PA: Psychology Software Tools.Google Scholar
  21. Siakaluk, P. D., Sears, C. R., &Lupker, S. J. (2002). Orthographic neighborhood effects in lexical decision: The effects of nonword orthographic neighborhood size.Journal of Experimental Psychology: Human Perception & Performance,28, 661–681.CrossRefGoogle Scholar
  22. Wike, E. L., &Church, J. D. (1976). Comments on Clark’s “The language-as-fixed-effect fallacy.”Journal of Verbal Learning & Verbal Behavior,15, 249–255.CrossRefGoogle Scholar
  23. Wright, D. B. (1998). Modeling clustered data in autobiographical memory research: The multilevel approach.Applied Cognitive Psychology,12, 339–357.CrossRefGoogle Scholar
  24. Yates, M., Locker, L., &Simpson, G. B. (2004). The influence of phonological neighborhood on visual word perception.Psychonomic Bulletin & Review,11, 452–452.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2007

Authors and Affiliations

  • Lawrence Locker
    • 1
    Email author
  • Lesa Hoffman
    • 2
  • James A. Bovaird
    • 2
  1. 1.Departmentof PsychologyGeorgia Southern UniversityStatesboro
  2. 2.University of NebraskaLincoln

Personalised recommendations