Psychonomic Bulletin & Review

, Volume 21, Issue 6, pp 1415–1430 | Cite as

Buy three but get only two: The smallest effect in a 2 × 2 ANOVA is always uninterpretable

  • Leonel Garcia-Marques
  • Teresa Garcia-Marques
  • Markus Brauer
Theoretical Review

Abstract

Loftus (Memory & Cognition 6:312–319, 1978) distinguished between interpretable and uninterpretable interactions. Uninterpretable interactions are ambiguous, because they may be due to two additive main effects (no interaction) and a nonlinear relationship between the (latent) outcome variable and its indicator. Interpretable interactions can only be due to the presence of a true interactive effect in the outcome variable, regardless of the relationship that it establishes with its indicator. In the present article, we first show that same problem can arise when an unmeasured mediator has a nonlinear effect on the measured outcome variable. Then we integrate Loftus’s arguments with a seemingly contradictory approach to interactions suggested by Rosnow and Rosenthal (Psychological Bulletin 105:143–146, 1989). We show that entire data patterns, not just interaction effects alone, produce interpretable or noninterpretable interactions. Next, we show that the same problem of interpretability can apply to main effects. Lastly, we give concrete advice on what researchers can do to generate data patterns that provide unambiguous evidence for hypothesized interactions.

Keywords

Statistics Statistical inference 

References

  1. Abelson, R. P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.Google Scholar
  2. Acheson, D. J., Postle, B. R., & MacDonald, M. C. (2010). The interaction of concreteness and phonological similarity in verbal working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(1), 17.Google Scholar
  3. Anderson, N. H. (1961). Scales and statistics: Parametric and nonparametric. Psychological Bulletin, 58, 305–316.PubMedCrossRefGoogle Scholar
  4. Beaman, C. P., Hanczakowski, M., Hodgetts, H. M., Marsh, J. E., & Jones, D. M. (2013). Memory as discrimination: What distraction reveals. Memory & cognition, 41(8), 1238–1251.Google Scholar
  5. Bogartz, R. S. (1976). On the meaning of statistical interactions. Journal of Experimental Child Psychology, 22, 178–183.CrossRefGoogle Scholar
  6. Conrad, R., & Hull, A. J. (1964). Information, acoustic confusion and memory span. British journal of psychology, 55(4), 429–432.Google Scholar
  7. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis for field settings. Chicago, IL: Rand McNally.Google Scholar
  8. Davison, M. L., & Sharma, A. R. (1988). Parametric statistics and levels of measurement. Psychology Bulletin, 104, 137–144.CrossRefGoogle Scholar
  9. Embretson, S. E. (1996). Item response theory models and spurious interaction effects in factorial ANOVA designs. Applied Psychological Measurement, 20, 201–212.CrossRefGoogle Scholar
  10. Kang, S.-M., & Waller, N. G. (2005). Moderated multiple regression, spurious interaction effects, and IRT. Applied Psychological Measurement, 29, 87–105.CrossRefGoogle Scholar
  11. Laudan, L. (1977). Progress and its problems: Towards a theory of scientific growth. Berkeley, CA: University of California Press.Google Scholar
  12. Loftus, G. R. (1978). On interpretation of interactions. Memory & Cognition, 6, 312–319.CrossRefGoogle Scholar
  13. Neely, C. B., & LeCompte, D. C. (1999). The importance of semantic similarity to the irrelevant speech effect. Memory & Cognition, 27(1), 37–44.Google Scholar
  14. Rosenthal, R., & Rosnow, R. L. (1985). Contrast analysis: Focused comparisons in the analysis of variance. Cambridge: Cambridge University Press.Google Scholar
  15. Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects. Psychological Bulletin, 105, 143–146.CrossRefGoogle Scholar
  16. Rosnow, R. L., & Rosenthal, R. (1991). If you’re looking at the cell means, you’re not looking at only the interaction (unless all main effects are zero). Psychological Bulletin, 110, 574–576.CrossRefGoogle Scholar
  17. Rosnow, R. L., & Rosenthal, R. (1995). “Some things you learn aren’t so”: Cohen’s paradox, Asch’s paradigm, and the interpretation of interaction. Psychological Science, 6, 3–9.CrossRefGoogle Scholar
  18. Rubin-Rabson, G. (1937). The influence of analytical prestudy in memorizing piano music. Archives of Psychology, 31(No. 220).Google Scholar
  19. Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103(2684), 677–680.CrossRefGoogle Scholar
  20. Wagenmakers, E.-J., Krypotos, A.-M., Criss, A. M., & Iverson, G. (2012). On the interpretation of removable interactions: A survey of the field 33 years after Loftus. Memory & Cognition, 40, 145–160.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2014

Authors and Affiliations

  • Leonel Garcia-Marques
    • 1
  • Teresa Garcia-Marques
    • 2
  • Markus Brauer
    • 3
  1. 1.Faculdade de PsicologiaUniversidade de Lisboa, Alameda da UniversidadeLisboaPortugal
  2. 2.ISPA-University InstituteLisbonPortugal
  3. 3.University of Wisconsin–MadisonMadisonUSA

Personalised recommendations