Buy three but get only two: The smallest effect in a 2 × 2 ANOVA is always uninterpretable
Loftus (Memory & Cognition 6:312–319, 1978) distinguished between interpretable and uninterpretable interactions. Uninterpretable interactions are ambiguous, because they may be due to two additive main effects (no interaction) and a nonlinear relationship between the (latent) outcome variable and its indicator. Interpretable interactions can only be due to the presence of a true interactive effect in the outcome variable, regardless of the relationship that it establishes with its indicator. In the present article, we first show that same problem can arise when an unmeasured mediator has a nonlinear effect on the measured outcome variable. Then we integrate Loftus’s arguments with a seemingly contradictory approach to interactions suggested by Rosnow and Rosenthal (Psychological Bulletin 105:143–146, 1989). We show that entire data patterns, not just interaction effects alone, produce interpretable or noninterpretable interactions. Next, we show that the same problem of interpretability can apply to main effects. Lastly, we give concrete advice on what researchers can do to generate data patterns that provide unambiguous evidence for hypothesized interactions.
KeywordsStatistics Statistical inference
The writing of this article was supported, in part, by Grant PTDC/PSI-PSO/111992/2009 from the FCT (Portuguese National Science Foundation) awarded to the first author. The authors want to thanks the valuable contribution of Geoffrey Iverson, a much appreciated reminder that sometimes the review system really works.
- Abelson, R. P. (1995). Statistics as principled argument. Hillsdale, NJ: Erlbaum.Google Scholar
- Acheson, D. J., Postle, B. R., & MacDonald, M. C. (2010). The interaction of concreteness and phonological similarity in verbal working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(1), 17.Google Scholar
- Beaman, C. P., Hanczakowski, M., Hodgetts, H. M., Marsh, J. E., & Jones, D. M. (2013). Memory as discrimination: What distraction reveals. Memory & cognition, 41(8), 1238–1251.Google Scholar
- Conrad, R., & Hull, A. J. (1964). Information, acoustic confusion and memory span. British journal of psychology, 55(4), 429–432.Google Scholar
- Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis for field settings. Chicago, IL: Rand McNally.Google Scholar
- Laudan, L. (1977). Progress and its problems: Towards a theory of scientific growth. Berkeley, CA: University of California Press.Google Scholar
- Neely, C. B., & LeCompte, D. C. (1999). The importance of semantic similarity to the irrelevant speech effect. Memory & Cognition, 27(1), 37–44.Google Scholar
- Rosenthal, R., & Rosnow, R. L. (1985). Contrast analysis: Focused comparisons in the analysis of variance. Cambridge: Cambridge University Press.Google Scholar
- Rubin-Rabson, G. (1937). The influence of analytical prestudy in memorizing piano music. Archives of Psychology, 31(No. 220).Google Scholar