# Buy three but get only two: The smallest effect in a 2 × 2 ANOVA is always uninterpretable

## Abstract

Loftus (*Memory & Cognition* 6:312–319, 1978) distinguished between interpretable and uninterpretable interactions. Uninterpretable interactions are ambiguous, because they may be due to two additive main effects (no interaction) and a nonlinear relationship between the (latent) outcome variable and its indicator. Interpretable interactions can only be due to the presence of a true interactive effect in the outcome variable, regardless of the relationship that it establishes with its indicator. In the present article, we first show that same problem can arise when an unmeasured mediator has a nonlinear effect on the measured outcome variable. Then we integrate Loftus’s arguments with a seemingly contradictory approach to interactions suggested by Rosnow and Rosenthal (*Psychological Bulletin* 105:143–146, 1989). We show that entire data patterns, not just interaction effects alone, produce interpretable or noninterpretable interactions. Next, we show that the same problem of interpretability can apply to main effects. Lastly, we give concrete advice on what researchers can do to generate data patterns that provide unambiguous evidence for hypothesized interactions.

## Keywords

Statistics Statistical inference## Notes

### Acknowledgments

The writing of this article was supported, in part, by Grant PTDC/PSI-PSO/111992/2009 from the FCT (Portuguese National Science Foundation) awarded to the first author. The authors want to thanks the valuable contribution of Geoffrey Iverson, a much appreciated reminder that sometimes the review system really works.

## References

- Abelson, R. P. (1995).
*Statistics as principled argument*. Hillsdale, NJ: Erlbaum.Google Scholar - Acheson, D. J., Postle, B. R., & MacDonald, M. C. (2010). The interaction of concreteness and phonological similarity in verbal working memory.
*Journal of Experimental Psychology: Learning, Memory, and Cognition, 36*(1), 17.Google Scholar - Anderson, N. H. (1961). Scales and statistics: Parametric and nonparametric.
*Psychological Bulletin, 58,*305–316.PubMedCrossRefGoogle Scholar - Beaman, C. P., Hanczakowski, M., Hodgetts, H. M., Marsh, J. E., & Jones, D. M. (2013). Memory as discrimination: What distraction reveals. Memory & cognition, 41(8), 1238–1251.Google Scholar
- Bogartz, R. S. (1976). On the meaning of statistical interactions.
*Journal of Experimental Child Psychology, 22,*178–183.CrossRefGoogle Scholar - Conrad, R., & Hull, A. J. (1964). Information, acoustic confusion and memory span.
*British journal of psychology, 55*(4), 429–432.Google Scholar - Cook, T. D., & Campbell, D. T. (1979).
*Quasi-experimentation: Design and analysis for field settings*. Chicago, IL: Rand McNally.Google Scholar - Davison, M. L., & Sharma, A. R. (1988). Parametric statistics and levels of measurement.
*Psychology Bulletin, 104,*137–144.CrossRefGoogle Scholar - Embretson, S. E. (1996). Item response theory models and spurious interaction effects in factorial ANOVA designs.
*Applied Psychological Measurement, 20,*201–212.CrossRefGoogle Scholar - Kang, S.-M., & Waller, N. G. (2005). Moderated multiple regression, spurious interaction effects, and IRT.
*Applied Psychological Measurement, 29,*87–105.CrossRefGoogle Scholar - Laudan, L. (1977).
*Progress and its problems: Towards a theory of scientific growth*. Berkeley, CA: University of California Press.Google Scholar - Loftus, G. R. (1978). On interpretation of interactions.
*Memory & Cognition, 6,*312–319.CrossRefGoogle Scholar - Neely, C. B., & LeCompte, D. C. (1999). The importance of semantic similarity to the irrelevant speech effect.
*Memory & Cognition, 27*(1), 37–44.Google Scholar - Rosenthal, R., & Rosnow, R. L. (1985).
*Contrast analysis: Focused comparisons in the analysis of variance*. Cambridge: Cambridge University Press.Google Scholar - Rosnow, R. L., & Rosenthal, R. (1989). Definition and interpretation of interaction effects.
*Psychological Bulletin, 105,*143–146.CrossRefGoogle Scholar - Rosnow, R. L., & Rosenthal, R. (1991). If you’re looking at the cell means, you’re not looking at only the interaction (unless all main effects are zero).
*Psychological Bulletin, 110,*574–576.CrossRefGoogle Scholar - Rosnow, R. L., & Rosenthal, R. (1995). “Some things you learn aren’t so”: Cohen’s paradox, Asch’s paradigm, and the interpretation of interaction.
*Psychological Science, 6,*3–9.CrossRefGoogle Scholar - Rubin-Rabson, G. (1937). The influence of analytical prestudy in memorizing piano music.
*Archives of Psychology, 31*(No. 220).Google Scholar - Stevens, S. S. (1946). On the theory of scales of measurement.
*Science, 103*(2684), 677–680.CrossRefGoogle Scholar - Wagenmakers, E.-J., Krypotos, A.-M., Criss, A. M., & Iverson, G. (2012). On the interpretation of removable interactions: A survey of the field 33 years after Loftus.
*Memory & Cognition, 40,*145–160.CrossRefGoogle Scholar