, Volume 14, Issue 5, pp 779-804

A practical solution to the pervasive problems ofp values

Abstract

In the field of psychology, the practice ofp value null-hypothesis testing is as widespread as ever. Despite this popularity, or perhaps because of it, most psychologists are not aware of the statistical peculiarities of thep value procedure. In particular,p values are based on data that were never observed, and these hypothetical data are themselves influenced by subjective intentions. Moreover,p values do not quantify statistical evidence. This article reviews thesep value problems and illustrates each problem with concrete examples. The three problems are familiar to statisticians but may be new to psychologists. A practical solution to thesep value problems is to adopt a model selection perspective and use the Bayesian information criterion (BIC) for statistical inference (Raftery, 1995). The BIC provides an approximation to a Bayesian hypothesis test, does not require the specification of priors, and can be easily calculated from SPSS output.

This research was supported by a Veni Grant from the Dutch Organization for Scientific Research (NWO). I thank Scott Brown, Peter Dixon, Simon Farrell, Raoul Grasman, Geoff Iverson, Michael Lee, Martijn Meeter, Jay Myung, Jeroen Raaijmakers, Jeff Rouder, and Rich Shiffrin for helpful comments on earlier drafts of this article. Mark Steyvers convinced me that this article would be seriously incomplete without a consideration of practical alternatives to thep value methodology.