Relationships Between p-values and Pearson Correlation Coefficients, Type 1 Errors and Effect Size Errors, Under a True Null Hypothesis

Abstract

The American Statistical Association (ASA) published a statement in 2016 in The American Statistician for “researchers, practitioners and science writers who are not primarily statisticians” on the proper use and interpretation of p-values. Three years later, the ASA acknowledged that telling researchers what not to do with p-values was insufficient. Consequently, 3 years later an open access, special issue appeared with 43 papers proposing various novel and sophisticated alternatives to classical p-values for use with scientific methods in the twenty-first century. In the opening remarks, the editors stated that “no p-value can reveal the plausibility, presence, truth, or importance of an association or effect” and banned statistical significance: “don’t say it, don’t use it.” This paper questions both statements with a simulated data study. It is shown that p-values are strongly related to correlation coefficients under a true null hypothesis; hence, can reveal the “importance of an association or effect.” Furthermore, it demonstrates why a cut point for statistical significance is still a viable, ancillary tool for assessing the substantive significance of statistical effects with small sample sizes (n < 1000).