Statistical Power and Sample Size

Abstract

Figure 1 shows 2 graphs of t-distributions. The lower graph (Hl)could be a probability distribution of a sample of data or of a sample of paired differences between two observations. N=20 and so 95% of the observations is within 2.901 ± 2.101 SEMs on the x-axis (usually called z-axis in statistics). The upper graph is identical, but centers around 0 instead of 2.901. It is called the null hypothesis H0, and represents the data of our sample if the mean results would be not different from zero. However, our mean result is 2.901 SEMs distant from zero. If we had many samples obtained by similar trials under the same null hypothesis, the chance of finding a mean value of more than 2.101 is < 5%, because the area under the curve (AUC) of HO right from 2.101 <5% of total AUC. We, therefore, reject the assumption that our results indicate a difference just by chance and decide that we have demonstrated a true difference. What is the power of this test. The power has as prior assumption that there is a difference from zero in our data. What is the chance of demonstrating a difference if there is one. If our experiment would be performed many times, the distribution of obtained mean values of those many experiments would center around 2.901, and about 70% of the AUC of H1 would be larger than 2.101. When smaller than 2.101, our statistical analysis would not be able to reject the null hypothesis of no difference, when larger, it would rightly be able to reject the null hypothesis of no difference. So, in fact 100−70=30% of the many trials would erroneously be unable to reject the null hypothesis of no difference, even when a true difference is in the data.