Distribution-free two-sample comparisons in the case of heterogeneous variances

Abstract

Behavioral ecologists are often faced with a situation where they need to compare the central tendencies of two samples. The standard tools of the t test and Mann–Whitney U test (equivalent to the Wilcoxon rank-sum test) are unreliable when the variances of the groups are different. The problem is particularly severe when sample sizes are different between groups. The unequal-variance t test (Welch test) may not be suitable for nonnormal data. Here, we propose the use of Brunner and Munzel’s generalized Wilcoxon test followed by randomization to allow for small sample sizes. This tests whether the probability of an individual from one population being bigger than an individual from the other deviates from random expectation. This probability may sometimes be a more clear and informative measure of difference between the groups than a difference in more commonly used measures of central tendency (such as the mean). We provide a recipe for carrying out a statistical test of the null hypothesis that this probability is 50% and demonstrate the effectiveness of this technique for sample sizes typical in behavioral ecology. Although the test is not available in any commercial software package, it is relatively straightforward to implement for anyone with some programming ability. Furthermore, implementations in R and SAS are freely available on the internet.