Permutation Test

Definition

In a randomization test, we compute the null distribution of the test statistic by enumerating all possible treatment assignments. The number of treatment assignments, however, increases fast with the sample size. Instead, we can approximate the null distribution by randomly permuting the treatment labels and repeating the simulations a large number of times. To evaluate the hypothesis of no treatment effect using the permutation test, one needs to carry out the following:

• 1. Compute a relevant test statistic for collected data. The statistic is chosen so that it could differentiate between the null and the alternative.

• 2. Given the randomization scheme and under null, for large S, randomly simulate S treatment assignments and compute the corresponding test statistics for each permutation.

• 3. Use the simulated S values to construct the distribution of the test statistic under the null and compute the p-value of the test statistic.

Cross-References

Randomization Test...