# Applications of conditional power function of two-sample permutation test

- 136 Downloads

## Abstract

Permutation or randomization test is a nonparametric test in which the null distribution (distribution under the null hypothesis of no relationship or no effect) of the test statistic is attained by calculating the values of the test statistic overall permutations (or by considering a large number of random permutation) of the observed dataset. The power of permutation test evaluated based on the observed dataset is called *conditional power*. In this paper, the conditional power of permutation tests is reviewed. The use of the conditional power function for sample size estimation is investigated. Moreover, reproducibility and generalizability probabilities are defined. The use of these probabilities for sample size adjustment is shown. Finally, an illustration example is used.

## Keywords

Generalizability probability Permutation test Reproducibility probability Sample size adjustment Sample size estimation## Notes

### Acknowledgements

The authors would like to thank the associated editor and referees for their comments that contribute in improving the paper. We also greatly appreciate Dr. Ibrahim Almasri, Department of Applied Mathematics and Physics - Palestine Polytechnic University, for being kind enough to read and improve the language of the paper.

## References

- Aguirre-Urreta M, Rönkkö M (2015) Sample size determination and statistical power analysis in PLS using R: an annotated tutorial. Commun Assoc Inf Syst 36(3):33–51Google Scholar
- Akobeng AK (2016) Understanding type I and type II errors, statistical power and sample size. Acta Paediatr 105(6):605–609CrossRefGoogle Scholar
- Amro L, Pauly M (2017) Permuting incomplete paired data: a novel exact and asymptotic correct randomization test. J Stat Comput Simul 87(6):1148–1159MathSciNetCrossRefGoogle Scholar
- Barton DE (1957) A comparison of two sorts of test for a change of location applicable to truncated data. J R Stat Soc 19:119–124MathSciNetzbMATHGoogle Scholar
- Basso D, Pesarin F, Salmaso L, Solari A (2009) Permutation tests for stochastic ordering and ANOVA: theory and applications in R. Springer, New YorkzbMATHGoogle Scholar
- Bell CB, Moser JM, Thompson R (1966) Goodness criteria for two-sample distribution-free tests. Ann Math Stat 37:133–142MathSciNetCrossRefzbMATHGoogle Scholar
- Brewer JK, Sindelar PT (1988) Adequate sample size: a priori and post hoc considerations. J Spec Educ 21:74–84CrossRefGoogle Scholar
- Chow SC, Liu JP (2004) Design and analysis of clinical trials: concepts and methodologies, 2nd edn. Wiley-Blackwell, New YorkzbMATHGoogle Scholar
- Chow SC, Shao J, Wang H (2002) A note on sample size calculation for mean comparisons based on non-central \(t\)-statistics. J Biopharm Stat 12:441–456CrossRefGoogle Scholar
- Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, HillsdalezbMATHGoogle Scholar
- Collings BJ, Hamilton MA (1988) Estimating the power of the two-sample Wilcoxon test for location shift. Biometrics 44:847–860CrossRefzbMATHGoogle Scholar
- Cooper H, Hedges LV (1997) The handbook of research synthesis. Russell Sage Foundation, New YorkGoogle Scholar
- De Capitani L, De Martini D (2016) Reproducibility probability estimation and RP-testing for some nonparametric tests. Entropy 18(4):142CrossRefGoogle Scholar
- De Martini D (2002) Pointwise estimate of the power and sample size determination for permutation tests. Statistica 62:779–790MathSciNetzbMATHGoogle Scholar
- Dixon WJ (1954) Power under normality of several nonparametric tests. Ann Math Stat 25:610–614MathSciNetCrossRefzbMATHGoogle Scholar
- Edgington ES (1995) Randomization tests, 3rd edn. Marcel Dekker, New YorkzbMATHGoogle Scholar
- Epstein B (1955) Comparison of some non-parametric tests against normal alternatives with an application to life testing. J Am Stat Assoc 50:894–900MathSciNetzbMATHGoogle Scholar
- Fisher RA (1934) Statistical methods for research workers. Oliver and Boyd, EdinburghzbMATHGoogle Scholar
- Fisher RA (1935) The design of experiments. Oliver and Boyd, EdinburghGoogle Scholar
- Giraudeau B, Higgins J, Tavernier E, Trinquart L (2016) Sample size calculation for meta-epidemiological studies. Stat Med 35(2):239–250MathSciNetCrossRefGoogle Scholar
- Good P (2005) Permutation, parametric and bootstrap tests of hypotheses, 3rd edn. Springer, New YorkzbMATHGoogle Scholar
- Goodman S (1992) A comment on replication, \(p\)-values and evidence. Stat Med 11:875–879CrossRefGoogle Scholar
- Hallahan M, Rosenthal R (1996) Statistical power: concepts, procedures, and applications. Behav Res Ther 34:489–499CrossRefGoogle Scholar
- Hamilton MA, Collings BJ (1991) Determining the appropriate sample size for nonparametric tests for location shift. Technometrics 33:327–337MathSciNetCrossRefzbMATHGoogle Scholar
- Haynam GE, Govindarajulu Z (1966) Exact power of the Mann–Whitney test for exponential and rectangular alternatives. Ann Math Stat 37:945–953MathSciNetCrossRefzbMATHGoogle Scholar
- Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, New YorkzbMATHGoogle Scholar
- Hemelrijk J (1961) Experimental comparison of Student’s and Wilcoxon’s two sample test. Quantitative Methods in Pharmacology. Inter-science, New York, pp 118–133Google Scholar
- Hoeffding W (1952) The large-sample power of tests based on permutations of observations. Ann Math Stat 23:169–192MathSciNetCrossRefzbMATHGoogle Scholar
- Kraemer HC, Thiemann S (1987) How many subjects? Statistical power analysis in research. Sage Publications, Newbury ParkGoogle Scholar
- Lehmann EL, Romano JP (2005) Testing statistical hypotheses, 3rd edn. Springer, New YorkzbMATHGoogle Scholar
- Lehmann EL, Stein C (1949) On the theory of some non-parametric hypotheses. Ann Math Stat 20:28–45CrossRefzbMATHGoogle Scholar
- Lenth RV (2007) Post hoc power: tables and commentary. Technical Report 378, The University of Iowa - Department of Statistics and Actuarial ScienceGoogle Scholar
- Levine M, Ensom MHH (2001) Post hoc power analysis: an idea whose time has passed? Pharmacotherapy 21:405–409CrossRefGoogle Scholar
- Lipsey MW (1990) Design sensitivity: statistical power for experimental research. Sage Publications, Newbury ParkGoogle Scholar
- Markowski EP, Markowski CA (1999) Practical uses of statistical power in business research studies. J Educ Bus 75:122–125CrossRefzbMATHGoogle Scholar
- McDonald J, Gerard PD, McMahan CS, Schucany WR (2016) Exact-permutation-based sign tests for clustered binary data via weighted and unweighted test statistics. J Agric Biol Environ Stat 21(4):698–712MathSciNetCrossRefzbMATHGoogle Scholar
- McHugh RB (1961) Confidence interval inference and sample size determination. Am Stat 15:14–17Google Scholar
- Milton RC (1970) Rank order probabilities: two-sample normal shift alternatives. Wiley, New YorkzbMATHGoogle Scholar
- Moher D, Dulberg CS, Wells GA (1994) Statistical power, sample size, and their reporting in randomized controlled trials. J Am Med Assoc 272:122–124CrossRefGoogle Scholar
- Noether GE (1987) Sample size determination for some common nonparametric tests. J Am Stat Assoc 82:645–647MathSciNetCrossRefzbMATHGoogle Scholar
- Onwuegbuzie AJ, Leech NL (2004) Post hoc power: a concept whose time has come. Underst Stat 3:201–230CrossRefGoogle Scholar
- Owen DB (1965) The power of Student’s \(t\)-test. J Am Stat Assoc 60:320–333MathSciNetzbMATHGoogle Scholar
- Pesarin F (2001) Multivariate permutation tests: with application in biostatistics. Wiley, ChichesterzbMATHGoogle Scholar
- Pesarin F, Salmaso L (2010) Permutation tests for complex data: theory application and software. Wiley, ChichesterCrossRefzbMATHGoogle Scholar
- Pitman EJG (1937a) Significance tests which may be applied to samples from any population. J R Stat Soc B 4:119–130zbMATHGoogle Scholar
- Pitman EJG (1937b) Significance tests which may be applied to samples from any population. II. The correlation coefficient test. J R Stat Soc B 4:225–232zbMATHGoogle Scholar
- Pitman EJG (1938) Significance tests which may be applied to samples from any population. III. The analysis of variance test. Biometrika 29:322–335zbMATHGoogle Scholar
- Randles RH, Wolfe DA (1979) Introduction to the theory of nonparametric statistics. Wiley, New YorkzbMATHGoogle Scholar
- Salmaso L (2003) Synchronized permutation tests in \(2^k\) factorial designs. Commun Stat Theory Methods 32:1419–1437MathSciNetCrossRefzbMATHGoogle Scholar
- Samonenko I, Robinson J (2015) A new permutation test statistic for complete block designs. Ann Stat 43(1):90–101MathSciNetCrossRefzbMATHGoogle Scholar
- Schmitt MC (1987) The effects on an elaborated directed reading activity on the metacomprehension skills of third graders. Ph.D. Thesis, Purdue UniversityGoogle Scholar
- Shao J, Chow SC (2002) Reproducibility probability in clinical trials. Stat Med 21:1727–1742CrossRefGoogle Scholar
- Shorack GR, Wellner JA (1986) Empirical processes with applications to statistics. Wiley Series in Probability & Mathematical Statistics, New YorkzbMATHGoogle Scholar
- Simonoff JS, Hochberg Y, Reiser B (1986) Alternative estimation procedures for \({P}_r({X} < {Y})\) in categorized data. Biometrics 42:895–907MathSciNetCrossRefzbMATHGoogle Scholar
- Teichroew D (1955) Empirical power functions for nonparametric two-sample tests for small samples. Ann Math Stat 26:340–344MathSciNetCrossRefzbMATHGoogle Scholar
- Thomas L (1997) Retrospective power analysis. Conserv Biol 11:276–280CrossRefGoogle Scholar
- Wang H, Chow SC, Chen M (2005) A Bayesian approach on sample size calculation for comparing means. J Biopharm Stat 15:799–807MathSciNetCrossRefGoogle Scholar