# Union–intersection permutation solution for two-sample equivalence testing

- 231 Downloads
- 8 Citations

## Abstract

One of the well-known problems with testing for sharp null hypotheses against two-sided alternatives is that, when sample sizes diverge, every consistent test rejects the null with a probability converging to one, even when it is true. This kind of problem emerges in practically all applications of traditional two-sided tests. The main purpose of the present paper is to overcome this very intriguing impasse by considering a general solution to the problem of testing for an equivalence null interval against a two one-sided alternative. Our goal is to go beyond the limitations of likelihood-based methods by working in a nonparametric permutation framework. This solution requires the nonparameteric Combination of dependent permutation tests, which is the methodological tool that achieves Roy’s Union–intersection principle. To obtain practical solutions, the related algorithm is presented. To appreciate its effectiveness for practical purposes, a simple example and some simulation results are also presented. In addition, for every pair of consistent partial test statistics it is proved that, if sample sizes diverge, when the effect lies in the open equivalence interval, the Rejection probability (RP) converges to zero. Analogously, if the effect lies outside that interval, the RP converges to one.

## Keywords

Nonparametric combination Permutation tests Testing equivalence Two one-sided tests Union–intersection principle## Notes

### Acknowledgments

Authors acknowledge anonymous referees and associate editor for comments which helped to improved the manuscript. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. All authors have contributed equally to the manuscript.

## References

- Basso, D., Pesarin, F., Salmaso, L., Solari, A.: Permutation Tests for Stochastic Ordering and ANOVA: Theory and Applications in R. Lecture Notes N. Springer, New York (2009)zbMATHGoogle Scholar
- Berger, R.L.: Multiparameter hypothesis testing and acceptance sampling. Technometrics
**24**, 295–300 (1982)MathSciNetCrossRefzbMATHGoogle Scholar - Bertoluzzo, F., Pesarin, F., Salmaso, L.: On multi-sided permutation tests. Commun. Stat. Simul. Comput.
**42**(6), 1380–1390 (2013)CrossRefzbMATHGoogle Scholar - Birnbaum, A.: Combining independent tests of significance. J. Am. Stat. Assoc.
**49**, 559–574 (1954a)MathSciNetzbMATHGoogle Scholar - Birnbaum, A.: Characterization of complete classes of tests of some multiparametric hypotheses, with application to likelihood ratio tests. Ann. Math. Stat.
**26**, 21–36 (1954b)MathSciNetCrossRefzbMATHGoogle Scholar - Brombin, C., Salmaso, L., Ferronato, G., Galzignato, P.F.: Multi-aspect procedures for paired data with application to biometric morphing. Commun. Stat. Simul. Comput.
**40**, 1–12 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - Cox, D.R., Hinkley, D.V.: Theoretical Statistics. Chapman and Hall, London (1974)CrossRefzbMATHGoogle Scholar
- Edgington, E.S., Onghena, P.: Randomization Tests, 4th edn. Chapman & Hall, Boca Raton (2007)zbMATHGoogle Scholar
- Frosini, B.V.: On Neyman–Pearson theory: information content of an experiment and a fancy paradox. Statistica
**64**, 271–286 (2004)Google Scholar - Good, P.: Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. Springer, New York (2000)CrossRefzbMATHGoogle Scholar
- Hoeffding, W.: The large-sample power of tests based on permutations of observations. Ann. Math. Stat.
**23**, 169–192 (1952)MathSciNetCrossRefzbMATHGoogle Scholar - Hung, H.M.J., Wang, S.U.: Some controversial multiple testing problems in regulatory applications. J. Biopharm. Stat.
**19**, 1–11 (2009)MathSciNetCrossRefGoogle Scholar - Lehmann, E.L.: Testing Statistical Hypotheses, 2nd edn. Wiley, New York (1986)CrossRefzbMATHGoogle Scholar
- Marozzi, M.: A bi-aspect nonparametric test for the two-sample location problem. Comput. Stat. Data Anal.
**44**, 639–648 (2004)MathSciNetCrossRefzbMATHGoogle Scholar - Marozzi, M.: Multivariate tests based on interpoint distances with application to magnetic resonance imaging. Stat. Methods Med. Res. (2014). doi: 10.1177/0962280214529104
- Marozzi, M., Salmaso, L.: Multivariate bi-aspect testing for the two-sample location problem. Commun. Stat. Theory Methods
**35**(3), 477–488 (2006)MathSciNetCrossRefzbMATHGoogle Scholar - Nunnally, J.: The place of statistics in psychology. Educ. Psychol. Measur.
**20**, 641–650 (1960)CrossRefGoogle Scholar - Pantsulaia, G., Kintsurashvili, M.: Why is the null hypothesis rejected for almost every infinite sample by some hypothesis testing of maximal reliability. J. Stat.
**11**, 45–70 (2014)Google Scholar - Pesarin, F.: On a nonparametric combination method for dependent permutation tests with applications. Psychom. Psychosom.
**54**, 172–179 (1990)CrossRefGoogle Scholar - Pesarin, F.: Multivariate Permutation Tests, with Applications in Biostatistics. Wiley, Chichester (2001)zbMATHGoogle Scholar
- Pesarin, F., Salmaso, L.: Permutation Tests for Complex Data, Theory, Applications and Software. Wiley, Chichester (2010a)CrossRefzbMATHGoogle Scholar
- Pesarin, F., Salmaso, L.: Finite-sample consistency of combination-based permutation tests with application to repeated measures designs. J. Nonparametric Stat.
**22**(5), 669–684 (2010b)MathSciNetCrossRefzbMATHGoogle Scholar - Pesarin, F., Salmaso, L.: On the weak consistency of permutation tests. Commun. Stat. Simul. Comput.
**42**, 1368–1397 (2013)CrossRefzbMATHGoogle Scholar - Romano, J.P.: Optimal testing of equivalence hypotheses. Ann. Stat.
**33**, 1036–1047 (2005)MathSciNetCrossRefzbMATHGoogle Scholar - Roy, S.N.: On a heuristic method of test construction and its use in multivariate analysis. Ann. Math. Stat.
**24**, 220–238 (1953)MathSciNetCrossRefzbMATHGoogle Scholar - Salmaso, L., Solari, A.: Multiple aspect testing for case-control designs. Metrika
**62**, 331–340 (2005)MathSciNetCrossRefzbMATHGoogle Scholar - Sen, P.K.: Union–intersection principle and constrained statistical inference. Stat. Plann. Inference
**137**, 3741–3752 (2007)MathSciNetCrossRefzbMATHGoogle Scholar - Sen, P.K., Tsai, M.T.: Two-stage likelihood ratio and union intersection tests for one-sided alternatives multivariate mean with nuisance dispersion matrix. J. Multivar. Anal.
**68**(264), 282 (1999)MathSciNetzbMATHGoogle Scholar - Wellek, S.: Testing Statistical Hypotheses of Equivalence and Noninferiority. Chapman & Hall, Boca Raton (2010)CrossRefzbMATHGoogle Scholar