Abstract
The present Monte Carlo study compares bootstrap and permutation tests for semiparametric heteroscedastic two-sample testing problems of Behrens-Fisher type. The underlying functionals to be tested are (a) the difference of the means and (b) the Wilcoxon functional P(Y < X) which is invariant under strictly increasing transformations. The consideration leads to semiparametric modifications of Welch type tests for the Behrens-Fisher model and an extended two-sample Wilcoxon test which also works under some null hypothesis with non-exchangeable distributions. The present Monte Carlo study confirms the high quality of studentized permutation tests at finite sample size. They are typically better than tests with asymptotic critical values and for many situations and they are also better than two-sample bootstrap tests when their type I error probabilities are compared.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beran, R. (1987). Prepivoting to reduce level error of confidence sets. Biometrika, 74, 457–468.
Beran, R. (1988). Prepivoting test statistics: A bootstrap view of asymptotic refinement. J. Am. Stat. Assoc., 83, 403, 687–697.
Best, D.J. and Rayner, J.C.W. (1987). Welch’s approximate solution for the Behrens-Fisher problem. Technometrics, 29, 205–210.
Efron, B. (1979). Bootstrap methods: Another look at the jacknife. Ann. Stat., 7, 1–27.
Fisher, R.A. (1936). The fiducial argument in statistical inference. Ann. Eugenics, 6, 391–422.
Fung, K.Y. (1979). A Monte Carlo study of the Wilcoxon statistic for the Behrens-Fisher problem. J. Statist. Comput. Simul., 10, 15–24.
Gupta, A.K. and Wang, Y. (1999). Some tests with specified size for the Behrens-Fisher problem. Commun. Statis., Theory Meth., 28, 3–4, 511–517.
Hájek, J., Šidák, Z. and Sen, P.K. (1999). Theory of rank tests. Academic Press, Orlando.
Heller, G. and Venkatraman, E.S. (1996). Resampling procedures to compare two survival distributions in the presence of right-censored data. Biometrics, 52, 1204–1213.
Janssen, A. and Brenner, S. (1990). Monte Carlo results for conditional survival tests under randomly censored data. J. Stat. Comput. Simulation, 39, 1–2, 1–26.
Janssen, A. (1997). Studentized permutation tests for non i.i.d. hypotheses and the generalized Behrens-Fisher problem. Stat. Prob. Letters, 36, 1, 9–21.
Janssen, A. (1999). Testing nonparametric statistical functionals with application to rank tests. J. Stat. Planning and Inference, 81, 1, 71–93.
Janssen, A. and Mayer, C.-D. (2001). Conditional studentized survival tests for randomly censored models. Scand. Journ. Stat., 28, 283–293.
Janssen, A. and Pauls, T. (2002). How do bootstrap and permutation tests work? to appear in Ann. Stat. (www.cs.uni-duesseldorf.de/∼stoch/docs/howdo.html)
Lee, A.J. (1990). U-Statistics: Theory and Practice, Dekker, New York.
Lehmann, E.L. (1986). Testing Statistical Hypotheses. Statistical Publishing Society, London.
Linnik, Y.W. (1968). Statistical problems with nuisance parameter. Translation of Mathematical Monograph, vol. 20. American Mathematical Society, Providence, RI.
Mason, D.M. and Shao, Q.-M. (2001). Bootstrapping the Student t-statistic. Ann. Probab., 29, 1435–1450.
Metha, J.S. and Srinivasan, R. (1970). On the Behrens-Fisher problem. Biometrika, 57, 649–655.
Neuhaus, N. (1993). Conditional rank tests for the two-sample problem under random censorship. Ann. Stat. 21, 1760–1779.
Pfanzagl, J. (1974). On the Behrens-Fisher problem. Biometrica, 61, 39–47.
Pauls, T. 2002. Resampling-Verfahren und ihre Anwendungen in der nichtparametrischen Testtheorie. Dissertation, University of Duesseldorf (in german).
Randles, R.H. and Wolfe, D.A. (1979). Introduction to the theory of nonparametric statistics. New York, John Wiley.
Romano, J.P. (1990). On the behaviour of randomization test without a group invariant assumption. J. Amer. Statist. Assoc., 85, 411, 686–692.
Wang, Y.Y. (1970). Probabilities of type I error of the Welch test for the Behrens-Fisher problem. J. Amer. Stat. Assoc., 66, 605–608.
Wellek, S. (1996). A new approach to equivalence assessment in standard comparative bioavailability trials by means of the Mann-Whitney statistic. Biom. J., 38, 6, 695–710.
Yuen, K.K. (1974). The two-sample trimmed t for unequal population variance. Biometrika, 61, 165–170.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Janssen, A., Pauls, T. A Monte Carlo comparison of studentized bootstrap and permutation tests for heteroscedastic two-sample problems. Computational Statistics 20, 369–383 (2005). https://doi.org/10.1007/BF02741303
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF02741303