Abstract
We review some results concerning the levels at which multiple testing procedures (MTPs) control certain type I error rates under a general and unknown dependence structure of the p-values on which the MTP is based. The type I error rates we deal with are (1) the classical family-wise error rate (FWER); (2) its immediate generalization: the probability of k or more false rejections (the generalized FWER); (3) the per-family error rate—the expected number of false rejections (PFER). The procedures considered are those satisfying the condition of monotonicity: reduction in some (or all) of the p-values used as input for the MTP can only increase the number of rejected hypotheses. It turns out that this natural condition, either by itself or combined with a property of being a step-down or step-up MTP (where the terms “step-down” and “step-up” are understood in their most general sense), has powerful consequences. Those include optimality results, inequalities, and identities involving different numerical characteristics of a procedure, and computational formulas.
Dedicated with deep gratitude and admiration to the memory of Andrei Yakovlev, who always inspired and encouraged those around him.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 57, 289–300.
Benjamini, Y., & Liu, W. (1999). A distribution-free multiple test procedure that controls the false discovery rate. Technical Report. RP-SOR-99-3, Department of Statistics and Operations Research, Tel Aviv University.
Dudoit, S., Shaffer, J. P., & Boldrich, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statistical Science, 18, 71–103.
Dudoit, S., & van der Laan, M. J. (2008). Multiple testing procedures with applications to genomics. New York, NY: Springer.
Finner, H., & Roters, M. (2002). Multiple hypothesis testing and expected number of type I errors. The Annals of Statistics, 30, 220–238.
Gordon, A. Y. (2007a). Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures. Journal of Statistical Planning and Inference, 137, 3497–3512.
Gordon, A. Y. (2007b). Family-wise error rate of a step-down procedure. Random Operators and Stochastic Equations, 15, 399–408.
Gordon, A. Y. (2007c). Unimprovability of the Bonferroni procedure in the class of general step-up multiple testing procedures. Statistics & Probability Letters, 77, 117–122.
Gordon, A. Y. (2009). Inequalities between generalized familywise error rates of a multiple testing procedure. Statistics & Probability Letters, 79, 1996–2004.
Gordon, A. Y. (2011). A new optimality property of the Holm step-down procedure. Statistical Methodology, 8, 129–135.
Gordon, A. Y. (2012). A sharp upper bound for the expected number of false rejections. Statistics & Probability Letters, 82, 1507–1514.
Gordon, A. Y., & Salzman, P. (2008). Optimality of the Holm procedure among general step-down multiple testing procedures. Statistics & Probability Letters, 78, 1878–1884.
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800–802.
Hochberg, Y., & Tamhane, A. C. (1987). Multiple comparison procedures. New York, NY: Wiley.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.
Hommel, G., & Hoffman, T. (1988). Controlled uncertainty. In P. Bauer, G. Hommel, & E. Sonnemann (Eds.), Multiple hypothesis testing (pp. 154–161). Heidelberg: Springer.
Korn, E. L., Troendle, J. F., McShane, L. M., & Simon, R. (2004). Controlling the number of false discoveries: application to high-dimensional genomic data. Journal of Statistical Planning and Inference, 124, 379–398.
Lehmann, E. L., & Romano, J. P. (2005a). Generalizations of the familywise error rate. The Annals of Statistics, 33, 1138–1154.
Lehmann, E. L., & Romano, J. P. (2005b). Testing statistical hypotheses (3rd ed.). New York, NY: Springer.
Liu, W. (1996). Multiple tests of a non-hierarchical family of hypotheses. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 58, 455–461.
Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. The Annals of Statistics, 30, 239–257.
Shaffer, J. (1995). Multiple hypothesis testing: A review. Annual Review of Psychology, 46, 561–584.
Tamhane, A. C., & Liu, W. (1995). On weighted Hochberg procedures. Biometrika, 95, 279–294.
Tamhane, A. C., Liu, W., & Dunnett, C. W. (1998). A generalized step-up-down multiple test procedure. The Canadian Journal of Statistics, 26, 353–363.
Tukey, J. W. (1953). The problem of multiple comparison. Unpublished manuscript. In The collected works of John W. Tukey VIII. Multiple comparisons: 1948–1983 (pp. 1–300). New York, NY: Chapman and Hall.
van der Laan, M. J., Dudoit, S., & Pollard, K. S. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Statistical Applications in Genetics and Molecular Biology, 3, Article 15.
Victor, N., (1982). Exploratory data analysis and clinical research. Methods of Information in Medicine, 21, 53–54.
Yang, H. Y., & Speed, T. (2003). Design and analysis of comparative microarray experiments. In T. Speed (ed.), Statistical analysis of gene expression microarray data (pp. 35–92). Boca Raton, FL: Chapman and Hall.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Gordon, A.Y. (2020). Multiple Testing Procedures: Monotonicity and Some of Its Implications. In: Almudevar, A., Oakes, D., Hall, J. (eds) Statistical Modeling for Biological Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-34675-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-34675-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34674-4
Online ISBN: 978-3-030-34675-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)