Abstract
Consider the problem of simultaneously testing null hypotheses H 1, … H s . The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, particularly if s- is large, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which we call the k-FWER. We derive both single-step and stepdown procedures that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the k-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] controls E(FDP). Here, we construct methods such that, for any γ and α, P{FDP > γ} ≤ α. Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of p-values, while the second is more conservative but holds without any dependence assumptions.
Received December 2003; revised June 2004.
AMS 2000 subject classifications. Primary 621 15; secondary 62G I 0.
Chapter PDF
Similar content being viewed by others
Key words and phrases
References
Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300.
Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035–1061.
HOCHBERG, Y. and TAMHANE, A. (1987). Multiple Comparison Procedures. Wiley, New York.
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65–70.
Hommel, G. (1983). Tests of the overall hypothesis for arbitrary dependence structures. Biometrical J. 25 423–430.
HOMMEL, G. and HOFFMAN, T. (1988). Controlled uncertainty. In Multiple Hypotheses Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154–161. Springer, Heidelberg.
Korn, E., Troendle, J., McShane, L. and Simon, R. (2004). Controlling the number of false discoveries: Application to high-dimensional genomic data. J. Statist. Plann. Inference 124 379–398.
Perone Pacifico, M., Genovese, C., Verdinelli, I. and Wasserman, L. (2004). False discovery control for random fields. J Amer. Statist. Assoc. 99 1002–1014.
Sarkar, S. (1998). Some probability inequalities for ordered MTP2 random variables: A proof of the Simes conjecture. Ann. Statist. 26 494–504.
Sarkar, S. and Chang, C. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601–1608.
Simes, R. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751–754.
van der Laan, M., Dudoit, S. and Pollard, K. (2004). Multiple testing. Part III. Procedures for control of the generalized family-wise error rate and proportion of false positives. Working Paper Series, Paper 141, Div. Biostatistics, Univ. California, Berkeley.
VICTOR, N. (1982). Exploratory data analysis and clinical research. Methods of Information in Medicine 21 53–54.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
This chapter is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
Copyright information
© 2012 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Lehmann, E.L., Romano, J.P. (2012). Generalizations of the Familywise Error Rate. In: Rojo, J. (eds) Selected Works of E. L. Lehmann. Selected Works in Probability and Statistics. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-1412-4_57
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1412-4_57
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4614-1411-7
Online ISBN: 978-1-4614-1412-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)