Skip to main content
SpringerLink
Log in

Classes of multiple decision functions strongly controlling FWER and FDR

  • Published:
Metrika Aims and scope Submit manuscript

Abstract

Two general classes of multiple decision functions, where each member of the first class strongly controls the family-wise error rate (FWER), while each member of the second class strongly controls the false discovery rate (FDR), are described. These classes offer the possibility that optimal multiple decision functions with respect to a pre-specified Type II error criterion, such as the missed discovery rate (MDR), could be found which control the FWER or FDR Type I error rates. The gain in MDR of the associated FDR-controlling procedure relative to the well-known Benjamini–Hochberg procedure is demonstrated via a modest simulation study with gamma-distributed component data. Such multiple decision functions may have the potential of being utilized in multiple testing, specifically in the analysis of high-dimensional data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Henceforth, decreasing will mean non-increasing, while increasing will mean non-decreasing

References

  • Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 57(1):289–300

    MATH  MathSciNet  Google Scholar 

  • Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29(4):1165–1188

    Article  MATH  MathSciNet  Google Scholar 

  • Blanchard G, Roquain E (2008) Two simple sufficient conditions for FDR control. Electron J Stat 2:963–992

    Article  MATH  MathSciNet  Google Scholar 

  • Blanchard G, Roquain É (2009) Adaptive false discovery rate control under independence and dependence. J Mach Learn Res 10:2837–2871

    MATH  MathSciNet  Google Scholar 

  • Bogdan M, Chakrabarti A, Frommlet F, Ghosh JK (2011) Asymptotic Bayes-optimality under sparsity of some multiple testing procedures. Ann Stat 39(3):1551–1579

    Article  MATH  MathSciNet  Google Scholar 

  • Doob JL (1953) Stochastic processes. Wiley, New York

    MATH  Google Scholar 

  • Dudoit S, van der Laan MJ (2008) Multiple testing procedures with applications to genomics. Springer series in statistics. Springer, New York

    Book  Google Scholar 

  • Efron B (2008) Microarrays, empirical Bayes and the two-groups model. Stat Sci 23(1):1–22

    Article  MathSciNet  Google Scholar 

  • Efron B (2008) Simultaneous inference: when should hypothesis testing problems be combined? Ann Appl Stat 1:197–223

    Article  MathSciNet  Google Scholar 

  • Finner H, Dickhaus T, Roters M (2009) On the false discovery rate and an asymptotically optimal rejection curve. Ann Stat 37(2):596–618

    Article  MATH  MathSciNet  Google Scholar 

  • Genovese CR, Roeder K, Wasserman L (2006) False discovery control with \(p\)-value weighting. Biometrika 93(3):509–524

    Article  MATH  MathSciNet  Google Scholar 

  • Genovese C, Wasserman L (2003) Bayesian and frequentist multiple testing. In: Bayesian statistics, 7 (Tenerife, 2002), pp. 145–161. Oxford Univ. Press, New York. With discussions by Merlise A. Clyde and Christian P. Robert and Judith Rousseau, and a reply by the authors.

  • Goeman JJ, Solari A (2010) The sequential rejection principle of familywise error control. Ann Stat 38(6):3782–3810

    Article  MATH  MathSciNet  Google Scholar 

  • Guindani M, Muller P, Zhang S (2009) A Bayesian discovery procedure. JRSS B 71:905–925

    Article  MathSciNet  Google Scholar 

  • Habiger JD, Peña EA (2011) Randomized \(P\)-values and nonparametric procedures in multiple testing. J Nonparametric Stat 23(3):583–604

    Article  MATH  Google Scholar 

  • Habiger JD, Peña EA (2014) Compound \(p\)-value statistics for multiple testing procedures. J Multivar Anal 126:153–166. doi: 10.1016/j.jmva.2014.01.007

    Article  MATH  Google Scholar 

  • Hoeffding W (1956) On the distribution of the number of successes in independent trials. Ann Math Stat 27:713–721

    Article  MATH  MathSciNet  Google Scholar 

  • Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70

    MATH  MathSciNet  Google Scholar 

  • Kang G, Ye K, Liu N, Allison D, Gao G (2009) Weighted multiple hypothesis testing procedures. Stat Appl Genet Mol Biol 8:1–21

    MathSciNet  Google Scholar 

  • Müller P, Parmigiani G, Robert C, Rousseau J (2004) Optimal sample size for multiple testing: the case of gene expression microarrays. J Am Stat Assoc 99(468):990–1001

    Article  MATH  Google Scholar 

  • Peña EA, Habiger JD, Wu W (2011) Supplement to ‘power-enhanced multiple decision functions controlling family-wise error and false discovery rates’. doi:10.1214/10-AOS844SUPP

  • Peña EA, Habiger JD, Wu W (2011) Power-enhanced multiple decision functions controlling family-wise error and false discovery rates. Ann Stat 39(1):556–583

    Article  MATH  Google Scholar 

  • Roeder K, Wasserman L (2009) Genome-wide significance levels and weighted hypothesis testing. Stat Sci 24(4):398–413

    Article  MathSciNet  Google Scholar 

  • Roquain E, van de Wiel MA (2009) Optimal weighting for false discovery rate control. Electron J Stat 3:678–711

    Article  MATH  MathSciNet  Google Scholar 

  • Sarkar SK (2008) On methods controlling the false discovery rate. Sankhyā 70(2, Ser. A):135–168

  • Sarkar SK, Chang CK (1997) The Simes method for multiple hypothesis testing with positively dependent test statistics. J Am Stat Assoc 92(440):1601–1608

    Article  MATH  MathSciNet  Google Scholar 

  • Sarkar SK (2007) Stepup procedures controlling generalized FWER and generalized FDR. Ann Stat 35(6):2405–2420

    Article  MATH  Google Scholar 

  • Sarkar SK, Zhou T, Ghosh D (2008) A general decision theoretic formulation of procedures controlling FDR and FNR from a Bayesian perspective. Stat Sin 18(3):925–945

    MATH  MathSciNet  Google Scholar 

  • Scott J, Berger J (2006) An exploration of aspects of Bayesian multiple testing. J Stat Plan Inference 136:2144–2162

    Article  MATH  MathSciNet  Google Scholar 

  • Šidák Z (1967) Rectangular confidence regions for the means of multivariate normal distributions. J Am Stat Assoc 62:626–633

    MATH  Google Scholar 

  • Storey J (2007) The optimal discovery procedure: a new approach to simultaneous significance testing. J R Stat Soc Ser B 69:347–368

    Article  MathSciNet  Google Scholar 

  • Sun W, Cai T (2007) Oracle and adaptive compound decision rules for false discovery rate control. J Am Stat Assoc 102:901–912

    Article  MATH  MathSciNet  Google Scholar 

  • Westfall PH, Krishen A, Young SS (1998) Using prior information to allocate significance levels for multiple endpoints. Stat Med 17:2107–2119

    Article  Google Scholar 

  • Westfall PH, Krishen A (2001) Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. J Stat Plan Inference 99(1):25–40

    Article  MATH  MathSciNet  Google Scholar 

  • Wu W, Peña EA (2013) Bayes multiple decision functions. Electron J Stat 7:1272–1300. doi:10.1214/13-EJS813

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We thank Professor Sanat Sarkar for helpful discussions and sincerely thank the reviewers of this work for their sharp and critical comments which were extremely helpful in improving the manuscript. We very much thank Metrika editors, Professor Norbert Henze and Professor Udo Kamps, for providing an outlet for this work.

Author information

Authors and Affiliations

  1. Department of Statistics, University of South Carolina, Columbia, SC, 29208, USA

    Edsel A. Peña

  2. Department of Statistics, Oklahoma State University, Stillwater, OK, 74078, USA

    Joshua D. Habiger

  3. Department of Mathematics and Statistics, Florida International University, Miami, FL, 33199, USA

    Wensong Wu

Authors
  1. Edsel A. Peña
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua D. Habiger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wensong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edsel A. Peña.

Additional information

The authors acknowledge support from National Science Foundation (NSF) Grants DMS 0805809 and DMS 1106435, National Institutes of Health (NIH) Grants P20RR17698, R01CA154731, and P30GM103336-01A1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peña, E.A., Habiger, J.D. & Wu, W. Classes of multiple decision functions strongly controlling FWER and FDR. Metrika 78, 563–595 (2015). https://doi.org/10.1007/s00184-014-0516-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00184-014-0516-6

Keywords

Mathematics Subject Classification

Search

Navigation