International Encyclopedia of Statistical Science

2011 Edition
| Editors: Miodrag Lovric

False Discovery Rate

  • John D. Storey
Reference work entry

Multiple Hypothesis Testing

In hypothesis testing, statistical significance is typically based on calculations involving  p-values and Type I error rates. A p-value calculated from a single statistical hypothesis test can be used to determine whether there is statistically significant evidence against the null hypothesis. The upper threshold applied to the p-value in making this determination (often 5% in the scientific literature) determines the Type I error rate; i.e., the probability of making a Type I error when the null hypothesis is true. Multiple hypothesis testing is concerned with testing several statistical hypotheses simultaneously. Defining statistical significance is a more complex problem in this setting.

A longstanding definition of statistical significance for multiple hypothesis tests involves the probability of making one or more Type I errors among the family of hypothesis tests, called the family-wise error rate. However, there exist other well established...

This is a preview of subscription content, log in to check access.

References and Further Reading

  1. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 85:289–300MathSciNetGoogle Scholar
  2. Benjamini Y, Liu W (1999) A step-down multiple hypothesis procedure that controls the false discovery rate under independence. J Stat Plann Infer 82:163–170zbMATHMathSciNetGoogle Scholar
  3. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29: 1165–1188zbMATHMathSciNetGoogle Scholar
  4. Devlin B, Roeder K (1999) Genomic control for association studies. Biometrics 55:997–1004zbMATHGoogle Scholar
  5. Efron B (2004) Large-scale simultaneous hypothesis testing: the choice of a null hypothesis. J Am Stat Assoc 99:96–104zbMATHMathSciNetGoogle Scholar
  6. Efron B, Tibshirani R, Storey JD, Tusher V (2001) Empirical Bayes analysis of a microarray experiment. J Am Stat Assoc 96: 1151–1160zbMATHMathSciNetGoogle Scholar
  7. Leek JT, Storey JD (2007) Capturing heterogeneity in gene expression studies by surrogate variable analysis. PLoS Genet 3:e161Google Scholar
  8. Leek JT, Storey JD (2008) A general framework for multiple testing dependence. Proc Natl Acad Sci 105:18718–18723Google Scholar
  9. Lindsay BG, Kettenring J, Siegmund DO (2004) A report on the future of statistics. Stat Sci 19:387–407zbMATHMathSciNetGoogle Scholar
  10. Morton NE (1955) Sequential tests for the detection of linkage. Am J Hum Genet 7:277–318Google Scholar
  11. Shaffer J (1995) Multiple hypothesis testing. Ann Rev Psychol 46:561–584Google Scholar
  12. Simes RJ (1986) An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751–754zbMATHMathSciNetGoogle Scholar
  13. Soric B (1989) Statistical discoveries and effect-size estimation. J Am Stat Assoc 84:608–610Google Scholar
  14. Storey JD (2001) The positive false discovery rate: a Bayesian interpretation and the q-value. Technical Report 2001–2012, Department of Statistics, Stanford UniversityGoogle Scholar
  15. Storey JD (2002) A direct approach to false discovery rates. J R Stat Soc Ser B 64:479–498zbMATHMathSciNetGoogle Scholar
  16. Storey JD (2003) The positive false discovery rate: a Bayesian interpretation and the q-value. Ann Stat 31:2013–2035zbMATHMathSciNetGoogle Scholar
  17. Storey JD (2007) The optimal discovery procedure: a new approach to simultaneous significance testing. J R Stat Soc Ser B 69: 347–368MathSciNetGoogle Scholar
  18. Storey JD, Dai JY, Leek JT (2007) The optimal discovery procedure for large-scale significance testing, with applications to comparative microarray experiments. Biostatistics 8:414–432zbMATHGoogle Scholar
  19. Storey JD, Taylor JE, Siegmund D (2004) Strong control, conservative point estimation, and simultaneous conservative consistency of false discovery rates: a unified approach. J R Stat Soc Ser B 66:187–205zbMATHMathSciNetGoogle Scholar
  20. Storey JD, Tibshirani R (2003) Statistical significance for genome-wide studies. Proc Natl Acad Sci 100:9440–9445zbMATHMathSciNetGoogle Scholar
  21. Zaykin DV, Young SS, Westfall PH (1998) Using the false discovery approach in the genetic dissection of complex traits: a response to weller et al. Genetics 150:1917–1918Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • John D. Storey
    • 1
  1. 1.Princeton UniversityPrincetonUSA