False Discovery Rate
Multiple Hypothesis Testing
In hypothesis testing, statistical significance is typically based on calculations involving p-values and Type I error rates. A p-value calculated from a single statistical hypothesis test can be used to determine whether there is statistically significant evidence against the null hypothesis. The upper threshold applied to the p-value in making this determination (often 5% in the scientific literature) determines the Type I error rate; i.e., the probability of making a Type I error when the null hypothesis is true. Multiple hypothesis testing is concerned with testing several statistical hypotheses simultaneously. Defining statistical significance is a more complex problem in this setting.
A longstanding definition of statistical significance for multiple hypothesis tests involves the probability of making one or more Type I errors among the family of hypothesis tests, called the family-wise error rate. However, there exist other well established...
References and Further Reading
- Leek JT, Storey JD (2007) Capturing heterogeneity in gene expression studies by surrogate variable analysis. PLoS Genet 3:e161Google Scholar
- Leek JT, Storey JD (2008) A general framework for multiple testing dependence. Proc Natl Acad Sci 105:18718–18723Google Scholar
- Morton NE (1955) Sequential tests for the detection of linkage. Am J Hum Genet 7:277–318Google Scholar
- Shaffer J (1995) Multiple hypothesis testing. Ann Rev Psychol 46:561–584Google Scholar
- Soric B (1989) Statistical discoveries and effect-size estimation. J Am Stat Assoc 84:608–610Google Scholar
- Storey JD (2001) The positive false discovery rate: a Bayesian interpretation and the q-value. Technical Report 2001–2012, Department of Statistics, Stanford UniversityGoogle Scholar
- Zaykin DV, Young SS, Westfall PH (1998) Using the false discovery approach in the genetic dissection of complex traits: a response to weller et al. Genetics 150:1917–1918Google Scholar