Adjustment for Multiplicity

  • Daniel Yekutieli
  • Dan Lin
  • Ziv Shkedy
  • Dhammika Amaratunga
Part of the Use R! book series (USE R)


The number of comparisons of gene expression level studied in a microarray experiment has been growing literarily at an exponential rate since the beginning of the 1990s. Considering a microarray data analyzed by testing each gene, multiple testing is an immediate concern. When many hypotheses are tested, the probability that a type I error is committed increases sharply with the number of hypotheses. This problem of multiplicity is not unique to microarray technology, yet its magnitude here, where a single experiment may involve many thousands of genes, dramatically intensifies the problem.In this chapter, we discuss a few procedures controlling for the FWER, such as the Bonferroni, Holm, and the maxT procedures. However, the focus of this chapter is controlling the FDR criterion, since it admits a more powerful outcome. We discuss several variations of the Benjamini and Hochberg step-up procedure (BH-FDR 1995), the permutation-based FDR controlling procedures, and the significance analysis of microarrays (SAM) approach of Tusher et al. (Proc Natl Acad Sci 98:5116–5121, 2001) and the Efron et al. (J Am Stat Assoc 96:1151–1160, 2001), and Storey (A direct approach to false discovery rates. Technical Report. Stanford University, Stanford, 2001) Bayesian interpretation of the FDR within the context of microarray data.


False Discovery Rate Permutation Matrix True Null Hypothesis False Null Hypothesis Animal Behavior Study 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Affymetrix GeneChip. (2004). Expression analysis technical manual, Rev.4. Santa Clara: Affymetrix.Google Scholar
  2. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of Royal Statistical Soceity B, 57, 289–300.Google Scholar
  3. Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006). Adaptive linear step-up false discovery rate controlling procedures. Biometrika, 93(3), 491–507.Google Scholar
  4. Benjamini, Y., & Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annal of Statistics, 29(4), 1165–1188.Google Scholar
  5. Dudoit, S., Fridlyand, J., & Speed, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association, 98, 77–87.Google Scholar
  6. Efron, B., Tibshirani, R., Storey, J. D., & Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association, 96, 1151–1160.Google Scholar
  7. Ge, Y., Dudoit, S., & Speed, P. T. (2003). Resampling based multiple testing for microarray data analysis (Technical report, 633). Berkeley: University of Berkeley.Google Scholar
  8. Hochberg, Y., & Tamhane, Y. C. (1987). Multiple comparison procedures. New York: Wiley.Google Scholar
  9. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70.Google Scholar
  10. Hommel, G., & Hoffman, T. (1998). Controlled uncertainty. In P. Bauer, G. Hommel, & E. Sonnemann (Eds.), Multiple hypotheses testing (pp. 154–161). Heidelberg: Springer.Google Scholar
  11. Lehmann, E.L., & Romano, J. P. (2005). Generalizations of the familywise error rate. Ann. Statist., 33(3), 1138–1154.Google Scholar
  12. Reiner, A., Yekutieli, D., & Benjamini, Y. (2003). Identifying differentially expressed genes using false discovery rate controlling procedures. Bioinformatics, 19(3), 368–375.Google Scholar
  13. Sarkar, S. K. (2007). Procedures controlling generalized FWER and generalized FDR. The Annals of Statistics, 35(6), 2405–2420.Google Scholar
  14. Smyth, G. K. (2004) Linear models and empirical Bayes methods for assessing differential expression in microarray experiments. Statistical Applications in Genetics and Molecular Biology, 3(1), Article 3.Google Scholar
  15. Storey, J. D. (2002) A direct approach to false discovery rates. Journal of the Royal Statistical Society B, 64(Pt 3), 479–498.Google Scholar
  16. Storey, J. D. (2003) The positive false discovery rate: A Bayesian interpretation and the q-value. The Annals of Statistics, 31(6), 2013–2035.Google Scholar
  17. Szechtman, H., Sulis, W., & Eilam, D. (1998). Quinpirole induces compulsive checking behavior in rats: A potential animal model of obsessive-compulsive disorder (OCD). Behavioral Neuroscience, 112, 1475–1485.Google Scholar
  18. Tusher, V. G., Tibshirani, R., & Chu, G. (2001). Significance analysis of microarrys applied to the ionizing radiation response, Proceedings of the National Academy of Sciences, 98, 5116–5121.Google Scholar
  19. Westfall, P. H., & Young, S. S. (1993). Resampling based multiple testing. New York: John Wiley & Sons.Google Scholar
  20. Xu, H., & Hsu, J. C. (2007). Using the partitioning principle to control the generalized family error rate. Biometrical Journal, 49, 52–67.Google Scholar
  21. Yekutieli, D., & Benjamini, Y. (1999). Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. Journal of Statistical Planning and Inference, 82, 171–196.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Daniel Yekutieli
    • 1
  • Dan Lin
    • 2
  • Ziv Shkedy
    • 3
  • Dhammika Amaratunga
    • 4
  1. 1.Department of Statistics and Operations Research, School of Mathematical SciencesTel-Aviv UniversityTel-AvivIsrael
  2. 2.Veterinary Medicine Research and DevelopmentPfizer Animal HealthZaventemBelgium
  3. 3.Interuniversity Institute for Biostatistics and Statistical Bioinformatics (I-BioStat), Center for Statistics (CenStat)Hasselt UniversityDiepenbeekBelgium
  4. 4.Biostatistics and ProgrammingJanssen Pharmaceutical Companies of Johnson & JohnsonRaritanUSA

Personalised recommendations