Statistical Papers

, Volume 58, Issue 3, pp 775–789 | Cite as

On the Simes test under dependence

  • H. Finner
  • M. Roters
  • K. Strassburger
Regular Article


In 1986, R. J. Simes proposed a modified Bonferroni test procedure for testing an overall null hypothesis in multiple testing problems, nowadays referred to as the Simes test. The paper of Simes may be considered as a basic step in the development of many new test procedures and new error rate criteria as for example control of the false discovery rate. A key issue is the validity of the Simes test and the underlying Simes inequality under dependence. Although it has been proved that the Simes inequality is valid under suitable assumptions on dependence structures, important cases are not covered yet. In this note we investigate p-values based on exchangeable test statistics in order to explore reasons for the validity or failure of the Simes inequality. We provide sufficient conditions for the asymptotic validity of the Simes inequality and its possible strictness. We also show by means of an easy-to-compute counterexample that exchangeability by itself is not sufficient for the validity of the Simes inequality.


Exchangeable random variables Multivariate total positivity of order 2 Positive regression dependence  Simes inequality 

Mathematics Subject Classification

62J15 62F03 62F05 



This work was supported by the Ministry of Science and Research of the State of North Rhine-Westphalia (MIWF NRW) and the German Federal Ministry of Health (BMG).


  1. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57:289–300MathSciNetzbMATHGoogle Scholar
  2. Barlow RE, Proschan F (1975) Statistical theory of reliability and life testing. Holt, Rinehart and Winston, New YorkzbMATHGoogle Scholar
  3. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29:1165–1188MathSciNetCrossRefzbMATHGoogle Scholar
  4. Daniels HE (1945) The statistical theory of the strength of bundles of threads. Proc R Soc Lond A 183:405–435MathSciNetCrossRefzbMATHGoogle Scholar
  5. Fang Z, Hu T (1997) Developments on MTP2 properties of absolute value multinormal variables with nonzero means. Acta Math Applicatae Sinica 13:376–384MathSciNetCrossRefzbMATHGoogle Scholar
  6. Finner H, Dickhaus T, Roters M (2007) Dependency and false discovery rate: asymptotics. Ann Stat 35:1432–1455MathSciNetCrossRefzbMATHGoogle Scholar
  7. Finner H, Dickhaus T, Roters M (2009) On the false discovery rate and an asymptotically optimal rejection curve. Ann Stat 37:596–618MathSciNetCrossRefzbMATHGoogle Scholar
  8. Finner H, Gontscharuk V (2013) Asymptotic FDR control under weak dependence: a counterexample. Stat Probab Lett 83(8):1888–1893MathSciNetCrossRefzbMATHGoogle Scholar
  9. Guo W, Rao MB (2008) On control of the false discovery rate under no assumption of dependency. J Stat Plan Inference 138:3176–3188MathSciNetCrossRefzbMATHGoogle Scholar
  10. Heesen P (2014) Adaptive step up tests for the false discovery rate (FDR) under independence and dependence. Dissertation, University of DüsseldorfGoogle Scholar
  11. Hochberg Y (1988) A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75:800–802MathSciNetCrossRefzbMATHGoogle Scholar
  12. Karlin S, Rinott Y (1980) Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions. J Multivar Anal 10:467–498MathSciNetCrossRefzbMATHGoogle Scholar
  13. Karlin S, Rinott Y (1981) Total positivity properties of absolute value multinormal variables with applications to confidence interval estimates and related probabilistic inequalities. Ann Stat 9:1035–1049MathSciNetCrossRefzbMATHGoogle Scholar
  14. Karlin S, Rinott Y (1983) M-Matrices as covariance matrices of multinormal distributions. Linear Algebra Appl 52–53:419–438MathSciNetCrossRefzbMATHGoogle Scholar
  15. Sarkar TK (1969) Some lower bounds of reliability. Tech Report No 124, Dept of Operations Research and Statistics, Stanford UniversityGoogle Scholar
  16. Sarkar SK, Chang CK (1997) The Simes method for multiple hypothesis testing with positively dependent statistics. J Am Stat Assoc 92:1601–1608MathSciNetCrossRefzbMATHGoogle Scholar
  17. Sarkar SK (1998) Some probability inequalities for ordered MTP2 random variables: a proof of the Simes conjecture. Ann Stat 26:494–504CrossRefzbMATHGoogle Scholar
  18. Sarkar SK (2002) Some results on false discovery rate in stepwise multiple testing procedures. Ann Stat 30:239–257MathSciNetCrossRefzbMATHGoogle Scholar
  19. Shorack GR, Wellner JA (2009) Empirical processes with applications to statistics. Classics in applied mathematics 59, SIAM edition, originally published by Wiley 1986Google Scholar
  20. Simes RJ (1986) An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751–754MathSciNetCrossRefzbMATHGoogle Scholar
  21. Yekutieli D (2008) False discovery rate control for non-positively regression dependent test statistics. J Stat Plan Inference 138:405–415MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Institute for Biometrics and Epidemiology, German Diabetes CenterLeibniz Institute for Diabetes Research at Heinrich-Heine-University DüsseldorfDüsseldorfGermany
  2. 2.Fachbereich IV – MathematikUniversität TrierTrierGermany

Personalised recommendations