, 17:417

Control of the false discovery rate under dependence using the bootstrap and subsampling

  • Joseph P. Romano
  • Azeem M. Shaikh
  • Michael Wolf
Invited Paper

DOI: 10.1007/s11749-008-0126-6

Cite this article as:
Romano, J.P., Shaikh, A.M. & Wolf, M. TEST (2008) 17: 417. doi:10.1007/s11749-008-0126-6


This paper considers the problem of testing s null hypotheses simultaneously while controlling the false discovery rate (FDR). Benjamini and Hochberg (J. R. Stat. Soc. Ser. B 57(1):289–300, 1995) provide a method for controlling the FDR based on p-values for each of the null hypotheses under the assumption that the p-values are independent. Subsequent research has since shown that this procedure is valid under weaker assumptions on the joint distribution of the p-values. Related procedures that are valid under no assumptions on the joint distribution of the p-values have also been developed. None of these procedures, however, incorporate information about the dependence structure of the test statistics. This paper develops methods for control of the FDR under weak assumptions that incorporate such information and, by doing so, are better able to detect false null hypotheses. We illustrate this property via a simulation study and two empirical applications. In particular, the bootstrap method is competitive with methods that require independence if independence holds, but it outperforms these methods under dependence.


Bootstrap Subsampling False discovery rate Multiple testing Stepdown procedure 

Mathematics Subject Classification (2000)

62G09 62G10 62G20 62H15 

Copyright information

© Sociedad de Estadística e Investigación Operativa 2008

Authors and Affiliations

  • Joseph P. Romano
    • 1
  • Azeem M. Shaikh
    • 2
  • Michael Wolf
    • 3
  1. 1.Departments of Economics and StatisticsStanford UniversityStanfordUSA
  2. 2.Department of EconomicsUniversity of ChicagoChicagoUSA
  3. 3.Institute for Empirical Research in EconomicsUniversity of ZurichZurichSwitzerland