AStA Advances in Statistical Analysis

, Volume 99, Issue 3, pp 281–310 | Cite as

Uncertainty quantification for the family-wise error rate in multivariate copula models

Original Paper


We derive confidence regions for the realized family-wise error rate (FWER) of certain multiple tests which are empirically calibrated at a given (global) level of significance. To this end, we regard the FWER as a derived parameter of a multivariate parametric copula model. It turns out that the resulting confidence regions are typically very much concentrated around the target FWER level, while generic multiple tests with fixed thresholds are in general not FWER-exhausting. Since FWER level exhaustion and optimization of power are equivalent for the classes of multiple test problems studied in this paper, the aforementioned findings militate strongly in favor of estimating the dependency structure (i.e., copula) and incorporating it in a multivariate multiple test procedure. We illustrate our theoretical results by considering two particular classes of multiple test problems of practical relevance in detail, namely multiple tests for components of a mean vector and multiple support tests.


Delta method Gumbel–Hougaard copula Multiple testing Simultaneous test procedure Subset pivotality 


  1. Bernstein, S.: Sur les fonctions absolument monotones. Acta Math. 52(1), 1–66 (1929). doi: 10.1007/BF02592679
  2. Bickel, P., Götze, F., van Zwet, W.: Resampling fewer than \(n\) observations: gains, losses, and remedies for losses. Stat. Sin. 7(1), 1–31 (1997)MATHGoogle Scholar
  3. Bickel, P.J., Freedman, D.: Some asymptotic theory for the bootstrap. Ann. Stat. 9, 1196–1217 (1981). doi: 10.1214/aos/1176345637 MathSciNetCrossRefMATHGoogle Scholar
  4. Block, H.W., Costigan, T., Sampson, A.R.: Product-type probability bounds of higher order. Probab. Eng. Inf. Sci. 6(3), 349–370 (1992). doi: 10.1017/S0269964800002588 CrossRefMATHGoogle Scholar
  5. Bodnar, T., Dickhaus, T.: False discovery rate control under Archimedean copula. Electron. J. Stat. 8(2), 2207–2241 (2014)Google Scholar
  6. Bodnar, T., Schmid, W.: A test for the weights of the global minimum variance portfolio in an elliptical model. Metrika 67, 127–143 (2008)MathSciNetCrossRefGoogle Scholar
  7. Bonferroni, C.E.: Il calcolo delle assicurazioni su gruppi di teste. Studi in onore Salvatore Ortu Carboni 13–60 (1935)Google Scholar
  8. Bonferroni, C.E.: Teoria statistica delle classi e calcolo delle probabilita. Pubbl. d. R. Ist. Super. di Sci. Econ. e Commerciali di Firenze, vol. 8. Libr. Internaz. Seeber, Firenze (1936)Google Scholar
  9. Cottin, C., Pfeifer, D.: From Bernstein polynomials to Bernstein copulas. J. Appl. Funct. Anal. 9(3–4), 277–288 (2014)MathSciNetGoogle Scholar
  10. DasGupta, A.: Asymptotic Theory of Statistics and Probability. Springer Texts in Statistics. Springer, New York (2008)Google Scholar
  11. Dickhaus, T.: Randomized \(p\)-values for multiple testing of composite null hypotheses. J. Stat. Plan. Inference 143(11), 1968–1979 (2013)MathSciNetCrossRefMATHGoogle Scholar
  12. Dickhaus, T.: Simultaneous Statistical Inference with Applications in the Life Sciences. Springer, Berlin (2014)CrossRefMATHGoogle Scholar
  13. Dickhaus, T., Gierl, J.: Simultaneous test procedures in terms of \(p\)-value copulae. In: Global Science and Technology Forum (GSTF), Proceedings on the 2nd Annual International Conference on Computational Mathematics, Computational Geometry and Statistics (CMCGS’13), pp. 75–80 (2013)Google Scholar
  14. Dickhaus, T., Royen, T.: On multivariate chi-square distributions and their applications in testing multiple hypotheses. WIAS Preprint No. 1913, Weierstrass Institute for Applied Analysis and Stochastics, Berlin (2014).
  15. Dickhaus, T., Stange, J.: Multiple point hypothesis test problems and effective numbers of tests for control of the family-wise error rate. Calcutta Stat. Assoc. Bull. 65(257–260), 123–144 (2013)Google Scholar
  16. Diers, D., Eling, M., Marek, S.D.: Dependence modeling in non-life insurance using the Bernstein copula. Insur. Math. Econ. 50(3), 430–436 (2012). doi: 10.1016/j.insmatheco.2012.02.007 CrossRefMATHGoogle Scholar
  17. Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7, 1–26 (1979). doi: 10.1214/aos/1176344552 MathSciNetCrossRefMATHGoogle Scholar
  18. Finner, H.: Testing Multiple Hypotheses: General Theory, Specific Problems, and Relationships to Other Multiple Decision Procedures. Fachbereich IV, Universität Trier, Habilitationsschrift (1994)Google Scholar
  19. Gabriel, K.R.: Simultaneous test procedures—some theory of multiple comparisons. Ann. Math. Stat. 40, 224–250 (1969). doi: 10.1214/aoms/1177697819 MathSciNetCrossRefMATHGoogle Scholar
  20. Genest, C., Rivest, L.P.: A characterization of Gumbel’s family of extreme value distributions. Stat. Probab. Lett. 8(3), 207–211 (1989)MathSciNetCrossRefMATHGoogle Scholar
  21. Genest, C., Nešlehová, J.: Estimators based on Kendall’s tau in multivariate copula models. Aust. N. Z. J. Stat. 53(2), 157–177 (2011). doi: 10.1111/j.1467-842X.2011.00622.x MathSciNetCrossRefMATHGoogle Scholar
  22. Genz, A., Bretz, F.: Computation of multivariate normal and \(t\) probabilities. In: Lecture Notes in Statistics, vol. 195. Springer, Berlin (2009). doi: 10.1007/978-3-642-01689-9
  23. Ghosh, D.: Generalized Benjamini–Hochberg Procedures Using Spacings. Technical Report, Penn State University, USA (2011)Google Scholar
  24. Gudendorf, G., Segers, J.: Extreme-value copulas. In: Jaworski, P., Durante, F., Härdle, W.K., Rychlik, T. (eds.) Copula Theory and its Applications. Lecture Notes in Statistics, pp. 127–145. Springer, Berlin (2010). doi: 10.1007/978-3-642-12465-5_6
  25. Gupta, A., Varga, T., Bodnar, T.: Elliptically Contoured Models in Statistics and Portfolio Theory. Springer, New York (2013)Google Scholar
  26. Hansen, L.P.: Large sample properties of generalized method of moments estimators. Econometrica 50, 1029–1054 (1982). doi: 10.2307/1912775 MathSciNetCrossRefMATHGoogle Scholar
  27. Hofert, M.: A stochastic representation and sampling algorithm for nested Archimedean copulas. J. Stat. Computation and Simulation 82(9), 1239–1255 (2012). doi: 10.1080/00949655.2011.574632
  28. Hofert, M., Mächler, M., McNeil, A.J.: Likelihood inference for Archimedean copulas in high dimensions under known margins. J. Multivar. Anal. 110, 133–150 (2012). doi: 10.1016/j.jmva.2012.02.019 CrossRefMATHGoogle Scholar
  29. Hothorn, T., Bretz, F., Westfall, P.: Simultaneous inference in general parametric models. Biom. J. 50(3), 346–363 (2008)MathSciNetCrossRefGoogle Scholar
  30. Janssen, P., Swanepoel, J., Veraverbeke, N.: Large sample behavior of the Bernstein copula estimator. J. Stat. Plan. Inference 142(5), 1189–1197 (2012). doi: 10.1016/j.jspi.2011.11.020 MathSciNetCrossRefMATHGoogle Scholar
  31. Janssen, P., Swanepoel, J., Veraverbeke, N.: A note on the asymptotic behavior of the Bernstein estimator of the copula density. J. Multivar. Anal. 124, 480–487 (2014). doi: 10.1016/j.jmva.2013.10.009 MathSciNetCrossRefMATHGoogle Scholar
  32. Kendall, M.G.: A New Measure of Rank Correlation, vol. 30, pp. 81–93. Biometrika, Cambridge (1938)Google Scholar
  33. Kendall, M.G., Babington Smith, B.: On the Method of Paired Comparisons, vol. 31, pp. 324–345. Biometrika, Cambridge (1940)Google Scholar
  34. Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005). doi: 10.1007/0-387-27605-X
  35. Littell, R.C., Pendergast, J., Natarajan, R.: Modelling covariance structure in the analysis of repeated measures data. Stat. Med. 19(13), 1793–1819 (2000)CrossRefGoogle Scholar
  36. Longin, F.M.: From value at risk to stress testing: the extreme value approach. J. Bank. Financ. 24, 1097–1130 (2000)CrossRefGoogle Scholar
  37. McNeil, A.J., Nešlehová, J.: Multivariate Archimedean copulas, \(d\)-monotone functions and \(\ell _{1}\)-norm symmetric distributions. Ann. Stat. 37, 3059–3097 (2009). doi:  10.1214/07-AOS556 CrossRefMATHGoogle Scholar
  38. McNeil, A.J., Frey, R., Embrechts, P.: Quantitative Risk Management: Concepts, Techniques and Tools. Princeton University Press, Princeton (2005)Google Scholar
  39. Meinshausen, N., Maathuis, M.H., Bühlmann, P.: Asymptotic optimality of the Westfall–Young permutation procedure for multiple testing under dependence. Ann. Stat. 39(6), 3369–3391 (2011). doi: 10.1214/11-AOS946 CrossRefMATHGoogle Scholar
  40. Nelsen, R.B.: An introduction to copulas. In: Springer Series in Statistics, 2nd edn. Springer, New York (2006)Google Scholar
  41. Nikoloulopoulos, A.K., Joe, H., Li, H.: Extreme value properties of multivariate \(t\) copulas. Extremes 12(2), 129–148 (2009). doi:  10.1007/s10687-008-0072-4 MathSciNetCrossRefMATHGoogle Scholar
  42. Roeder, K., Wasserman, L.: Genome-wide significance levels and weighted hypothesis testing. Stat. Sci. 24(4), 398–413 (2009)MathSciNetCrossRefGoogle Scholar
  43. Romano, J.P., Wolf, M.: Exact and approximate stepdown methods for multiple hypothesis testing. J. Am. Stat. Assoc. 100(469), 94–108 (2005). doi: 10.1198/016214504000000539 MathSciNetCrossRefMATHGoogle Scholar
  44. Sancetta, A., Satchell, S.: The Bernstein copula and its applications to modeling and approximations of multivariate distributions. Econ. Theory 20(3), 535–562 (2004). doi: 10.1017/S026646660420305X MathSciNetCrossRefMATHGoogle Scholar
  45. Sarkar, S.: Rejoinder: on methods controlling the false discovery rate. Sankhyā Indian J. Stat. Ser. A 70(2), 183–185 (2008)Google Scholar
  46. Šidák, Z.: Rectangular confidence regions for the means of multivariate normal distributions. J. Am. Stat. Assoc. 62, 626–633 (1967). doi: 10.2307/2283989 MATHGoogle Scholar
  47. Singer, J.D., Willett, J.B.: Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford University Press, Oxford (2003)Google Scholar
  48. Sklar, A.: Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Stat. Univ. Paris 8, 229–231 (1959)MathSciNetGoogle Scholar
  49. Sklar, A.: Random variables, distribution functions, and copulas—a personal look backward and forward. In: Distributions with Fixed Marginals and Related Topics, pp. 1–14. Institute of Mathematical Statistics, Hayward (1996)Google Scholar
  50. Stange, J., Bodnar, T., Dickhaus, T.: Uncertainty quantification for the family-wise error rate in multivariate copula models. WIAS Preprint 1862, Weierstrass Institute for Applied Analysis and Stochastics, Berlin (2013).
  51. Swanepoel, J.W.: A note on proving that the (modified) bootstrap works. Commun. Stat. Theory Methods 15, 3193–3203 (1986). doi: 10.1080/03610928608829303 MathSciNetCrossRefMATHGoogle Scholar
  52. Westfall, P.H., Young, S.S.: Resampling-based multiple testing: examples and methods for \(p\)-value adjustment. In: Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics. Wiley, New York (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Weierstrass Institute for Applied Analysis and StochasticsBerlinGermany
  2. 2.Department of MathematicsHumboldt-UniversityBerlinGermany

Personalised recommendations