Skip to main content
SpringerLink
Log in

Uncertainty quantification for the family-wise error rate in multivariate copula models

  • Original Paper
  • Published:
AStA Advances in Statistical Analysis Aims and scope Submit manuscript

Abstract

We derive confidence regions for the realized family-wise error rate (FWER) of certain multiple tests which are empirically calibrated at a given (global) level of significance. To this end, we regard the FWER as a derived parameter of a multivariate parametric copula model. It turns out that the resulting confidence regions are typically very much concentrated around the target FWER level, while generic multiple tests with fixed thresholds are in general not FWER-exhausting. Since FWER level exhaustion and optimization of power are equivalent for the classes of multiple test problems studied in this paper, the aforementioned findings militate strongly in favor of estimating the dependency structure (i.e., copula) and incorporating it in a multivariate multiple test procedure. We illustrate our theoretical results by considering two particular classes of multiple test problems of practical relevance in detail, namely multiple tests for components of a mean vector and multiple support tests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Bernstein, S.: Sur les fonctions absolument monotones. Acta Math. 52(1), 1–66 (1929). doi:10.1007/BF02592679

  • Bickel, P., Götze, F., van Zwet, W.: Resampling fewer than \(n\) observations: gains, losses, and remedies for losses. Stat. Sin. 7(1), 1–31 (1997)

    MATH  Google Scholar 

  • Bickel, P.J., Freedman, D.: Some asymptotic theory for the bootstrap. Ann. Stat. 9, 1196–1217 (1981). doi:10.1214/aos/1176345637

    Article  MathSciNet  MATH  Google Scholar 

  • Block, H.W., Costigan, T., Sampson, A.R.: Product-type probability bounds of higher order. Probab. Eng. Inf. Sci. 6(3), 349–370 (1992). doi:10.1017/S0269964800002588

    Article  MATH  Google Scholar 

  • Bodnar, T., Dickhaus, T.: False discovery rate control under Archimedean copula. Electron. J. Stat. 8(2), 2207–2241 (2014)

  • Bodnar, T., Schmid, W.: A test for the weights of the global minimum variance portfolio in an elliptical model. Metrika 67, 127–143 (2008)

    Article  MathSciNet  Google Scholar 

  • Bonferroni, C.E.: Il calcolo delle assicurazioni su gruppi di teste. Studi in onore Salvatore Ortu Carboni 13–60 (1935)

  • Bonferroni, C.E.: Teoria statistica delle classi e calcolo delle probabilita. Pubbl. d. R. Ist. Super. di Sci. Econ. e Commerciali di Firenze, vol. 8. Libr. Internaz. Seeber, Firenze (1936)

  • Cottin, C., Pfeifer, D.: From Bernstein polynomials to Bernstein copulas. J. Appl. Funct. Anal. 9(3–4), 277–288 (2014)

    MathSciNet  Google Scholar 

  • DasGupta, A.: Asymptotic Theory of Statistics and Probability. Springer Texts in Statistics. Springer, New York (2008)

    Google Scholar 

  • Dickhaus, T.: Randomized \(p\)-values for multiple testing of composite null hypotheses. J. Stat. Plan. Inference 143(11), 1968–1979 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Dickhaus, T.: Simultaneous Statistical Inference with Applications in the Life Sciences. Springer, Berlin (2014)

    Book  MATH  Google Scholar 

  • Dickhaus, T., Gierl, J.: Simultaneous test procedures in terms of \(p\)-value copulae. In: Global Science and Technology Forum (GSTF), Proceedings on the 2nd Annual International Conference on Computational Mathematics, Computational Geometry and Statistics (CMCGS’13), pp. 75–80 (2013)

  • Dickhaus, T., Royen, T.: On multivariate chi-square distributions and their applications in testing multiple hypotheses. WIAS Preprint No. 1913, Weierstrass Institute for Applied Analysis and Stochastics, Berlin (2014). http://www.wias-berlin.de/preprint/1913/wias_preprints_1913.pdf

  • Dickhaus, T., Stange, J.: Multiple point hypothesis test problems and effective numbers of tests for control of the family-wise error rate. Calcutta Stat. Assoc. Bull. 65(257–260), 123–144 (2013)

  • Diers, D., Eling, M., Marek, S.D.: Dependence modeling in non-life insurance using the Bernstein copula. Insur. Math. Econ. 50(3), 430–436 (2012). doi:10.1016/j.insmatheco.2012.02.007

    Article  MATH  Google Scholar 

  • Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7, 1–26 (1979). doi:10.1214/aos/1176344552

    Article  MathSciNet  MATH  Google Scholar 

  • Finner, H.: Testing Multiple Hypotheses: General Theory, Specific Problems, and Relationships to Other Multiple Decision Procedures. Fachbereich IV, Universität Trier, Habilitationsschrift (1994)

  • Gabriel, K.R.: Simultaneous test procedures—some theory of multiple comparisons. Ann. Math. Stat. 40, 224–250 (1969). doi:10.1214/aoms/1177697819

    Article  MathSciNet  MATH  Google Scholar 

  • Genest, C., Rivest, L.P.: A characterization of Gumbel’s family of extreme value distributions. Stat. Probab. Lett. 8(3), 207–211 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Genest, C., Nešlehová, J.: Estimators based on Kendall’s tau in multivariate copula models. Aust. N. Z. J. Stat. 53(2), 157–177 (2011). doi:10.1111/j.1467-842X.2011.00622.x

    Article  MathSciNet  MATH  Google Scholar 

  • Genz, A., Bretz, F.: Computation of multivariate normal and \(t\) probabilities. In: Lecture Notes in Statistics, vol. 195. Springer, Berlin (2009). doi:10.1007/978-3-642-01689-9

  • Ghosh, D.: Generalized Benjamini–Hochberg Procedures Using Spacings. Technical Report, Penn State University, USA (2011)

  • Gudendorf, G., Segers, J.: Extreme-value copulas. In: Jaworski, P., Durante, F., Härdle, W.K., Rychlik, T. (eds.) Copula Theory and its Applications. Lecture Notes in Statistics, pp. 127–145. Springer, Berlin (2010). doi:10.1007/978-3-642-12465-5_6

  • Gupta, A., Varga, T., Bodnar, T.: Elliptically Contoured Models in Statistics and Portfolio Theory. Springer, New York (2013)

  • Hansen, L.P.: Large sample properties of generalized method of moments estimators. Econometrica 50, 1029–1054 (1982). doi:10.2307/1912775

    Article  MathSciNet  MATH  Google Scholar 

  • Hofert, M.: A stochastic representation and sampling algorithm for nested Archimedean copulas. J. Stat. Computation and Simulation 82(9), 1239–1255 (2012). doi:10.1080/00949655.2011.574632

  • Hofert, M., Mächler, M., McNeil, A.J.: Likelihood inference for Archimedean copulas in high dimensions under known margins. J. Multivar. Anal. 110, 133–150 (2012). doi:10.1016/j.jmva.2012.02.019

    Article  MATH  Google Scholar 

  • Hothorn, T., Bretz, F., Westfall, P.: Simultaneous inference in general parametric models. Biom. J. 50(3), 346–363 (2008)

    Article  MathSciNet  Google Scholar 

  • Janssen, P., Swanepoel, J., Veraverbeke, N.: Large sample behavior of the Bernstein copula estimator. J. Stat. Plan. Inference 142(5), 1189–1197 (2012). doi:10.1016/j.jspi.2011.11.020

    Article  MathSciNet  MATH  Google Scholar 

  • Janssen, P., Swanepoel, J., Veraverbeke, N.: A note on the asymptotic behavior of the Bernstein estimator of the copula density. J. Multivar. Anal. 124, 480–487 (2014). doi:10.1016/j.jmva.2013.10.009

    Article  MathSciNet  MATH  Google Scholar 

  • Kendall, M.G.: A New Measure of Rank Correlation, vol. 30, pp. 81–93. Biometrika, Cambridge (1938)

  • Kendall, M.G., Babington Smith, B.: On the Method of Paired Comparisons, vol. 31, pp. 324–345. Biometrika, Cambridge (1940)

  • Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005). doi:10.1007/0-387-27605-X

  • Littell, R.C., Pendergast, J., Natarajan, R.: Modelling covariance structure in the analysis of repeated measures data. Stat. Med. 19(13), 1793–1819 (2000)

    Article  Google Scholar 

  • Longin, F.M.: From value at risk to stress testing: the extreme value approach. J. Bank. Financ. 24, 1097–1130 (2000)

    Article  Google Scholar 

  • McNeil, A.J., Nešlehová, J.: Multivariate Archimedean copulas, \(d\)-monotone functions and \(\ell _{1}\)-norm symmetric distributions. Ann. Stat. 37, 3059–3097 (2009). doi: 10.1214/07-AOS556

    Article  MATH  Google Scholar 

  • McNeil, A.J., Frey, R., Embrechts, P.: Quantitative Risk Management: Concepts, Techniques and Tools. Princeton University Press, Princeton (2005)

    Google Scholar 

  • Meinshausen, N., Maathuis, M.H., Bühlmann, P.: Asymptotic optimality of the Westfall–Young permutation procedure for multiple testing under dependence. Ann. Stat. 39(6), 3369–3391 (2011). doi:10.1214/11-AOS946

    Article  MATH  Google Scholar 

  • Nelsen, R.B.: An introduction to copulas. In: Springer Series in Statistics, 2nd edn. Springer, New York (2006)

  • Nikoloulopoulos, A.K., Joe, H., Li, H.: Extreme value properties of multivariate \(t\) copulas. Extremes 12(2), 129–148 (2009). doi: 10.1007/s10687-008-0072-4

    Article  MathSciNet  MATH  Google Scholar 

  • Roeder, K., Wasserman, L.: Genome-wide significance levels and weighted hypothesis testing. Stat. Sci. 24(4), 398–413 (2009)

    Article  MathSciNet  Google Scholar 

  • Romano, J.P., Wolf, M.: Exact and approximate stepdown methods for multiple hypothesis testing. J. Am. Stat. Assoc. 100(469), 94–108 (2005). doi:10.1198/016214504000000539

    Article  MathSciNet  MATH  Google Scholar 

  • Sancetta, A., Satchell, S.: The Bernstein copula and its applications to modeling and approximations of multivariate distributions. Econ. Theory 20(3), 535–562 (2004). doi:10.1017/S026646660420305X

    Article  MathSciNet  MATH  Google Scholar 

  • Sarkar, S.: Rejoinder: on methods controlling the false discovery rate. Sankhyā Indian J. Stat. Ser. A 70(2), 183–185 (2008)

  • Šidák, Z.: Rectangular confidence regions for the means of multivariate normal distributions. J. Am. Stat. Assoc. 62, 626–633 (1967). doi:10.2307/2283989

    MATH  Google Scholar 

  • Singer, J.D., Willett, J.B.: Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford University Press, Oxford (2003)

  • Sklar, A.: Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Stat. Univ. Paris 8, 229–231 (1959)

    MathSciNet  Google Scholar 

  • Sklar, A.: Random variables, distribution functions, and copulas—a personal look backward and forward. In: Distributions with Fixed Marginals and Related Topics, pp. 1–14. Institute of Mathematical Statistics, Hayward (1996)

  • Stange, J., Bodnar, T., Dickhaus, T.: Uncertainty quantification for the family-wise error rate in multivariate copula models. WIAS Preprint 1862, Weierstrass Institute for Applied Analysis and Stochastics, Berlin (2013). http://www.wias-berlin.de/preprint/1862/wias_preprints_1862.pdf

  • Swanepoel, J.W.: A note on proving that the (modified) bootstrap works. Commun. Stat. Theory Methods 15, 3193–3203 (1986). doi:10.1080/03610928608829303

    Article  MathSciNet  MATH  Google Scholar 

  • Westfall, P.H., Young, S.S.: Resampling-based multiple testing: examples and methods for \(p\)-value adjustment. In: Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics. Wiley, New York (1993)

Download references

Author information

Authors and Affiliations

  1. Weierstrass Institute for Applied Analysis and Stochastics, Mohrenstr. 39, 10117, Berlin, Germany

    Jens Stange & Thorsten Dickhaus

  2. Department of Mathematics, Humboldt-University, Unter den Linden 6, 10099, Berlin, Germany

    Taras Bodnar

Authors
  1. Jens Stange
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Taras Bodnar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thorsten Dickhaus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thorsten Dickhaus.

Additional information

We thank the Editor Prof. Göran Kauermann and two anonymous referees for their detailed and constructive comments which considerably improved the presentation. This research was partly supported by the Deutsche Forschungsgemeinschaft via the Research Unit FOR 1735 “Structural Inference in Statistics: Adaptation and Efficiency” (Taras Bodnar) and via Grant No. DI 1723/3-1 (Jens Stange).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stange, J., Bodnar, T. & Dickhaus, T. Uncertainty quantification for the family-wise error rate in multivariate copula models. AStA Adv Stat Anal 99, 281–310 (2015). https://doi.org/10.1007/s10182-014-0241-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10182-014-0241-5

Keywords

Search

Navigation