Skip to main content

Advertisement

SpringerLink
Log in

Heteroscedasticity: multiple degrees of freedom vs. sandwich estimation

  • Regular Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

This article addresses the problem of multiple contrast tests in the presence of heteroscedasticity. A procedure applying multiple degrees of freedom and a procedure based on sandwich estimation are described and compared concerning the familywise error type I. The former procedure seems to be most robust in this regard, especially for situations where high variances meet small sample sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Adler ID, Kliesch U (1990) Comparison of single and multiple treatment regiments in the mouse bone marrow micronucleus assay for hydroquinone and cyclophosphamide. Mutat Res 234:115–123

    Article  Google Scholar 

  • Bretz F (2006) An extension of the Williams trend test to general unbalanced linear models. Comput Stat Data Anal 50(7):1735–1748

    Article  MathSciNet  MATH  Google Scholar 

  • Bretz F, Genz A, Hothorn LA (2001) On the numerical availability of multiple comparison procedures. Biom J 43:645–656

    Article  MathSciNet  MATH  Google Scholar 

  • Brown MB, Forsythe AB (1974) Anova and multiple comparisons for data with heterogeneous variances. Biometrics 30(4):719–724

    Article  MathSciNet  MATH  Google Scholar 

  • Demirhan H, Dolgun NA, Demirhan YP, Dolgun MO (2010) Performance of some multiple comparison tests under heteroscedasticity and dependency. J Stat Comput Simul 80(10):1083–1100

    Article  MathSciNet  MATH  Google Scholar 

  • Djira GD, Hasler M, Gerhard D, Schaarschmidt F (2012) mratios: Inferences for ratios of coefficients in the general linear model. http://CRAN.R-project.org/package=mratios, R package version 1.3.17. Accessed 2 Nov 2012

  • Dunnett CW (1955) A multiple comparison procedure for comparing several treatments with a control. J Am Stat Assoc 50(272):1096–1121

    Article  MATH  Google Scholar 

  • Dunnett CW (1980) Pairwise multiple comparisons in the unequal variance case. J Am Stat Assoc 75(372):796–800

    Article  Google Scholar 

  • Freedman DA (2006) On the so-called “Huber Sandwich Estimator” and “Robust Standard Errors”. Am Stat 60(4):299–302

    Article  Google Scholar 

  • Games PA, Howell JF (1976) Pairwise multiple comparison procedures with unequal n’s and/or variances: a monte carlo study. J Educ Stat 1(2):113–125

    Article  Google Scholar 

  • Genz A, Bretz F (2002) Methods for the computation of multivariate \(t\)-probabilities. J Comput Graph Stat 11:950–971

    Article  MathSciNet  Google Scholar 

  • Genz A, Bretz F (2009) Computation of multivariate normal and \(t\) probabilities, vol 195., Lecture notes in statisticsSpringer, Heidelberg

    Book  Google Scholar 

  • Genz A, Bretz F, Miwa T, Mi X, Leisch F, Scheipl F, Hothorn T (2014) mvtnorm: Multivariate normal and \(t\) distributions. http://CRAN.R-project.org/package=mvtnorm, R package version 1.0-0. Accessed 8 July 2014

  • Guilbaud O (2011) Note on simultaneous inferences about non-inferiority and superiority for a primary and a secondary endpoint. Biom J 53(6, SI):927–937

    Article  MathSciNet  MATH  Google Scholar 

  • Hartung J, Argac D, Makambi KH (2002) Small sample properties of tests on homogeneity in one-way anova and meta-analysis. Stat Pap 43(2):197–235

    Article  MathSciNet  MATH  Google Scholar 

  • Hasler M (2012) Multiple comparisons to both a negative and a positive control. Pharm Stat 11(1):74–81

    Article  MathSciNet  Google Scholar 

  • Hasler M (2014) SimComp: simultaneous comparisons for multiple endpoints. http://CRAN.R-project.org/package=SimComp, R package version 2.2. Accessed 12 Sept 2014

  • Hasler M, Hothorn LA (2008) Multiple contrast tests in the presence of heteroscedasticity. Biom J 50(5):793–800

    Article  MathSciNet  Google Scholar 

  • Hasler M, Hothorn LA (2012) A multivariate Williams-type trend procedure. Stat Biopharm Res 4:57–65

    Article  Google Scholar 

  • Hasler M, Vonk R, Hothorn LA (2008) Assessing non-inferiority of a new treatment in a three-arm trial in the presence of heteroscedasticity. Stat Med 27(4):490–503

    Article  MathSciNet  Google Scholar 

  • Hayter AJ (1984) A proof of the conjecture that the Tukey–Kramer multiple comparisons procedure is conservative. Ann Stat 12(1):61–75

    Article  MathSciNet  MATH  Google Scholar 

  • Herberich E, Sikorski J, Hothorn T (2010) A robust procedure for comparing multiple means under heteroscedasticity in unbalanced designs. PLoS One 5(3):e9788

    Article  Google Scholar 

  • Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70

    MathSciNet  MATH  Google Scholar 

  • Homma Y, Yamaguchi T, Yamaguchi O (2008) A randomized, double-blind, placebo-controlled phase ii dose-finding study of the novel anti-muscarinic agent imidafenacin in Japanese patients with overactive bladder. Int J Urol 15(9):809–815

    Article  Google Scholar 

  • Hommel G (1988) A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75(2):383–386

    Article  MATH  Google Scholar 

  • Hothorn T, Bretz F, Genz A (2001) On multivariate \(t\) and Gauss probabilities in R. R News 1(2):27–29

    Google Scholar 

  • Hothorn T, Bretz F, Westfall PH, Heiberger RM, Schuetzenmeister A (2014) multcomp: Simultaneous inference in general parametric models. http://CRAN.R-project.org/package=multcomp, R package version 1.3-7. Accessed 2 Oct 2014

  • Huber PJ (1967) The behavior of maximum likelihood estimates under nonstandard conditions. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol 1. pp 221–233

  • Kromrey JD, LaRocca MA (2004) Power and type i error rates of new pairwise multiple comparison procedures under heterogeneous variances. J Exp Educ 63(4):343–362

    Article  Google Scholar 

  • Li H, Ning W (2012) Multiple comparisons with a control under heteroscedasticity. J Appl Stat 39(10):2275–2283

    Article  MathSciNet  Google Scholar 

  • Lumley T, Zeileis A (2014) Sandwich: robust covariance matrix estimators. http://CRAN.R-project.org/package=sandwich, R package version 2.3-2. Accessed 24 Aug 2014

  • MacKinnon JG, White H (1985) Some heteroskedasticity-consistent covariance-matrix estimators with improved finite-sample properties. J Econom 29(3):305–325

    Article  Google Scholar 

  • Mehrotra DV (1997) Improving the Brown–Forsythe solution to the generalized Behrens–Fisher problem. Commun Stat Simul Comput 26(3):1139–1145

    Article  MathSciNet  MATH  Google Scholar 

  • R Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/

  • Ramsey PH, Ramsey PP (2009) Power and type I errors for pairwise comparisons of means in the unequal variances case. Br J Math Stat Psychol 62:263–281

    Article  MathSciNet  Google Scholar 

  • Satterthwaite FE (1946) An approximate distribution of estimates of variance components. Biometrics 2:110–114

    Article  Google Scholar 

  • Strassburger K, Bretz F (2008) Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni-based closed tests. Stat Med 27(24):4914–4927

    Article  MathSciNet  Google Scholar 

  • Sun WG, McLain AC (2012) Multiple testing of composite null hypotheses in heteroscedastic models. J Am Stat Assoc 107(498):673–687

    Article  MathSciNet  MATH  Google Scholar 

  • Tamhane AC, Logan BR (2004) Finding the maximum safe dose level for heteroscedastic data. J Biopharm Stat 14(4):843–856

    Article  MathSciNet  Google Scholar 

  • Tukey JW (1953) The problem of multiple comparisons, dittoed manuscript of 396 pages. Department of Statistics, Princeton University, New Jersey

  • Welch BL (1938) The significance of the difference between two means when the population variances are unequal. Biometrika 29:350–362

    Article  MATH  Google Scholar 

  • Welch BL (1951) On the comparison of several mean values—an alternative approach. Biometrika 38(3–4):330–336

    Article  MathSciNet  MATH  Google Scholar 

  • Westfall PH, Young SS (1993) Resampling-based multiple testing, Examples and methods for p-value adjustment. Wiley-Interscience, Berlin

    Google Scholar 

  • Williams DA (1971) A test for differences between treatment means when several dose levels are compared with a zero dose control. Biometrics 27:103–117

    Article  Google Scholar 

  • Zeileis A (2004) Econometric computing with HC and HAC covariance matrix estimators. J Stat Softw 11(10):1–17

  • Zeileis A (2006) Object-oriented computation of sandwich estimators. J Stat Softw 16(9):1–16

Download references

Author information

Authors and Affiliations

  1. Lehrfach Variationsstatistik, Christian-Albrechts-Universität zu Kiel, Hermann-Rodewald-Straße 9, 24118, Kiel, Germany

    Mario Hasler

Authors
  1. Mario Hasler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mario Hasler.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hasler, M. Heteroscedasticity: multiple degrees of freedom vs. sandwich estimation. Stat Papers 57, 55–68 (2016). https://doi.org/10.1007/s00362-014-0640-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-014-0640-4

Keywords

Search

Navigation