Survey data are often obtained through a multilevel structure and, as such, require hierarchical modeling. While large sample approximation provides a mechanism to construct confidence intervals for the intraclass correlation coefficients (ICCs) in large datasets, challenges arise when we are faced with small-size clusters and binary outcomes. In this paper, we examine two bootstrapping methods, cluster bootstrapping and split bootstrapping. We use these methods to construct the confidence intervals for the ICCs (based on a latent variable approach) for small binary data obtained through a three-level or higher hierarchical data structure. We use 26 scenarios in our simulation study with the two bootstrapping methods. We find that the latent variable method performs well in terms of coverage. The split bootstrapping method provides confidence intervals close to the nominal coverage when the ratio of the ICC for the primary cluster to the ICC for the secondary cluster is small. While the cluster bootstrapping is preferred when the cluster size is larger and the ratio of the ICCs is larger. A numerical example based on teacher effectiveness is assessed.
Generalized linear mixed model Small sample inference Resampling scheme
This is a preview of subscription content, log in to check access.
Bland JM, Altman DG (1990) A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med 20(5):337–340CrossRefGoogle Scholar
Braschel MC, Svec I, Darlington GA, Donner A (2016) A comparison of confidence interval methods for the intraclass correlation coefficient in community-based cluster randomization trials with a binary outcome. Clin Trials 13(2):180–187CrossRefGoogle Scholar
Breslow NE, Clayton DG (1993) Approximate inference in generalized linear mixed models. J Am Stat Assoc 88(421):9–25zbMATHGoogle Scholar
Capanu M, Gönen M, Begg CB (2013) An assessment of estimation methods for generalized linear mixed models with binary outcomes. Stat Med 32(26):4550–4566MathSciNetCrossRefGoogle Scholar
Cramér H (1999) Mathematical methods of statistics. Princeton University Press, PrincetonzbMATHGoogle Scholar
Davison AC, Hinkley DV (1997) Bootstrap methods and their application. Cambridge University Press, CambridgeCrossRefGoogle Scholar
Donner A (1986) A review of inference methods for the intraclass correlation coefficient in the one-way random effects model. Int Stat Rev 54(1):67CrossRefGoogle Scholar
Efron B, Tibshirani R (1986) Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci 1(1):54–75MathSciNetCrossRefGoogle Scholar
Snijders TA, Bosker RJ (2012) Multilevel analysis: an introduction to basic and advanced multilevel modeling. SAGE, Los AngeleszbMATHGoogle Scholar
Tan M, Qu Y, Mascha E, Schubert A (1999) A Bayesian hierarchical model for multi-level repeated ordinal data: analysis of oral practice examinations in a large anesthesiology training programme. Stat Med 18(15):1983–1992CrossRefGoogle Scholar
Ukoumunne OC, Davison AC, Gulliford MC, Chinn S (2003) Non-parametric bootstrap confidence intervals for the intraclass correlation coefficient. Stat Med 22(24):3805–3821CrossRefGoogle Scholar
Wu S, Crespi CM, Wong WK (2012) Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemp Clin Trials 33(5):869–880CrossRefGoogle Scholar
Zou G, Donner A (2004) confidence interval estimation of the intraclass correlation coefficient for binary outcome data. Biometrics 60(3):807–811MathSciNetCrossRefGoogle Scholar