A New Approximation to the True Randomization-Based Design Effect

Abstract

It is generally difficult or even impossible to obtain a closed-form randomization-based true design effect formula for a nonlinear estimator under a complex sample design. A model that captures different salient features of the sample design is often used to approximate the randomization-based true design effect. Our simulation results show that the usual model-based design effect for the sample mean could significantly differ from the randomization-based true design effect for different replications of the finite population, even when different replicates of the finite population are generated by the same model used to derive the model-based design effect formula. We propose a new model-assisted design effect formula obtained from an appropriate model-based design effect formula when we replace the model intra-cluster correlation by an ANOVA “estimator” if observations for all units of the finite population were available. For one-stage cluster sampling with equal cluster size, we examine the accuracy of the new model-assisted design effect formula analytically and by a Monte carlo simulation. This new model-assisted design effect tracks the true randomization-based design effect much better than the corresponding model-based design effect formulae and the approximation to the true randomization-based design effect proposed by Kish (1965). The main advantage of the new model-assisted design effect is that it can be readily extended to more complex estimators and/or complex designs where the Kish’s approximation is unavailable. Our proposed model-assisted design effect is generally much closer to the randomization-based design effect formula than the corresponding model-based design effect, even when model used to obtain the model-based design effect holds for the finite population.