A simulation-intensive approach for checking hierarchical models

Abstract

Recent computational advances have made it feasible to fit hierarchical models in a wide range of serious applications. In the process, the question of model adequacy arises. While model checking usually addresses the entire model specification, model failures can occur at each hierarchical stage. Such failures include outliers, mean structures errors, dispersion misspecification, and inappropriate exchangeabilities. We propose an approach which is entirely simulation based. Given a model specification and a dataset, we need only be able to simulate draws from the resultant posterior. By replicating a posterior of interest using data obtained under the model we can “see” the extent of variability in such a posterior. Then, we can compare the posterior obtained under the observed data with this medley of posterior replicates to ascertain whether the former is in agreement with them and accordingly, whether it is plausible that the observed data came from the proposed model. Many such comparisons can be run, each focusing on a different potential model failure. Focusing on generalized linear mixed models, we explore the questions of when hierarchical model stages are separable and checkable and illustrate the approach with both real and simulated data.