The Data Augmentation Algorithm

Abstract

Analogous to the EM algorithm, the data augmentation algorithm exploits the simplicity of the likelihood function or posterior distribution of the parameter given the augmented data. In contrast to the EM algorithm, the present goal is to obtain the entire (normalized) likelihood or posterior distribution, not just the maximizer and the curvature at the maximizer. In large samples, it is comforting that the posterior or likelihood is consistent with the normal approximation, though in practice it is not often clear when one is in a large sample setting. In a small sample situation, the data augmentation algorithm will provide a way of improving the inference, based on the entire posterior distribution or the entire likelihood function.