Mixture Models

Abstract

Suppose that ℱ = {F _θ : θ ( S} is a parametric family of distributions on a sample space X, and let Q denote a probability distribution defined on the parameter space S. The distribution

$${{F}_{Q}}=\int {{{F}_{\theta }}dQ\left( \theta \right)} $$

(1)

is a mixture distribution. An observation X drawn from F q can be thought of as being obtained in a two-step procedure: first, a random Θ is drawn from the distribution Q and then, conditional on Θ = θ, X is drawn from the distribution F _θ . Suppose we have a random sample X ₁ ,…, X _n from F q . We can view this as a missing data problem in that the ‘full data’ consists of pairs (X ₁,Θ₁),…, (X _n ,Θ _n ), with Θi ∼ Q and X_i|Θ_i = θ ∼ F _θ , but then only the first member X_i of each pair is observed; the labels Θ_i are hidden.