Global EM Learning of Finite Mixture Models using the Greedy Elimination Method

Abstract

The standard learning method for finite mixture models is the Expectation-Maximization (EM) algorithm, which performs hill-climbing from an initial solution to obtain the local maximum likelihood solution. However, given that the solution space is large and multimodal, EM is prone to produce inconsistent and sub-optimal solutions over multiple runs.

This paper presents a novel global greedy learning method called the Greedy Elimination Method (GEM) to alleviate these problems. GEM is simple to implement in any finite mixture model, yet effective to enhance the global optimality and the consistency of the solutions. It is also very efficient as its complexity grows only linearly with the number of data patterns. GEM is demonstrated on clustering synthetic datasets using the mixture of Gaussian model, and on clustering the shrinking spiral data set using the mixture of Factor Analyzers.