Hierarchical Cluster Methods as Maximum Likelihood Estimators

Abstract

The algorithms of hierarchical cluster analysis are mainly heuristically motivated. This is especially true from the viewpoint of a mathematical statistician, who misses a precise probabilistic model. There are, though, statistical models in hierarchical cluster analysis which use such probabilistic statistical methods. In either approach one can consider hierarchical cluster analysis as a transformation from a dissimilarity matrix to an ultrametric matrix. If the data in a dissimilarity matrix are only disturbed values of the “true” ultrametric matrix, one can consider cluster analysis as estimating the true ultrametric matrix. With the maximum likelihood method one can for example find estimators — here cluster analysis methods — for each error distribution, i.e. for each way in which the data are disturbed. In this way we can develop the single-linkage, modified complete-linkage, median-linkage and average-linkage methods. Rigorous inspection of the models (error distributions) shows the limitations of these (perhaps too) simple models.