Information Retrieval

, Volume 12, Issue 4, pp 461–486

A comparison of extrinsic clustering evaluation metrics based on formal constraints

  • Enrique Amigó
  • Julio Gonzalo
  • Javier Artiles
  • Felisa Verdejo
Article

DOI: 10.1007/s10791-008-9066-8

Cite this article as:
Amigó, E., Gonzalo, J., Artiles, J. et al. Inf Retrieval (2009) 12: 461. doi:10.1007/s10791-008-9066-8

Abstract

There is a wide set of evaluation metrics available to compare the quality of text clustering algorithms. In this article, we define a few intuitive formal constraints on such metrics which shed light on which aspects of the quality of a clustering are captured by different metric families. These formal constraints are validated in an experiment involving human assessments, and compared with other constraints proposed in the literature. Our analysis of a wide range of metrics shows that only BCubed satisfies all formal constraints. We also extend the analysis to the problem of overlapping clustering, where items can simultaneously belong to more than one cluster. As Bcubed cannot be directly applied to this task, we propose a modified version of Bcubed that avoids the problems found with other metrics.

Keywords

ClusteringEvaluation metricsFormal constraints

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Enrique Amigó
    • 1
  • Julio Gonzalo
    • 1
  • Javier Artiles
    • 1
  • Felisa Verdejo
    • 1
  1. 1.Departamento de Lenguajes y Sistemas InformáticosUNEDMadridSpain