Article

Information Retrieval

, Volume 12, Issue 4, pp 461-486

First online:

A comparison of extrinsic clustering evaluation metrics based on formal constraints

  • Enrique AmigóAffiliated withDepartamento de Lenguajes y Sistemas Informáticos, UNED Email author 
  • , Julio GonzaloAffiliated withDepartamento de Lenguajes y Sistemas Informáticos, UNED
  • , Javier ArtilesAffiliated withDepartamento de Lenguajes y Sistemas Informáticos, UNED
  • , Felisa VerdejoAffiliated withDepartamento de Lenguajes y Sistemas Informáticos, UNED

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access

Abstract

There is a wide set of evaluation metrics available to compare the quality of text clustering algorithms. In this article, we define a few intuitive formal constraints on such metrics which shed light on which aspects of the quality of a clustering are captured by different metric families. These formal constraints are validated in an experiment involving human assessments, and compared with other constraints proposed in the literature. Our analysis of a wide range of metrics shows that only BCubed satisfies all formal constraints. We also extend the analysis to the problem of overlapping clustering, where items can simultaneously belong to more than one cluster. As Bcubed cannot be directly applied to this task, we propose a modified version of Bcubed that avoids the problems found with other metrics.

Keywords

Clustering Evaluation metrics Formal constraints