Data Mining and Knowledge Discovery

, Volume 29, Issue 3, pp 820–865

Evaluation measures for hierarchical classification: a unified view and novel approaches

  • Aris Kosmopoulos
  • Ioannis Partalas
  • Eric Gaussier
  • Georgios Paliouras
  • Ion Androutsopoulos
Article

DOI: 10.1007/s10618-014-0382-x

Cite this article as:
Kosmopoulos, A., Partalas, I., Gaussier, E. et al. Data Min Knowl Disc (2015) 29: 820. doi:10.1007/s10618-014-0382-x

Abstract

Hierarchical classification addresses the problem of classifying items into a hierarchy of classes. An important issue in hierarchical classification is the evaluation of different classification algorithms, an issue which is complicated by the hierarchical relations among the classes. Several evaluation measures have been proposed for hierarchical classification using the hierarchy in different ways without however providing a unified view of the problem. This paper studies the problem of evaluation in hierarchical classification by analysing and abstracting the key components of the existing performance measures. It also proposes two alternative generic views of hierarchical evaluation and introduces two corresponding novel measures. The proposed measures, along with the state-of-the-art ones, are empirically tested on three large datasets from the domain of text classification. The empirical results illustrate the undesirable behaviour of existing approaches and how the proposed methods overcome most of these problems across a range of cases.

Keywords

Evaluation Evaluation measures Hierarchical classification Tree-structured class hierarchies DAG-structured class hierarchies 

Copyright information

© The Author(s) 2014

Authors and Affiliations

  • Aris Kosmopoulos
    • 1
    • 2
  • Ioannis Partalas
    • 3
  • Eric Gaussier
    • 3
  • Georgios Paliouras
    • 1
  • Ion Androutsopoulos
    • 2
  1. 1.National Center for Scientific Research “Demokritos”AthensGreece
  2. 2.Athens University of Economics and BusinessAthensGreece
  3. 3.Laboratoire d’Informatique de GrenobleUnivesité Joseph FourierGrenobleFrance