Learning and Evaluation in the Presence of Class Hierarchies: Application to Text Categorization

  • Svetlana Kiritchenko
  • Stan Matwin
  • Richard Nock
  • A. Fazel Famili
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4013)


This paper deals with categorization tasks where categories are partially ordered to form a hierarchy. First, it introduces the notion of consistent classification which takes into account the semantics of a class hierarchy. Then, it presents a novel global hierarchical approach that produces consistent classification. This algorithm with AdaBoost as the underlying learning procedure significantly outperforms the corresponding “flat” approach, i.e. the approach that does not take into account the hierarchical information. In addition, the proposed algorithm surpasses the hierarchical local top-down approach on many synthetic and real tasks. For evaluation purposes, we use a novel hierarchical evaluation measure that has some attractive properties: it is simple, requires no parameter tuning, gives credit to partially correct classification and discriminates errors by both distance and depth in a class hierarchy.


Directed Acyclic Graph Text Categorization Global Approach Hierarchical Categorization Class Hierarchy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Koller, D., Sahami, M.: Hierarchically Classifying Documents Using Very Few Words. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 170–178 (1997)Google Scholar
  2. 2.
    Dumais, S., Chen, H.: Hierarchical Classification of Web Content. In: Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR), pp. 256–263 (2000)Google Scholar
  3. 3.
    Sun, A., Lim, E.P.: Hierarchical Text Classification and Evaluation. In: Proceedings of the IEEE International Conference on Data Mining (ICDM), pp. 521–528 (2001)Google Scholar
  4. 4.
    Ruiz, M., Srinivasan, P.: Hierarchical Text Categorization Using Neural Networks. Information Retrieval 5, 87–118 (2002)MATHCrossRefGoogle Scholar
  5. 5.
    Wang, K., Zhou, S., He, Y.: Hierarchical Classification of Real Life Documents. In: Proceedings of the SIAM International Conference on Data Mining (2001)Google Scholar
  6. 6.
    Blockeel, H., Bruynooghe, M., Dzeroski, S., Ramon, J., Struyf, J.: Hierarchical Multi-Classification. In: Proceedings of the SIGKDD Workshop on Multi-Relational Data Mining (MRDM), pp. 21–35 (2002)Google Scholar
  7. 7.
    Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Machine Learning for Interdependent and Structured Output Spaces. In: Proceedings of the International Conference on Machine Learning (ICML) (2004)Google Scholar
  8. 8.
    McCallum, A., Rosenfeld, R., Mitchell, T., Ng, A.: Improving Text Classification by Shrinkage in a Hierarchy of Classes. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 359–367 (1998)Google Scholar
  9. 9.
    Schapire, R., Singer, Y.: Improved Boosting Algorithms Using Confidence-rated Predictions. Machine Learning 37, 297–336 (1999)MATHCrossRefGoogle Scholar
  10. 10.
    Huang, J., Ling, C.: Using AUC and Accuracy in Evaluating Learning Algorithms. IEEE Trans. on Data and Knowledge Engineering 17(3), 299–310 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Svetlana Kiritchenko
    • 1
    • 4
  • Stan Matwin
    • 1
    • 2
  • Richard Nock
    • 3
  • A. Fazel Famili
    • 4
  1. 1.University of OttawaCanada
  2. 2.Institute of Computer SciencePolish Academy of SciencesWarsawPoland
  3. 3.Université Antilles-GuyaneMartiniqueFrance
  4. 4.Institute for Information TechnologyNational Research CouncilCanada

Personalised recommendations