Skip to main content
SpringerLink
Log in
Book cover

Advances in Data Analysis, Data Handling and Business Intelligence pp 83–92Cite as

Evaluation Strategies for Learning Algorithms of Hierarchies

  • Conference paper
  • First Online:

Abstract

Several learning tasks comprise hierarchies. Comparison with a “gold-standard” is often performed to evaluate the quality of a learned hierarchy. We assembled various similarity metrics that have been proposed in different disciplines and compared them in a unified interdisciplinary framework for hierarchical evaluation which is based on the distinction of three fundamental dimensions. Identifying deficiencies for measuring structural similarity, we suggest three new measures for this purpose, either extending existing ones or based on new ideas. Experiments with an artificial dataset were performed to compare the different measures. As shown by our results, the measures vary greatly in their properties.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Bade, K., & Hermkes, M. (2008). Collection browsing through automatic hierarchical tagging. In Proceedings of the 5th International Conference on Adaptive Hypermedia and Adaptive Web-based Systems.

    Google Scholar 

  • Bade, K., Hüllermeier, E., & Nürnberger, A. (2006). Hierarchical classification by expected utility maximization. In Proceedings of the 2006 IEEE International Conference on Data Mining.

    Google Scholar 

  • Bade, K., & Nürnberger, A. (2008). Creating a cluster hierarchy under constraints of a partially known hierarchy. In Proceedings of the 2008 SIAM International Conference on Data Mining (pp. 13–24).

    Google Scholar 

  • Bille, P. (2005). A survey on tree edit distance and related problems. Theoretical Computer Science, 337(1–3), 217–239.

    Article  MATH  MathSciNet  Google Scholar 

  • Brank, J., Madenic, D., & Groblenik, M. (2006, May). Gold standard based ontology evaluation using instance assignment. In Proceedings of 4th EON Workshop.

    Google Scholar 

  • Cai, L., & Hofmann, T. (2004). Hierarchical document categorization with support vector machines. In Proceedings of the 13th ACM CIKM (pp. 78–87).

    Google Scholar 

  • Ceci, M., & Malerba, D. (2003). Hierarchical classification of html documents with webclassii. In Proceedings of the 25th European Conference on IR (pp. 57–72).

    Google Scholar 

  • Cimiano, P. (2006). Ontology learning and population from text: Algorithms, evaluation and applications. Berlin: Springer.

    Google Scholar 

  • Dellschaft, K., & Staab, S. (2006). On how to perform a gold standard based evaluation of ontology learning. In Proceedings of ISWC-2006 International Semantic Web Conference.

    Google Scholar 

  • Hubert, L., & Arabie, P. (1985, December). Comparing partitions. Journal of Classification, 2(1), 193–218.

    Google Scholar 

  • Maedche, A. (2002). Ontology learning for the semantic web. Boston: Kluwer.

    MATH  Google Scholar 

  • Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66, 622–626.

    Article  Google Scholar 

  • Schenker, A., Bunke, H., Last, M., & Kandel, A. (2005). Graph-theoretic techniques for web content mining. Singapore: World Scientific.

    Book  MATH  Google Scholar 

  • Sun, A., & Lim, E. (2001). Hierarchical text classification and evaluation. In Proceedings of the 2001 IEEE International Conference on Data Mining (pp. 521–528).

    Google Scholar 

  • Treeratpituk, P., & Callan, J. (2006). Automatically labeling hierarchical clusters. In Proceedings of the International Conference on Digital Government Research (pp. 167–176).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Otto-von-Guericke-University Magdeburg, 39106, Magdeburg, Germany

    Korinna Bade (Faculty of Computer Science)

Authors
  1. Korinna Bade
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominik Benz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Korinna Bade .

Editor information

Editors and Affiliations

  1. Universität der Bundeswehr, Fak. Wirtschafts-/Sozialwissenschaften, Helmut-Schmidt-Universität, Holstenhofweg 85, Hamburg, 22043, Germany

    Andreas Fink

  2. Dept. Mathematical Sciences, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, United Kingdom

    Berthold Lausen

  3. Universität der Bundeswehr, Fak. Wirtschafts-/Sozialwissenschaften, Helmut-Schmidt-Universität, Holstenhofweg 85, Hamburg, 22043, Germany

    Wilfried Seidel

  4. FB 12 Mathematik und Informatik, Datenbionik AG, Universität Marburg, Hans-Meerwein-Straße, Marburg, 35032, Germany

    Alfred Ultsch

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bade, K., Benz, D. (2009). Evaluation Strategies for Learning Algorithms of Hierarchies. In: Fink, A., Lausen, B., Seidel, W., Ultsch, A. (eds) Advances in Data Analysis, Data Handling and Business Intelligence. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01044-6_7

Download citation

Publish with us

Policies and ethics

Search

Navigation