Investigating Classifier Learning Behavior with Experiment Databases

  • Joaquin Vanschoren
  • Hendrik Blockeel
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Experimental assessment of the performance of classification algorithms is an important aspect of their development and application on real-world problems. To facilitate this analysis, large numbers of such experiments can be stored in an organized manner and in complete detail in an experiment database. Such databases serve as a detailed log of previously performed experiments and a repository of verifiable learning experiments that can be reused by different researchers. We present an existing database containing 250,000 runs of classifier learning systems, and show how it can be queried and mined to answer a wide range of questions on learning behavior. We believe such databases may become a valuable resource for classification researchers and practitioners alike.


Learn Behavior Experiment Database Classifier Learning System Dataset Property Practical Machine Learn Tool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. BLOCKEEL, H. (2006): Experiment databases: A novel methodology for experimental re-search. Lecture Notes in Computer Science, 3933, 72-85.CrossRefGoogle Scholar
  2. BLOCKEEL, H. and Vanschoren J. (2007): Experiment Databases: Towards an Improved Ex-perimental Methodology in Machine Learning. Lecture Notes in Computer Science, 4702, to appear.Google Scholar
  3. KALOUSIS, A. and HILARIO, M. (2000): Building Algorithm Profiles for prior Model Se-lection in Knowledge Discovery Systems. Engineering Intelligent Syst., 8(2).Google Scholar
  4. PENG, Y. et al. (2002): Improved Dataset Characterisation for Meta-Learning. Lecture Notes in Computer Science, 2534, 141-152.CrossRefGoogle Scholar
  5. VAN SOMEREN, M. (2001): Model Class Selection and Construction: Beyond the Pro-crustean Approach to Machine Learning Applications. Lecture Notes in Computer Sci-ence, 2049, 196-217.Google Scholar
  6. WITTEN, I.H. and FRANK, E. (2005): Data Mining: Practical Machine Learning Tools and Techniques (2nd edition). Morgan Kaufmann.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Joaquin Vanschoren
    • 1
  • Hendrik Blockeel
    • 1
  1. 1.Computer Science DeptK.U.LeuvenLeuvenBelgium

Personalised recommendations