Investigating Classifier Learning Behavior with Experiment Databases
Experimental assessment of the performance of classification algorithms is an important aspect of their development and application on real-world problems. To facilitate this analysis, large numbers of such experiments can be stored in an organized manner and in complete detail in an experiment database. Such databases serve as a detailed log of previously performed experiments and a repository of verifiable learning experiments that can be reused by different researchers. We present an existing database containing 250,000 runs of classifier learning systems, and show how it can be queried and mined to answer a wide range of questions on learning behavior. We believe such databases may become a valuable resource for classification researchers and practitioners alike.
KeywordsLearn Behavior Experiment Database Classifier Learning System Dataset Property Practical Machine Learn Tool
Unable to display preview. Download preview PDF.
- BLOCKEEL, H. and Vanschoren J. (2007): Experiment Databases: Towards an Improved Ex-perimental Methodology in Machine Learning. Lecture Notes in Computer Science, 4702, to appear.Google Scholar
- KALOUSIS, A. and HILARIO, M. (2000): Building Algorithm Profiles for prior Model Se-lection in Knowledge Discovery Systems. Engineering Intelligent Syst., 8(2).Google Scholar
- VAN SOMEREN, M. (2001): Model Class Selection and Construction: Beyond the Pro-crustean Approach to Machine Learning Applications. Lecture Notes in Computer Sci-ence, 2049, 196-217.Google Scholar
- WITTEN, I.H. and FRANK, E. (2005): Data Mining: Practical Machine Learning Tools and Techniques (2nd edition). Morgan Kaufmann.Google Scholar