Machine Learning

, Volume 87, Issue 2, pp 127–158

Experiment databases

A new way to share, organize and learn from experiments

Authors

    • LIACSUniversiteit Leiden
    • Dept. of Computer ScienceKatholieke Universiteit Leuven
  • Hendrik Blockeel
    • LIACSUniversiteit Leiden
    • Dept. of Computer ScienceKatholieke Universiteit Leuven
  • Bernhard Pfahringer
    • Dept. of Computer ScienceThe University of Waikato
  • Geoffrey Holmes
    • Dept. of Computer ScienceThe University of Waikato
Article

DOI: 10.1007/s10994-011-5277-0

Cite this article as:
Vanschoren, J., Blockeel, H., Pfahringer, B. et al. Mach Learn (2012) 87: 127. doi:10.1007/s10994-011-5277-0

Abstract

Thousands of machine learning research papers contain extensive experimental comparisons. However, the details of those experiments are often lost after publication, making it impossible to reuse these experiments in further research, or reproduce them to verify the claims made. In this paper, we present a collaboration framework designed to easily share machine learning experiments with the community, and automatically organize them in public databases. This enables immediate reuse of experiments for subsequent, possibly much broader investigation and offers faster and more thorough analysis based on a large set of varied results. We describe how we designed such an experiment database, currently holding over 650,000 classification experiments, and demonstrate its use by answering a wide range of interesting research questions and by verifying a number of recent studies.

Keywords

Experimental methodologyMachine learningDatabasesMeta-learning
Download to read the full article text

Copyright information

© The Author(s) 2011