Mining the Hidden Structure of Inductive Learning Data Sets

Abstract

This paper proposes a method for extracting the hidden characteristics of machine learning domains. It does so by evaluating the performance of various classifiers on these domains as well as on artificial data whose characteristics are visible since they were purposely included in the generation process. The results obtained on both the real and artificial data are analyzed simultaneously using a classical visualization tool for hierarchical clustering called a dendogram. The idea is to map the real-world domains to the artificial ones according to how well they are learnt by a variety of classifiers and, through this relationship, extract their characteristics. The method is able to determine how difficult it is to classify a specific domain and whether this difficulty stems from the complexity of the concept it embodies, the amount of overlap between each class, the dearth of training data or its dimensionality. This is an important contribution as it allows researchers to understand the underlying nature of their data, and, thus converge quickly toward novel, well-adapted solutions to their particular problems.