Mining the Hidden Structure of Inductive Learning Data Sets
This paper proposes a method for extracting the hidden characteristics of machine learning domains. It does so by evaluating the performance of various classifiers on these domains as well as on artificial data whose characteristics are visible since they were purposely included in the generation process. The results obtained on both the real and artificial data are analyzed simultaneously using a classical visualization tool for hierarchical clustering called a dendogram. The idea is to map the real-world domains to the artificial ones according to how well they are learnt by a variety of classifiers and, through this relationship, extract their characteristics. The method is able to determine how difficult it is to classify a specific domain and whether this difficulty stems from the complexity of the concept it embodies, the amount of overlap between each class, the dearth of training data or its dimensionality. This is an important contribution as it allows researchers to understand the underlying nature of their data, and, thus converge quickly toward novel, well-adapted solutions to their particular problems.
KeywordsClassification Meta-Learning dataset characterization artificial data generation
Unable to display preview. Download preview PDF.
- [Aha, D. 1992]Generalizing from case studies: A case study. In: ICML 1992, pp. 1–10 (1992)Google Scholar
- [Alaiz et al., 2008]Alaiz-Rodriguez, R., Japkowicz, N., Tischer, P.: Visualizing Classifier Performance. In: ICTAI 2008 (2008)Google Scholar
- [Batista et al., 2005]Batista, G.E.A.P.A., Prati, R.C., Monard, M.C.: Balancing Strategies and Class Overlapping. Intelligent Data Analysis, 24–35 (2005)Google Scholar
- [R. Caruana and A. Niculescu-Mizil, 2004]Data mining in metric space: An empirical analysis of supervised learning performance criteria. In: KDD 2004 (2004)Google Scholar
- [T. Cox and M. Cox, 1994]Multidimensional Scaling. Chapman and Hall (1994)Google Scholar
- [Frank, A. & Asuncion, A. (2010)]UCI Machine Learning Repository Google Scholar
- [Japkowicz and Stephen, 2002]
- [Smith-Miles, K. A.,]Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Computing Surveys 41(1), article 6 (2008)Google Scholar
- [I. H. Witten, E. Frank and M.A. Hall, 2011]Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufman (January 2011)Google Scholar