Standardizing the Comparison of Partitions
- 387 Downloads
We propose a standardized partition space that offers a unifying framework for the comparison of a wide variety of classification rules. Using standardized partition spaces, one can define measures for the performance of classifiers w.r.t. goodness concepts beyond the expected rate of correct classifications such that they are comparable for rules from so different techniques as support vector machines, neural networks, discriminant analysis, and many more. For classification problems with up to four classes, one can visualize partitions from classification rules that allow for a direct comparison of characteristic patterns of the rules. We use these visualizations to motivate measures for accuracy and non-resemblance in the sense of (Hand 1997), enhanced for non-probabilistic classifiers.
- Blake, C. & Merz, C. (1998), ‘UCI repository of machine learning databases’, http://www.ics.uci.edu/~mlearn/MLRepository.html
- Bridle, J. S. (1990), Probabilistic interpretation of feedforward classification network outputs, with relationships to, statistical pattern recognition, in F. Fogelman Soulié & J. Hérault, eds, ‘Neuro-computing: Algorithms, Architectures and Applications’, Springer, Berlin, pp. 227–236.CrossRefGoogle Scholar
- Demuth, H. & Beale, M. (1998), Neural Network Toolbox User’s Guide.Google Scholar
- German, B. (1987), ‘Glass identification database’. http://www.ics.uci.edu/~mlearn/MLRepository.html
- Ihaka, R. & Gentleman, R. (1996), ‘R: A language for data analysis and graphics’, Journal of Computational and Graphical Statistics 5(3), 299–314.Google Scholar