Abstract
In this paper, we propose approaching the problem of classifier evaluation in terms of a projection from a high-dimensional space to a visualizable two-dimensional one. Rather than collapsing confusion matrices into a single measure the way traditional evaluation methods do, we consider the vector composed of the entries of the confusion matrix (or the confusion matrices in case several domains are considered simultaneously) as the performance evaluation vector, and project it into a two dimensional space using a recently proposed distance-preserving projection method. This approach is shown to be particularly useful in the case of comparison of several classifiers on many domains as well as in the case of multiclass classification. Furthermore, by providing simultaneous multiple views of the same evaluation data, it allows for a quick and accurate assessment of classifier performance.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Fawcett, T.: ROC graphs: Notes and practical considerations for data mining re searchers. Technical Report HPL 2003-4, HP Laboratories, Palo Alto, CA (January 2003)
Drummond, C., Holte, R.: Cost curves: An improved method for visualizing classifier performance. Machine Learning 65(1), 95–130 (2006)
Jain, A.K., Duin, R.P., Mao, J.: Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1) (2000)
Lee, R., Slagle, J., Blum, H.: A Triangulation Method for the Sequential Mapping of Points from N-Space to Two-Space. IEEE Transactions on Computers 26(3), 288–292 (1977)
Yang, L.: Distance-preserving projection of high dimensional data. Pattern Recognition Letters 25(2), 259–266 (2004)
Blake, C., Merz, C.: Uci repository of machine learning databases (1998)
Witten, I.H., Frank, E.: WEKA Software, v3.5.2. University of Waikato
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Japkowicz, N., Sanghi, P., Tischer, P. (2008). A Projection-Based Framework for Classifier Performance Evaluation. In: Daelemans, W., Goethals, B., Morik, K. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2008. Lecture Notes in Computer Science(), vol 5211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87479-9_54
Download citation
DOI: https://doi.org/10.1007/978-3-540-87479-9_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87478-2
Online ISBN: 978-3-540-87479-9
eBook Packages: Computer ScienceComputer Science (R0)