Visualizing High Dimensional Classifier Performance Data

  • Rocio Alaiz-Rodríguez
  • Nathalie Japkowicz
  • Peter Tischer
Part of the Studies in Computational Intelligence book series (SCI, volume 223)


Classifier performance evaluation, which typically yields a vast number of results, may be approached as a problem of analyzing high dimensional data. Conducting an exploratory analysis of visual representations of this evaluation data enables us to exploit the advantages of the powerful human visual capabilities. This allows us to gain insight into the performance data, interact with it and draw meaningful conclusions about the classifiers and domains under study. We illustrate how visual techniques, based on a projection from a high dimensional space to a lower dimensional one, enable such an exploratory process. Moreover, this approach can be viewed as a generalization of conventional evaluation procedures based on point metrics that necessarily imply a higher loss of information. Finally, we show that within this framework, the user is able to study the evaluation data from a classifier point of view and from a domain point of view, which is infeasible with traditional evaluation methods.


Root Mean Square Error Performance Data Projection Method High Dimensional Space Original Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alaiz-Rodriguez, R., Japkowicz, N., Tischer, P.: Visualizing classifier performance on different domains. In: Proceedings of the 20th IEEE International Conference on Tools for Artificial Intelligence, ICTAI 2008 (2008)Google Scholar
  2. 2.
    Borg, I., Groenen, P.: Modern Multidimensional Scaling: Theory and Applications. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  3. 3.
    Buja, A., Cook, D., Swayne, D.F.: Interactive high-dimensional data visualization. Journal of Computational and Graphical Statistics 5, 78–99 (1996)CrossRefGoogle Scholar
  4. 4.
    Caruana, R., Niculescu-Mizil, A.: Data mining in metric space: An empirical analysis of suppervised learning performance criteria. In: Proceedings of the Tenth International Conference on Knowledge Discovery and Data Mining, KDD 2004 (2004)Google Scholar
  5. 5.
    Cox, T., Cox, M.: Multidimensional Scaling. Chapman and Hall, Boca Raton (1994)zbMATHGoogle Scholar
  6. 6.
    Ferreira de Oliveira, M.C., Levkowitz, H.: From visual data exploration to visual data mining: A survey. IEEE Transactions on Visualization and Computer Graphics 9(3), 378–394 (2003)CrossRefGoogle Scholar
  7. 7.
    Drummond, C., Holte, R.C.: Cost curves: An improved method for visualizing classifier performance. Machine Learning 65(1), 95–130 (2006)CrossRefGoogle Scholar
  8. 8.
    Fodor, I.K.: A survey of dimension reduction techniques. Technical report (2002)Google Scholar
  9. 9.
    Hand, D.J.: Classifier technology and the illusion of progress. Statistical Sciences 21(1), 1–15 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Holmström, L.: Nonlinear dimensionality reduction by john a. lee, michel verleysen. International Statistical Review 76(2), 308–309 (2008)CrossRefGoogle Scholar
  11. 11.
    Keim, D.A.: Information visualization and visual data mining. IEEE Transactions on Visualization and Computer Graphics 8(1), 1–8 (2002)CrossRefGoogle Scholar
  12. 12.
    Lee, R.C.T., Slagle, J.R., Blum, H.: A triangulation method for the sequential mapping of points from n-space to two-space. IEEE Trans. Comput. 26(3), 288–292 (1977)CrossRefGoogle Scholar
  13. 13.
    Provost, F., Fawcett, T.: Robust classification systems for imprecise environments. Machine Learning 42(3), 203–231 (2001)zbMATHCrossRefGoogle Scholar
  14. 14.
    Soukup, T., Davidson, I.: Visual Data Mining: Techniques and Tools for Data Visualization and Mining. Wiley, Chichester (2002)Google Scholar
  15. 15.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (1999)Google Scholar
  16. 16.
    Yang, L.: Distance-preserving projection of high-dimensional data for nonlinear dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9), 1243–1246 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Rocio Alaiz-Rodríguez
    • 1
  • Nathalie Japkowicz
    • 2
  • Peter Tischer
    • 3
  1. 1.Dpto. de Ingeniería Eléctrica y de Sistemas y AutomaticaUniversidad de LeónLeónSpain
  2. 2.School of Information Technology and EngineeringUniversity of OttawaStn. A OttawaCanada
  3. 3.Clayton School of Information TechnologyMonash UniversityAustralia

Personalised recommendations