Using Nonlinear Dimensionality Reduction to Visualize Classifiers

  • Alexander Schulz
  • Andrej Gisbrecht
  • Barbara Hammer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7902)


Nonlinear dimensionality reduction (DR) techniques offer the possibility to visually inspect a given finite high-dimensional data set in two dimensions. In this contribution, we address the problem to visualize a trained classifier on top of these projections. We investigate the suitability of popular DR techniques for this purpose and we point out the benefit of integrating auxiliary information as provided by the classifier into the pipeline based on the Fisher information.


Visualization of Classifiers Supervised Dimensionality Reduction Fisher Information 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12, 2385–2404 (2000)CrossRefGoogle Scholar
  2. 2.
    Bunte, K., Biehl, M., Hammer, B.: A general framework for dimensionality reducing data visualization mapping. Neural Computation 24(3), 771–804 (2012)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., Biehl, M.: Limited rank matrix learning, discriminative dimension reduction and visualization. Neural Networks 26, 159–173 (2012)CrossRefGoogle Scholar
  4. 4.
    Caragea, D., Cook, D., Wickham, H., Honavar, V.G.: Visual methods for examining svm classifiers. In: Simoff, S.J., Böhlen, M.H., Mazeika, A. (eds.) Visual Data Mining. LNCS, vol. 4404, pp. 136–153. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011), Software available at
  6. 6.
    Cohn, D.: Informed projections. In: Becker, S., Thrun, S., Obermayer, K. (eds.) NIPS, pp. 849–856. MIT Press (2003)Google Scholar
  7. 7.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  8. 8.
    Gisbrecht, A., Hofmann, D., Hammer, B.: Discriminative dimensionality reduction mappings (2012)Google Scholar
  9. 9.
    Gisbrecht, A., Lueks, W., Mokbel, B., Hammer, B.: Out-of-sample kernel extensions for nonparametric dimensionality reduction. In: ESANN 2012, pp. 531–536 (2012)Google Scholar
  10. 10.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Advances in Neural Information Processing Systems 17, pp. 513–520. MIT Press (2004)Google Scholar
  11. 11.
    Hernandez-Orallo, J., Flach, P., Ferri, C.: Brier curves: a new cost-based visualisation of classifier performance. In: International Conference on Machine Learning (June 2011)Google Scholar
  12. 12.
    Jakulin, A., Možina, M., Demšar, J., Bratko, I., Zupan, B.: Nomograms for visualizing support vector machines. In: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, KDD 2005, pp. 108–117. ACM, New York (2005)Google Scholar
  13. 13.
    Kohonen, T., Hynninen, J., Kangas, J., Laaksonen, J., Torkkola, K.: LVQ_PAK: The Learning Vector Quantization program package. Report A30, Helsinki University of Technology, Laboratory of Computer and Information Science (January 1996)Google Scholar
  14. 14.
    Kreßel, U.H.-G.: Advances in kernel methods. In: Chapter Pairwise Classification and Support Vector Machines, pp. 255–268. MIT Press, Cambridge (1999)Google Scholar
  15. 15.
    Lee, J.A., Verleysen, M.: Nonlinear dimensionality reduction. Springer (2007)Google Scholar
  16. 16.
    Ma, B., Qu, H., Wong, H.: Kernel clustering-based discriminant analysis. Pattern Recognition 40(1), 324–327 (2007)CrossRefzbMATHGoogle Scholar
  17. 17.
    Peltonen, J., Klami, A., Kaski, S.: Improved learning of riemannian metrics for exploratory analysis. Neural Networks 17, 1087–1100 (2004)CrossRefzbMATHGoogle Scholar
  18. 18.
    Poulet, F.: Visual svm. In: Chen, C.-S., Filipe, J., Seruca, I., Cordeiro, J. (eds.) ICEIS (2), pp. 309–314 (2005)Google Scholar
  19. 19.
    Roweis, S.: Machine learning data sets (2012),
  20. 20.
    Schulz, A., Gisbrecht, A., Bunte, K., Hammer, B.: How to visualize a classifier? In: New Challenges in Neural Computation, pp. 73–83 (2012)Google Scholar
  21. 21.
    Tenenbaum, J., da Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  22. 22.
    van der Maaten, L., Hinton, G.: Visualizing high-dimensional data using t-sne. Journal of Machine Learning Research 9, 2579–2605 (2008)zbMATHGoogle Scholar
  23. 23.
    Vellido, A., Martin-Guerroro, J., Lisboa, P.: Making machine learning models interpretable. In: ESANN 2012 (2012)Google Scholar
  24. 24.
    Venna, J., Peltonen, J., Nybo, K., Aidos, H., Kaski, S.: Information retrieval perspective to nonlinear dimensionality reduction for data visualization. Journal of Machine Learning Research 11, 451–490 (2010)zbMATHMathSciNetGoogle Scholar
  25. 25.
    Wang, X., Wu, S., Wang, X., Li, Q.: Svmv - a novel algorithm for the visualization of svm classification results. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3971, pp. 968–973. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  26. 26.
    Ward, M., Grinstein, G., Keim, D.A.: Interactive Data Visualization: Foundations, Techniques, and Application. A. K. Peters, Ltd. (2010)Google Scholar
  27. 27.
    Weinberger, K.Q., Saul, L.K.: Unsupervised learning of image manifolds by semidefinite programming. Int. J. Comput. Vision 70(1), 77–90 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Alexander Schulz
    • 1
  • Andrej Gisbrecht
    • 1
  • Barbara Hammer
    • 1
  1. 1.CITEC Centre of ExcellenceUniversity of BielefeldGermany

Personalised recommendations