Discriminatory Data Mapping by Matrix-Based Supervised Learning Metrics

  • M. Strickert
  • P. Schneider
  • J. Keilwagen
  • T. Villmann
  • M. Biehl
  • B. Hammer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5064)


Supervised attribute relevance detection using cross-comparisons (SARDUX), a recently proposed method for data-driven metric learning, is extended from dimension-weighted Minkowski distances to metrics induced by a data transformation matrix Ω for modeling mutual attribute dependence. Given class labels, parameters of Ω are adapted in such a manner that the inter-class distances are maximized, while the intra-class distances get minimized. This results in an approach similar to Fisher’s linear discriminant analysis (LDA), however, the involved distance matrix gets optimized, and it can be finally utilized for generating discriminatory data mappings that outperform projection pursuit methods with LDA index. The power of matrix-based metric optimization is demonstrated for spectrum data and for cancer gene expression data.


Supervised feature characterization adaptive matrix metrics attribute dependence modeling projection pursuit LDA 


  1. 1.
    Blake, C., Merz, C.: UCI repository of machine learning databases (1998)Google Scholar
  2. 2.
    Cook, D., Swayne, D.: Interactive and Dynamic Graphics for Data Analysis with R and GGobi. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  3. 3.
    Faith, J., Mintram, R., Angelova, M.: Targeted projection pursuit for visualizing gene expression data classifications. Bioinformatics 22(21), 2667–2673 (2006)CrossRefGoogle Scholar
  4. 4.
    Friedman, J.: Exploratory projection pursuit. Journal of the American Statistical Association 82, 249–266 (1987)CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., Lander, E.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439), 531–537 (1999)CrossRefGoogle Scholar
  6. 6.
    Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.: Feature Extraction: Foundations and Applications. Springer, Berlin (2006)zbMATHGoogle Scholar
  7. 7.
    Hammer, B., Strickert, M., Villmann, T.: Supervised neural gas with general similarity measure. Neural Processing Letters 21(1), 21–44 (2005)CrossRefGoogle Scholar
  8. 8.
    Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15, 1059–1068 (2002)CrossRefGoogle Scholar
  9. 9.
    Hu, S., Rao, J.: Statistical redundancy testing for improved gene selection in cancer classification using microarray data. Cancer Informatics 2, 29–41 (2007)Google Scholar
  10. 10.
    Hyvärinen, A., Oja, E.: Independent component analysis: Algorithms and applications. Neural Networks 13(4–5), 411–430 (2000)CrossRefGoogle Scholar
  11. 11.
    Kaski, S.: From learning metrics towards dependency exploration. In: Cottrell, M. (ed.) Proceedings of the 5th International Workshop on Self-Organizing Maps (WSOM), pp. 307–314 (2005)Google Scholar
  12. 12.
    Lee, E., Cook, D., Klinke, S., Lumley, T.: Projection pursuit for exploratory supervised classification. Journal of Computational and Graphical Statistics 14(4), 831–846 (2005)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization (Submitted to Machine Learning) (2008)Google Scholar
  14. 14.
    Strickert, M., Seiffert, U., Sreenivasulu, N., Weschke, W., Villmann, T., Hammer, B.: Generalized relevance LVQ (GRLVQ) with correlation measures for gene expression data. Neurocomputing 69, 651–659 (2006)CrossRefGoogle Scholar
  15. 15.
    Strickert, M., Witzel, K., Mock, H.-P., Schleif, F.-M., Villmann, T.: Supervised attribute relevance determination for protein identification in stress experiments. In: Proc. of Machine Learning in Systems Biology (MLSB), pp. 81–86 (2007)Google Scholar
  16. 16.
    Sun, Y.: Iterative relief for feature weighting: Algorithms, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(6), 1035–1051 (2007)CrossRefGoogle Scholar
  17. 17.
    Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 1473–1480. MIT Press, Cambridge (2006)Google Scholar
  18. 18.
    Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning, with application to clustering with side-information. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems 15 (NIPS), pp. 505–512. MIT Press, Cambridge (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • M. Strickert
    • 1
  • P. Schneider
    • 2
  • J. Keilwagen
    • 1
  • T. Villmann
    • 3
  • M. Biehl
    • 2
  • B. Hammer
    • 4
  1. 1.Leibniz Institute of Plant Genetics and Crop Plant Research Gatersleben 
  2. 2.Institute for Mathematics and Computing ScienceUniversity of Groningen 
  3. 3.Research group Computational IntelligenceUniversity of Leipzig 
  4. 4.Institute of Computer ScienceTechnical University of Clausthal 

Personalised recommendations