Abstract
In this chapter, one of themost popular and intuitive prototype-based classification algorithms, learning vector quantization (LVQ), is revisited, and recent extensions towards automatic metric adaptation are introduced. Metric adaptation schemes extend LVQ in two aspects: on the one hand a greater flexibility is achieved since the metric which is essential for the classification is adapted according to the given classification task at hand. On the other hand a better interpretability of the results is gained since the metric parameters reveal the relevance of single dimensions as well as correlations which are important for the classification. Thereby, the flexibility of the metric can be scaled from a simple diagonal term to full matrices attached locally to the single prototypes. These choices result in a more complex form of the classification boundaries of the models, whereby the excellent inherent generalization ability of the classifier is maintained, as can be shown by means of statistical learning theory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Alegre, E., Biehl, M., Petkov, N., Sanchez, L.: Automatic classification of the acrosome status of boar spermatozoa using digital image processing and LVQ. Computers in Biology and Medicine 38, 461–468 (2008)
Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian complexities: risk bounds and structural risks. Journal of Machine Learning Research 3, 463–481 (2002)
Biehl, M., Breitling, R., Li, Y.: Analysis of Tiling Microarray Data by Learning Vector Quantization and Relevance Learning. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 880–889. Springer, Heidelberg (2007)
Biehl, M., Caticha, N., Riegler, P.: Statistical mechanics of on-line learning. In: Biehl, M., Hammer, B., Verleysen, M., Villmann, T. (eds.) Similarity-based clustering, biomedical applications, and beyond. Springer, Heidelberg (2009)
Biehl, M., Gosh, A., Hammer, B.: Dynamics and generalization ability of LVQ algorithms. Journal of Machine Learning Research 8, 323–360 (2007)
Biehl, M., Hammer, B., Schneider, P.: Matrix Learning in Learning Vector Quantization, Technical Report Clausthal University of Technology, Department of Computer Science, IfI-06-14 (2006)
Bojer, T., Hammer, B., Koers, C.: Monitoring technical systems with prototype based clustering. In: Verleysen, M. (ed.) European Symposium on Artificial Neural Networks 2003, pp. 433–439. D-side publications (2003)
Bojer, T., Hammer, B., Schunk, D., Tluk von Toschanowitz, K.: Relevance determination in learning vector quantization. In: Proc. of European Symposium on Artificial Neural Networks (ESANN 2001), pp. 271–276. D facto publications (2001)
Bunte, K., Petkov, N., Bosman, H.H.W.J., Biehl, M., Jonkman, M.: Efficient color features for content based image retrieval in dermatolgoy (submitted, 2009)
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., Biehl, M.: Discriminative visualization by limited rank matrix learning. Machine Learning Reports MLR-03-2008 (2008), http://www.uni-leipzig.de/~compint/mlr/mlr_03_2008.pdf , ISSN:1865-3960
Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, A.: Margin analysis of the LVQ algorithm. In: Advances of Neural Information Processing Systems (2002)
Denecke, A., Wersing, H., Steil, J.J., Körner, E.: Robust object segmentation by adaptive metrics in Generalized LVQ (submitted, 2009)
Duda, R.O., Hart, P.E., Storck, D.G.: Pattern Classification. Wiley, Chichester (2001)
Grandvalet, Y.: Anisotropic noise injection for input variable relevance determination. IEEE Transactions on Neural Networks 11(6), 1201–1212 (2000)
Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection 3, 1157–1182 (2003)
Hammer, B., Strickert, M., Villmann, T.: On the generalization ability of GRLVQ networks. Neural Processing Letters 21(2), 109–120 (2005)
Hammer, B., Strickert, M., Villmann, T.: Supervised neural gas with general similarity measure. Neural Processing Letters 21(1), 21–44 (2005)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15, 1059–1068 (2002)
Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (2001)
Lee, J.A., Verleysen, M.: Nonlinear dimensionality reduction. Springer, Heidelberg (2007)
Pregenzer, M., Pfurtscheller, G., Flotzinger, D.: Automated feature selection with distinction sensitive learning vector quantization. Neurocomputing 11, 19–29 (1996)
Sato, A.S., Yamada, K.: An analysis of convergence in generalized LVQ. In: Niklasson, L., Boden, M., Ziemke, T. (eds.) ICANN 1998, pp. 172–176. Springer, Heidelberg (1998)
Sato, A.S., Yamada, K.: Generalized learning vector quantization. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 423–429. MIT Press, Cambridge (1995)
Schleif, F.-M., Hammer, B., Kostrzewa, M., Villmann, T.: Exploration of Mass-Spectrometric Data in Clinical Proteomics Using Learning Vector Quantization Methods. Briefings in Bioinformatics 9(2), 129–143 (2007)
Schneider, P., Biehl, M., Hammer, B.: Matrix adaptation in discriminative vector quantization. Technical Report Clausthal University of Technology, Department of Computer Science, IfI-08-08 (2008)
Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization (submitted, 2009)
Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. Machine Learning Reports MLR-02-2008 (2008), http://www.uni-leipzig.de/~compint/mlr/mlr_02_2008.pdf , ISSN:1865-3960
Seo, S., Bode, M., Obermayer, K.: Soft nearest prototype classification. IEEE Transations on Neural Networks 14(2), 390–398 (2003)
Seo, S., Obermayer, K.: Soft learning vector quantization. Neural Computation 15(7), 1589–1604 (2003)
Sontag, E.D.: Feedforward nets for interpolation and classification. Journal of Computer and System Sciences 45 (1992)
Strickert, M., Seiffert, U., Sreenivasulu, N., Weschke, W., Villmann, T., Hammer, B.: Generalized Relevance LVQ (GRLVQ) with Correlation Measures for Gene Expression Data. Neurocomputing 69, 651–659 (2006)
Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58, 267–288 (1996)
Valiant, L.: A Theory of the Learnable. Communications of the ACM 27(11), 1134–1142 (1984)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Villmann, T., Merenyi, E., Hammer, B.: Neural maps in remote sensing image analysis. Neural Networks 16(3-4), 389–403 (2003)
Villmann, T., Schleif, F.-M., Hammer, B.: Comparison of Relevance Learning Vector Quantization with other Metric Adaptive Classification Methods. Neural Networks 19, 610–622 (2006)
Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: Weiss, Y., Scholkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 1473–1480. MIT Press, Cambridge (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Biehl, M., Hammer, B., Schneider, P., Villmann, T. (2009). Metric Learning for Prototype-Based Classification. In: Bianchini, M., Maggini, M., Scarselli, F., Jain, L.C. (eds) Innovations in Neural Information Paradigms and Applications. Studies in Computational Intelligence, vol 247. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04003-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-04003-0_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04002-3
Online ISBN: 978-3-642-04003-0
eBook Packages: EngineeringEngineering (R0)