Abstract
Prototype based classification approaches are powerful classifiers for class discrimination of vectorial data. Famous examples are learning vector quantization models (LVQ) and support vector machines (SVMs). In this paper we propose the application of kernel distances in LVQ such that the LVQ-algorithm can handle the data in a topologically equivalent data space compared to the feature mapping space in SVMs. Further, we provide strategies to force the LVQ-prototypes to be class border sensitive. In this way an alternative to SVMs based on Hebbian learning is established. After presenting the theoretical background, we demonstrate the abilities of the model for an illustrative toy example and for the more challenging task of classification of Wilson’s disease patients according to their neurophysiological impairments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aronszajn, N.: Theory of reproducing kernels. Transactions of the American Mathematical Society 68, 337–404 (1950)
Barthel, H., Villmann, T., Hermann, W., Hesse, S., Kühn, H.-J., Wagner, A., Kluge, R.: Different patterns of brain glucose consumption in Wilsons disease. Zeitschrift für Gastroenterologie 39, 241 (2001)
Biehl, M., Hammer, B., Schneider, P., Villmann, T.: Metric learning for prototype-based classification. In: Bianchini, M., Maggini, M., Scarselli, F., Jain, L. (eds.) Innovations in Neural Information Paradigms and Applications. SCI, vol. 247, pp. 183–199. Springer, Berlin (2009)
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., Biehl, M.: Limited rank matrix learning, discriminative dimension reduction and visualization. Neural Networks 26(1), 159–173 (2012)
Hammer, B., Strickert, M., Villmann, T.: Supervised neural gas with general similarity measure. Neural Processing Letters 21(1), 21–44 (2005)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15(8-9), 1059–1068 (2002)
Haykin, S.: Neural Networks. A Comprehensive Foundation. Macmillan, New York (1994)
Hermann, W., Barthel, H., Hesse, S., Grahmann, F., Kühn, H.-J., Wagner, A., Villmann, T.: Comparison of clinical types of Wilson’s disease and glucose metabolism in extrapyramidal motor brain regions. Journal of Neurology 249(7), 896–901 (2002)
Hermann, W., Eggers, B., Barthel, H., Clark, D., Villmann, T., Hesse, S., Grahmann, F., Kühn, H.-J., Sabri, O., Wagner, A.: Correlation between automated writing movements and striatal dopaminergic innervation in patients with Wilson’s disease. Journal of Neurology 249(8), 1082–1087 (2002)
Hermann, W., Günther, P., Wagner, A., Villmann, T.: Klassifikation des Morbus Wilson auf der Basis neurophysiologischer Parameter. Der Nervenarzt 76, 733–739 (2005)
Hermann, W., Villmann, T., Grahmann, F., Kühn, H., Wagner, A.: Investigation of fine motoric disturbances in Wilson’s disease. Neurological Sciences 23(6), 279–285 (2003)
Hermann, W., Villmann, T., Wagner, A.: Elektrophysiologisches Schädigungsprofil von Patienten mit einem Morbus Wilson’. Der Nervenarzt 74(10), 881–887 (2003)
Hermann, W., Wagner, A., Kühn, H.-J., Grahmann, F., Villmann, T.: Classification of fine-motoric disturbances in Wilson’s disease using artificial neural networks. Acta Neurologica Scandinavia 111(6), 400–406 (2005)
Herrmann, M., Bauer, H.-U., Der, R.: The ‘perceptual magnet’ effect: A model based on self-organizing feature maps. In: Smith, L.S., Hancock, P.J.B. (eds.) Neural Computation and Psychology, Stirling, pp. 107–116. Springer (1994)
Kästner, M., Hammer, B., Biehl, M., Villmann, T.: Functional relevance learning in generalized learning vector quantization. Neurocomputing 90(9), 85–95 (2012)
Kohonen, T.: Self-Organizing Maps (Second Extended Edition). Springer Series in Information Sciences, vol. 30. Springer, Heidelberg (1997)
Martinetz, T.M., Berkovich, S.G., Schulten, K.J.: ‘Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4(4), 558–569 (1993)
Mercer, J.: Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, London, A 209, 415–446 (1909)
Sato, A., Yamada, K.: Generalized learning vector quantization. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Proceedings of the 1995 Conference on Advances in Neural Information Processing Systems 8, pp. 423–429. MIT Press, Cambridge (1996)
Schleif, F.-M., Villmann, T., Hammer, B., Schneider, P.: Efficient kernelized prototype based classification. International Journal of Neural Systems 21(6), 443–457 (2011)
Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press (2002)
Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Transactions on Neural Networks 21(5), 831–840 (2010)
Schneider, P., Hammer, B., Biehl, M.: Adaptive relevance matrices in learning vector quantization. Neural Computation 21, 3532–3561 (2009)
Schneider, P., Hammer, B., Biehl, M.: Distance learning in discriminative vector quantization. Neural Computation 21, 2942–2969 (2009)
Steinwart, I.: On the influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research 2, 67–93 (2001)
Strickert, M.: Enhancing M|G|RLVQ by quasi step discriminatory functions using 2nd order training. Machine Learning Reports 5(MLR-06-2011), 5–15 (2011), http://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr_06_2011.pdf , ISSN:1865-3960
Strickert, M., Schleif, F.-M., Seiffert, U., Villmann, T.: Derivatives of Pearson correlation for gradient-based analysis of biomedical data. Inteligencia Artificial, Revista Iberoamericana de Inteligencia Artificial (37), 37–44 (2008)
Villmann, T.: Sobolev metrics for learning of functional data - mathematical and theoretical aspects. Machine Learning Reports, 1(MLR-03-2007), 1–15 (2007), http://www.uni-leipzig.de/~compint/mlr/mlr_01_2007.pdf , ISSN:1865-3960
Villmann, T., Geweniger, T., Kästner, M.: Border sensitive fuzzy classification learning in fuzzy vector quantization. Machine Learning Reports, 6(MLR-06-2012), 23–39 (2012), http://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr_06_2012.pdf , ISSN:1865-3960
Villmann, T., Haase, S.: Divergence based vector quantization. Neural Computation 23(5), 1343–1392 (2011)
Villmann, T., Haase, S., Kästner, M.: Gradient based learning in vector quantization using differentiable kernels. In: Estevez, P.A., Principe, J.C., Zegers, P. (eds.) Advances in Self-Organizing Maps. AISC, vol. 198, pp. 193–204. Springer, Heidelberg (2013)
Witoelar, A., Gosh, A., de Vries, J., Hammer, B., Biehl, M.: Window-based example selection in learning vector quantization. Neural Computation 22(11), 2924–2961 (2010)
Yin, C., Mu, S., Tian, S.: Using cooperative clustering to solve multiclass problems. In: Wang, Y., Li, T. (eds.) Foundations of Intelligent Systems. AISC, vol. 122, pp. 327–334. Springer, Heidelberg (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kästner, M., Riedel, M., Strickert, M., Hermann, W., Villmann, T. (2013). Border-Sensitive Learning in Kernelized Learning Vector Quantization. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-38679-4_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38678-7
Online ISBN: 978-3-642-38679-4
eBook Packages: Computer ScienceComputer Science (R0)