Abstract
Prototype based classification offers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suffer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less affected by local optima; (II) the method can be combined with any differentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H.-U. Bauer M. Herrmann T. Villmann (1999) ArticleTitleNeural maps and topographic vector quantization Neural Networks 12 IssueID4-5 659–676
H.-U. Bauer T. Villmann (1997) ArticleTitleGrowing a hypercubical output space in a self-organizing feature map IEEE Transactions on Neural Network 8 IssueID2 218–226
J. C. Bezdek (1981) Pattern Recognition with Fuzzy Objective Function Algorithms Plenum Press New York
Blake, C. L. and Merz, C. J.: UCI repository of machine learning databases. Irvine, CA: University of California, Department of Information and Computer Science, http://www.ics.uci.edu/$\sim$mlearn/MLRepository.html, 1998.
Bojer, T., Hammer, B., Schunk, D. and Tluk von Toschanowitz, K.: Relevance determination in learning vector quantization, In: Proc. of European Symposium on Artificial Neural Networks (ESANN’01), (2001) pp. 271--276, Brussels, Belgium, D facto publications.
Neural Networks Research Centre, Otaniemi: Helsinki University of Technology. Bibliography on the self-organizing map (SOM) and learning vector quantization (LVQ). http://\break liinwww.ira.uka.de/bibliography/Neural/SOM.LVQ.html.
C. Cortes V. Vapnik (1995) ArticleTitleSupport vector network Machine Learning, 20 1–20
Crammer, K., Gilad-Bachrach, R., Navot, A. and Tishby, A.: Margin analysis of the LVQ algorithm. In: Advances in Neural Information Processing Systems (2002), to appear.
R.N. Dav\’e (1990) ArticleTitleFuzzy shell-clustering and application to circle detection in digital images International Journal of General Systems, 16 343–355
W. Duch R. Adamczak K. Grabczewski (2001) ArticleTitleA new methodology of extraction, optimization, and application of crisp and fuzzy logical rules IEEE Transactions on Neural Networks 12 277–306
ECON-Data, A source of economic time series data, INFORUM, University of Maryland, available online at http://www.inform.umd.edu/econdata/ Contents.html.
E. Erwin K. Obermayer K. Schulten (1992) ArticleTitleSelf-organizing maps: ordering, convergence properties, and energy functions Biological Cybernetics 67 IssueID1 47–55
I. Gath A. B. Geva (1989) ArticleTitleUnsupervised optimal fuzzy clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 11 773–781
T. Graepel M. Burger K. Obermayer (1998) ArticleTitleSelf-organizing maps: generalizations and new optimization techniques Neurocomputing 20 173–190 Occurrence Handle10.1016/S0925-2312(98)00010-1 Occurrence Handle0910.68186
E. E. Gustafson W. C. Kessel (Eds) (1979) Fuzzy clustering with a fuzzy covariance matrix. In: IEEE CDC San Diego California 761–766
B. Hammer K. Gersmann (2003) ArticleTitleA note on the universal approximation capability of support vector machines Neural Processing Letters 17 43–53
Hammer, B. Strickert, M. and Villmann, T.: Learning vector quantization for multimodal data. In: J. R. Dorronsoro (ed.),Artificial Neural Networks---ICANN 2002, Springer, (2002), 370--375.
B. Hammer T. Villmann M. Verleysen (Eds) (2002) Batch-RLVQ. European Symposium on Artificial Neural Networks’2002 D-side publications 295–300
B. Hammer T. Villmann (2002) ArticleTitleGeneralized relevance learning vector quantization Neural Networks 15 1059–1068
D. Haussler (1999) Convolutional kernels for dicrete structures. Technical Report UCSC-CRL-99-10, Computer Science Department University of California Santa Cruz
Heskes, T.: Energy functions for self-organizing maps, In: E. Oja and S. Kaski, (eds),\break Kohonen Maps, (1999) 303–315 Springer.
T. Heskes (2001) ArticleTitleOn self-organizing maps, vector quantization, and mixture modeling IEEE Transactions on Neural Networks, 12 IssueID6 1299–1305
T. Jaakkola M. Diekhans D. Haussler (2000) ArticleTitleA discrimitive framework for detecting remote protein homologies Journal of Computational Biology 7 IssueID1-2 95–114
B. H. Juang S. Katagiri (1992) ArticleTitleDiscriminative learning for minimum error classifications IEEE Transactions on Signal Processing 40 IssueID12 3043–3054
Kaski, S. and Sinkkonen, J. A topography-preserving latent variable model with learning metrics. In: N. Allinson, H. Yin, L. Allinson, & J. Slack (eds.), Advances in Self-Organizing Maps, (2001), 224–229, Springer.
S. Kaski (2001) ArticleTitleBankruptcy analysis with self-organizing maps in learning metrics IEEE Transactions on Neural Networks, 12 936–947
Kohonen, T.: Learning vector quantization. In: M. Arbib, (ed.), The Handbook of Brain Theory and Neural Networks, (1995), 537–540. MIT Press.
Kohonen, T.: Self-Organizing Maps, Springer, 1997.
T. Martinetz S. Berkovich K. Schulten (1993) ArticleTitle\lq Neural-gas\rq\ network for vector quantization and its application to time-series prediction IEEE TNN 4 IssueID4 558–569
T. Martinetz K. Schulten (1993) ArticleTitleTopology representing networks IEEE Transactions on Neural Networks 7 IssueID3 507–522
G. Patan\’e M. Russo (2001) ArticleTitleThe enhanced LBG algorithm Neural Networks 14 1219–1237
M. Pregenzer (1996) ArticleTitlePfurtscheller, G. and Flotzinger, D.: Automated feature selection with distinction sensitive learning vector quantization Neurocomputing 11 19–29
H. Ritter T. Martinetz K. Schulten (Eds) (1992) Neural Computation and Self-Organizing Maps An Introduction Addison-Wesley
Sato, A. S. and Yamada, K.: Generalized learning vector quantization. In: G. Tesauro, D. Touretzky, & T. Leen, (eds.), Advances in Neural Information Processing Systems, 7, (1995), 423–429. MIT Press.
Sato, A. S. and Yamada, K.: An analysis of convergence in generalized LVQ. In: L. Niklasson, M. Bod\’en, & T. Ziemke (eds.), ICANN’98, (1998), 172–176. Springer.
B. Schölkopf (2000) The kernel trick for distances. Technical Report MSR-TR-2000-51. Microsoft Research Redmond WA
Schölkopf B. and Smola, A.: Learning with Kernels. MIT Press, 2002.
S. Seo K. Obermayer (2003) ArticleTitleSoft learning vector quantization Neural Computation, 15 1589–1604
Sonnenburg, S., R\”atsch, G., Jagota, A. and M\”uller, K.-R.: New methods for splice site recognition. In: J. R. Dorronsoro (ed.), ICANN’2002, (2002), 329–336, Springer.
I. Steinwart (2001) ArticleTitleOn the influence of the kernel on the consistency of support vector machines Journal of Machine Learning Research, 2 67–93
Strickert, M. Bojer, T. and Hammer, B. Generalized relevance LVQ for time series. In: G. Dorffner, H. Bischof, K. Hornik (eds.), Artificial Neural Networks – ICANN’2001, 2001 Springer, 677--683.
T. Villmann R. Der M. Herrmann T. M. Martinetz (1997) ArticleTitleToplogy preservation in self-organizing feature maps: exact definition and precise measurement IEEE TNN 8 IssueID2 256–266
T. Villmann E. Merenyi B. Hammer (2003) ArticleTitleNeural maps in remote sensing image analysis Neural Networks 16 IssueID3-4 389–403
N. Vlassis A. Likas (2002) ArticleTitleA greedy algorithm for Gaussian mixture learning Neural Processing Letters 15 IssueID1 77–87
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hammer, B., Strickert, M. & Villmann, T. Supervised Neural Gas with General Similarity Measure. Neural Process Lett 21, 21–44 (2005). https://doi.org/10.1007/s11063-004-3255-2
Issue Date:
DOI: https://doi.org/10.1007/s11063-004-3255-2