Skip to main content
Log in

Supervised Neural Gas with General Similarity Measure

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Prototype based classification offers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suffer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less affected by local optima; (II) the method can be combined with any differentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • H.-U. Bauer M. Herrmann T. Villmann (1999) ArticleTitleNeural maps and topographic vector quantization Neural Networks 12 IssueID4-5 659–676

    Google Scholar 

  • H.-U. Bauer T. Villmann (1997) ArticleTitleGrowing a hypercubical output space in a self-organizing feature map IEEE Transactions on Neural Network 8 IssueID2 218–226

    Google Scholar 

  • J. C. Bezdek (1981) Pattern Recognition with Fuzzy Objective Function Algorithms Plenum Press New York

    Google Scholar 

  • Blake, C. L. and Merz, C. J.: UCI repository of machine learning databases. Irvine, CA: University of California, Department of Information and Computer Science, http://www.ics.uci.edu/$\sim$mlearn/MLRepository.html, 1998.

  • Bojer, T., Hammer, B., Schunk, D. and Tluk von Toschanowitz, K.: Relevance determination in learning vector quantization, In: Proc. of European Symposium on Artificial Neural Networks (ESANN’01), (2001) pp. 271--276, Brussels, Belgium, D facto publications.

  • Neural Networks Research Centre, Otaniemi: Helsinki University of Technology. Bibliography on the self-organizing map (SOM) and learning vector quantization (LVQ). http://\break liinwww.ira.uka.de/bibliography/Neural/SOM.LVQ.html.

  • C. Cortes V. Vapnik (1995) ArticleTitleSupport vector network Machine Learning, 20 1–20

    Google Scholar 

  • Crammer, K., Gilad-Bachrach, R., Navot, A. and Tishby, A.: Margin analysis of the LVQ algorithm. In: Advances in Neural Information Processing Systems (2002), to appear.

  • R.N. Dav\’e (1990) ArticleTitleFuzzy shell-clustering and application to circle detection in digital images International Journal of General Systems, 16 343–355

    Google Scholar 

  • W. Duch R. Adamczak K. Grabczewski (2001) ArticleTitleA new methodology of extraction, optimization, and application of crisp and fuzzy logical rules IEEE Transactions on Neural Networks 12 277–306

    Google Scholar 

  • ECON-Data, A source of economic time series data, INFORUM, University of Maryland, available online at http://www.inform.umd.edu/econdata/ Contents.html.

  • E. Erwin K. Obermayer K. Schulten (1992) ArticleTitleSelf-organizing maps: ordering, convergence properties, and energy functions Biological Cybernetics 67 IssueID1 47–55

    Google Scholar 

  • I. Gath A. B. Geva (1989) ArticleTitleUnsupervised optimal fuzzy clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 11 773–781

    Google Scholar 

  • T. Graepel M. Burger K. Obermayer (1998) ArticleTitleSelf-organizing maps: generalizations and new optimization techniques Neurocomputing 20 173–190 Occurrence Handle10.1016/S0925-2312(98)00010-1 Occurrence Handle0910.68186

    Article  MATH  Google Scholar 

  • E. E. Gustafson W. C. Kessel (Eds) (1979) Fuzzy clustering with a fuzzy covariance matrix. In: IEEE CDC San Diego California 761–766

    Google Scholar 

  • B. Hammer K. Gersmann (2003) ArticleTitleA note on the universal approximation capability of support vector machines Neural Processing Letters 17 43–53

    Google Scholar 

  • Hammer, B. Strickert, M. and Villmann, T.: Learning vector quantization for multimodal data. In: J. R. Dorronsoro (ed.),Artificial Neural Networks---ICANN 2002, Springer, (2002), 370--375.

  • B. Hammer T. Villmann M. Verleysen (Eds) (2002) Batch-RLVQ. European Symposium on Artificial Neural Networks’2002 D-side publications 295–300

    Google Scholar 

  • B. Hammer T. Villmann (2002) ArticleTitleGeneralized relevance learning vector quantization Neural Networks 15 1059–1068

    Google Scholar 

  • D. Haussler (1999) Convolutional kernels for dicrete structures. Technical Report UCSC-CRL-99-10, Computer Science Department University of California Santa Cruz

    Google Scholar 

  • Heskes, T.: Energy functions for self-organizing maps, In: E. Oja and S. Kaski, (eds),\break Kohonen Maps, (1999) 303–315 Springer.

  • T. Heskes (2001) ArticleTitleOn self-organizing maps, vector quantization, and mixture modeling IEEE Transactions on Neural Networks, 12 IssueID6 1299–1305

    Google Scholar 

  • T. Jaakkola M. Diekhans D. Haussler (2000) ArticleTitleA discrimitive framework for detecting remote protein homologies Journal of Computational Biology 7 IssueID1-2 95–114

    Google Scholar 

  • B. H. Juang S. Katagiri (1992) ArticleTitleDiscriminative learning for minimum error classifications IEEE Transactions on Signal Processing 40 IssueID12 3043–3054

    Google Scholar 

  • Kaski, S. and Sinkkonen, J. A topography-preserving latent variable model with learning metrics. In: N. Allinson, H. Yin, L. Allinson, & J. Slack (eds.), Advances in Self-Organizing Maps, (2001), 224–229, Springer.

  • S. Kaski (2001) ArticleTitleBankruptcy analysis with self-organizing maps in learning metrics IEEE Transactions on Neural Networks, 12 936–947

    Google Scholar 

  • Kohonen, T.: Learning vector quantization. In: M. Arbib, (ed.), The Handbook of Brain Theory and Neural Networks, (1995), 537–540. MIT Press.

  • Kohonen, T.: Self-Organizing Maps, Springer, 1997.

  • T. Martinetz S. Berkovich K. Schulten (1993) ArticleTitle\lq Neural-gas\rq\ network for vector quantization and its application to time-series prediction IEEE TNN 4 IssueID4 558–569

    Google Scholar 

  • T. Martinetz K. Schulten (1993) ArticleTitleTopology representing networks IEEE Transactions on Neural Networks 7 IssueID3 507–522

    Google Scholar 

  • G. Patan\’e M. Russo (2001) ArticleTitleThe enhanced LBG algorithm Neural Networks 14 1219–1237

    Google Scholar 

  • M. Pregenzer (1996) ArticleTitlePfurtscheller, G. and Flotzinger, D.: Automated feature selection with distinction sensitive learning vector quantization Neurocomputing 11 19–29

    Google Scholar 

  • H. Ritter T. Martinetz K. Schulten (Eds) (1992) Neural Computation and Self-Organizing Maps An Introduction Addison-Wesley

    Google Scholar 

  • Sato, A. S. and Yamada, K.: Generalized learning vector quantization. In: G. Tesauro, D. Touretzky, & T. Leen, (eds.), Advances in Neural Information Processing Systems, 7, (1995), 423–429. MIT Press.

  • Sato, A. S. and Yamada, K.: An analysis of convergence in generalized LVQ. In: L. Niklasson, M. Bod\’en, & T. Ziemke (eds.), ICANN’98, (1998), 172–176. Springer.

  • B. Schölkopf (2000) The kernel trick for distances. Technical Report MSR-TR-2000-51. Microsoft Research Redmond WA

    Google Scholar 

  • Schölkopf B. and Smola, A.: Learning with Kernels. MIT Press, 2002.

  • S. Seo K. Obermayer (2003) ArticleTitleSoft learning vector quantization Neural Computation, 15 1589–1604

    Google Scholar 

  • Sonnenburg, S., R\”atsch, G., Jagota, A. and M\”uller, K.-R.: New methods for splice site recognition. In: J. R. Dorronsoro (ed.), ICANN’2002, (2002), 329–336, Springer.

  • I. Steinwart (2001) ArticleTitleOn the influence of the kernel on the consistency of support vector machines Journal of Machine Learning Research, 2 67–93

    Google Scholar 

  • Strickert, M. Bojer, T. and Hammer, B. Generalized relevance LVQ for time series. In: G. Dorffner, H. Bischof, K. Hornik (eds.), Artificial Neural Networks – ICANN’2001, 2001 Springer, 677--683.

  • T. Villmann R. Der M. Herrmann T. M. Martinetz (1997) ArticleTitleToplogy preservation in self-organizing feature maps: exact definition and precise measurement IEEE TNN 8 IssueID2 256–266

    Google Scholar 

  • T. Villmann E. Merenyi B. Hammer (2003) ArticleTitleNeural maps in remote sensing image analysis Neural Networks 16 IssueID3-4 389–403

    Google Scholar 

  • N. Vlassis A. Likas (2002) ArticleTitleA greedy algorithm for Gaussian mixture learning Neural Processing Letters 15 IssueID1 77–87

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hammer, B., Strickert, M. & Villmann, T. Supervised Neural Gas with General Similarity Measure. Neural Process Lett 21, 21–44 (2005). https://doi.org/10.1007/s11063-004-3255-2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-004-3255-2

Keywords

Navigation