Skip to main content
SpringerLink
Log in

On the Generalization Ability of GRLVQ Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We derive a generalization bound for prototype-based classifiers with adaptive metric. The bound depends on the margin of the classifier and is independent of the dimensionality of the data. It holds for classifiers based on the Euclidean metric extended by adaptive relevance terms. In particular, the result holds for relevance learning vector quantization (RLVQ) [4] and generalized relevance learning vector quantization (GRLVQ) [19].

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anguita, D., Ridella, S., Rivieccio, F. and Zunino, R.: Automatic hyperparameter tuning for support vector machines. In: J. R. Dorronsoro (ed.), ICANN 2002, pp. 1345–1350, Springer, 2002.

  2. P. L. Bartlett S. Mendelson (2002) ArticleTitleRademacher and Gaussian complexities: risk bounds and structural results J Mach Learn Res 3 463–482

    Google Scholar 

  3. J.C. Bezdek (1981) Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press New York

    Google Scholar 

  4. Bojer, T., Hammer, B., Schunk, D. and Tluk von Toschanowitz, K.: Relevance determination in learning vector quantization. In: Proc. of European Symposium on Artificial Neural Networks (ESANN’01), pp. 271–276, Brussels, Belgium, 2001. D facto publications.

  5. Bousquet, O. and Herrmann, D. J. L.: On the complexity of learning the kernel matrix. In: Advances in Neural Information Processing Systems 2002, to appear.

  6. Neural Networks Research Centre, Otaniemi: Helsinki University of Technology. Bibliography on the self-organizing map (SOM) and learning vector quantization (LVQ). Available at: http://liinwww.ira.uka.de/bibliography/Neural/SOM.LVQ.html.

  7. V. Cherkassky D. Gehring F. Mulier (1996) ArticleTitleComparison of adaptive methods for function estimation from samples IEEE Tr Neural Networ 7 969–984

    Google Scholar 

  8. C. Cortes V. Vapnik (1995) ArticleTitleSupport vector network Mach Learn 20 1–20

    Google Scholar 

  9. Crammer, K., Gilad-Bachrach, R., Navot, A. and Tishby, A.: Margin analysis of the LVQ algorithm. In: Advances in Neural Information Processing Systems 2002, to appear.

  10. R. N. Davé (1990) ArticleTitleFuzzy shell-clustering and application to circle detection in digital images Int J of Gen Syst 16 343–355

    Google Scholar 

  11. I. Gath A. B. Geva (1989) ArticleTitleUnsupervised optimal fuzzy clustering IEEE Tn Pattern Anal Mach Intelligence 11 773–781

    Google Scholar 

  12. van Gestel, T., Suykens, J. A. K., de Moor, B. and Vandewalle, J.: Automatic relevance determination for least squares support vector machine classifiers. In: M. Verleysen (ed.), European Symposium on Artificial Neural Networks 2001, pp. 13–18.

  13. Y. Grandvalet (2000) ArticleTitleAnisotropic noise injection for input variables relevance determination IEEE T Neural Networ 11 IssueID6 1201–1212

    Google Scholar 

  14. S. Günter H. Bunke (2002) ArticleTitleSelf-organizing map for clustering in the graph domain Pattern Recog Lett 23 401–417

    Google Scholar 

  15. E.E. Gustafson W.C. Kessel (1979) Fuzzy clustering with a fuzzy covariance matrix IEEE CDC San Diego, California 761–766

    Google Scholar 

  16. Hammer, B., Rechtien, A., Strickert, M., Villmann, T.: Rule extraction from self-organizing networks. In: J. R. Dorronsoro (ed.), ICANN 2002, pp. 877–882, Springer, 2002.

  17. Hammer, B., Strickert, M. and Villmann, T.: Supervised neural gas with general similarity measure. To appear in Neural Process Lett.

  18. Hammer, B., Strickert, M. and Villmann, T.: Prototype recognition of splice sites. In: Seiffert, U. Jain L. C. and Schweitzer P. (eds.), Bioinformatics using Computational Intelligence Paradigms, Springer, 2004, in press.

  19. B. Hammer T. Villmann (2002) ArticleTitleGeneralized relevance learning vector quantization Neural Networks 15 1059–1068

    Google Scholar 

  20. Heskes, T.: Energy functions for self-organizing maps. In: Oja E. and Kaski S. (eds.), Kohonen Maps, pp. 303–315. Springer, 1999.

  21. S. Kaski (2001) ArticleTitleBankruptcy analysis with self-organizing maps in learning metrics IEEE T Neural Networ 12 936–947

    Google Scholar 

  22. Kaski, S. and Sinkkonen, J.: A topography-preserving latent variable model with learning metrics. In: Allinson, N. Yin, H. Allinson L. and Slack, J. (eds.), Advances in Self-Organizing Maps, pp. 224–229, Springer, 2001.

  23. Kohonen, T.: Learning vector quantization. In: Arbib M. (ed.), The Handbook of Brain Theory and Neural Networks, pp. 537–540. MIT Press, 1995.

  24. Kohonen, T.: Self-Organizing Maps. Springer, 1997.

  25. T. Kohonen P. Somervuo (2002) ArticleTitleHow to make large self-organizing maps for nonvectorial data Neural Networks 15 IssueID8–9 945–952

    Google Scholar 

  26. M. Marchand J. Shawe-Taylor (2002) ArticleTitleThe set covering machine J Mach Learn Resh 3 723–746

    Google Scholar 

  27. F. Mulier (1994) Statistical Analysis of Self-Organization University of Minnesota Minneapolis

    Google Scholar 

  28. Michie, D., Spiegelhalter, D. J. and Taylor, C. C.: (eds.), Machine Learning, Neural and Statistical Classification, Ellis Horwood, 1994.

  29. G. Patané M. Russo (2001) ArticleTitleThe enhanced LBG algorithm Neural Networks 14 1219–1237

    Google Scholar 

  30. M. Pregenzer G. Pfurtscheller D. Flotzinger (1996) ArticleTitleAutomated feature selection with distinction sensitive learning vector quantization Neurocomputing 11 19–29

    Google Scholar 

  31. S. Ridella S. Rovetta R. Zunino (2001) ArticleTitleK-winner machines for pattern classification IEEE T Neural Networ 12 IssueID2 371–385

    Google Scholar 

  32. Sato, A. S. and Yamada, K.: Generalized learning vector quantization. In: G. Tesauro, D. Touretzky and T. Leen (eds.), Advances in Neural Information Processing Systems, Vol. 7, pp. 423–429, MIT Press, 1995.

  33. A. S. Sato and Yamada, K.: An analysis of convergence in generalized LVQ Niklasson, L. Bodén M. and Ziemke T. (eds.), ICANN’98, pp. 172–176, Springer 1998

  34. S. Seo K. Obermeyer (2003) ArticleTitleSoft learning vector quantization Neural computation 15 1589–1604

    Google Scholar 

  35. Strickert, M., Bojer, T. and Hammer, B.: Generalized relevance LVQ for time series. In: Dorffner, G. Bischof, H. and Hornik K. (eds.), Artificial Neural Networks - ICANN’2001, pp. 677–683, Springer, 2001.

  36. R. Tibshirani (1996) ArticleTitleRegression shrinkage and selection via the lasso. J Ro Stat Soc Ser B 58 267–288

    Google Scholar 

  37. M. E. Tipping (2001) ArticleTitleThe relevance vector machine J Mach Lear Res 1 211–244

    Google Scholar 

  38. V. Vapnik:Statistical Learning Theory.Wiley-Interscience, 1998

  39. V. Vapnik A. Chervonenkis (1971) ArticleTitleOn the uniform convergence of relative frequencies of events to their probabilities. Theor Probab Appl 16 IssueID2 264–280

    Google Scholar 

  40. T. Villmann B. Hammer (2003) Metric adaptation and relevance learning in learning vector quantization. Mathematik/Informatik, Universität Osnabrück

    Google Scholar 

  41. T. Villmann E. Merenyi B. Hammer (2003) ArticleTitleNeural maps in remote sensing image analysis. Neural Networks 16 IssueID(3-4) 389–403

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics/Computer Science, Universität Osnabrück, Germany

    Barbara Hammer & Marc Strickert

  2. Clinique for Psychotherapy, Universität Leipzig, Germany

    Thomas Villmann

Authors
  1. Barbara Hammer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Strickert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Villmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barbara Hammer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hammer, B., Strickert, M. & Villmann, T. On the Generalization Ability of GRLVQ Networks. Neural Process Lett 21, 109–120 (2005). https://doi.org/10.1007/s11063-004-1547-1

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-004-1547-1

Keywords

Search

Navigation