Advertisement

Neural Computing and Applications

, Volume 25, Issue 3–4, pp 511–524 | Cite as

A review of learning vector quantization classifiers

  • David Nova
  • Pablo A. Estévez
Invited Review

Abstract

In this work, we present a review of the state of the art of learning vector quantization (LVQ) classifiers. A taxonomy is proposed which integrates the most relevant LVQ approaches to date. The main concepts associated with modern LVQ approaches are defined. A comparison is made among eleven LVQ classifiers using one real-world and two artificial datasets.

Keywords

Learning vector quantization Supervised learning Neural networks Margin maximization Likelihood ratio maximization 

Notes

Acknowledgments

This work was funded by CONICYT-CHILE under Grant FONDECYT 1110701.

References

  1. 1.
    Ahn KK, Nguyen HTC (2007) Intelligent switching control of a pneumatic muscle robot arm using learning vector quantization neural network. Mechatronics 17(4):255–262CrossRefGoogle Scholar
  2. 2.
    Anagnostopoulos C, Anagnostopoulos J, Vergados D, Kayafas E, Loumos V, Theodoropoulos G (2001) Training a learning vector quantization network for biomedical classification. In: Proceedings of the international joint conference on neural networks, National Technical University of Athens (NTUA), Electrical and Computer Engineering Deparment, vol 4, pp 2506–2511Google Scholar
  3. 3.
    Bashyal S, Venayagamoorthy GK (2008) Recognition of facial expressions using gabor wavelets and learning vector quantization. Eng Appl Artif Intell 21(7):1056–1064CrossRefGoogle Scholar
  4. 4.
    Bassiuny A, Li X, Du R (2007) Fault diagnosis of stamping process based on empirical mode decomposition and learning vector quantization. Int J Mach Tools Manuf 47(15):2298–2306CrossRefGoogle Scholar
  5. 5.
    Baum EB (1991) Neural net algorithms that learn in polynomial time from examples and queries. IEEE Trans Neural Netw 2(1):5–19CrossRefGoogle Scholar
  6. 6.
    Bezdek JC, Pal NR (1995) Two soft relatives of learning vector quantization. Neural Netw 8(5):729–743CrossRefGoogle Scholar
  7. 7.
    Biehl M, Hammer B (2007) Dynamics and generalization ability of LVQ algorithms 8:323–360MATHMathSciNetGoogle Scholar
  8. 8.
    Blume M, Ballard DR (1997) Image annotation based on learning vector quantization and localized Haar wavelet transform features. In: Rogers SK (ed) Society of photo-optical instrumentation engineers (SPIE) conference series, society of photo-optical instrumentation engineers (SPIE) conference series, vol 3077, pp 181–190Google Scholar
  9. 9.
    Chang CY, Chang CH, Li CH, Der Jeng M (2007) Learning vector quantization neural networks for led wafer defect inspection. In: Innovative computing, information and control, 2007. ICICIC’07. Second international conference on, IEEE, pp 229–229Google Scholar
  10. 10.
    Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning, vol 2. MIT press, CambridgeGoogle Scholar
  11. 11.
    Chen CY (2012) Accelerometer-based hand gesture recognition using fuzzy learning vector quantization. Adv Sci Lett 9(1):38–44CrossRefGoogle Scholar
  12. 12.
    Crammer K, Gilad-Bachrach R, Navot A, Tishby A (2002) Margin analysis of the LVQ algorithm. Adv Neural Inf Process Syst 15:462–469Google Scholar
  13. 13.
    Dieterle F, Muller-Hagedorn S, Liebich HM, Gauglitz G (2003) Urinary nucleosides as potential tumor markers evaluated by learning vector quantization. Artif Intell Med 28(3):265–280CrossRefGoogle Scholar
  14. 14.
    Dutta S, Chatterjee A, Munshi S (2011) Identification of ecg beats from cross-spectrum information aided learning vector quantization. Measurement 44(10):2020–2027CrossRefGoogle Scholar
  15. 15.
    Frank A, Asuncion A (2010) UCI machine learning repository. http://archive.ics.uci.edu/ml
  16. 16.
    Fritzke B, et al (1995) A growing neural gas network learns topologies. Adv Neural Inf Process Syst 7:625–632Google Scholar
  17. 17.
    González AI, Grana M, D’Anjou A (1995) An analysis of the glvq algorithm. IEEE Trans Neural Netw 6(4):1012–1016CrossRefGoogle Scholar
  18. 18.
    Hammer B, Villmann T (2002) Generalized relevance learning vector quantization. Neural Netw 15(8–9):1059–1068CrossRefGoogle Scholar
  19. 19.
    Hammer B, Strickert M, Villmann T (2004) Relevance lvq versus svm. In: Rutkowski L, Siekmann J, Tadeusiewicz R, Zadeh L (eds) Artificial intelligence and soft computing (ICAISC 2004). Lecture notes in artificial intelligence, vol 3070, Springer, Berlin, pp 592–597Google Scholar
  20. 20.
    Hammer B, Strickert M, Villmann T (2005) On the generalization ability of grlvq networks. Neural Process Lett 21(2):109–120CrossRefGoogle Scholar
  21. 21.
    Hammer B, Strickert M, Villmann T (2005) Supervised neural gas with general similarity measure. Neural Process Lett 21(1):21–44CrossRefGoogle Scholar
  22. 22.
    Hammer B, Mokbel B, Schleif FM, Zhu X (2011) Prototype-based classification of dissimilarity data. In: Gama J, Bradley E, Hollmén J (eds) Advances in intelligent data analysis X. Lecture notes in computer science, vol 7014, pp 185–197Google Scholar
  23. 23.
    Hammer B, Schleif FM, Zhu X (2011) Relational extensions of learning vector quantization. In: Neural information processing, Springer, Berlin, pp 481–489Google Scholar
  24. 24.
    Hammer B, Gisbrecht A, Schulz A (2013) How to visualize large data sets? In: Estévez PA, Príncipe JC, Zegers P (eds) Advances in self-organizing maps. In: Advances in intelligent systems and computing, vol 198. Springer, Berlin, pp 1–12Google Scholar
  25. 25.
    Hastie T, Tibshirani R, Friedman JJH (2001) The elements of statistical learning, vol 1. Springer, New YorkCrossRefGoogle Scholar
  26. 26.
    Hochberg Y, Tamhane AC (1987) Multiple comparison procedures. Wiley, NJMATHCrossRefGoogle Scholar
  27. 27.
    Hofmann D, Hammer B (2012) Kernel robust soft learning vector quantization. Lecture Notes Artif Intell 7477:14–23Google Scholar
  28. 28.
    Hofmann D, Gisbrecht A, Hammer B (2013) Efficient approximations of kernel robust soft lvq. In: Estévez PA, Príncipe JC, Zegers P (eds) Advances in self-organizing maps. In: Advances in intelligent systems and computing, vol 198. Springer, Berlin, pp 183–192Google Scholar
  29. 29.
    Hung WL, Chen DH, Yang MS (2011) Suppressed fuzzy-soft learning vector quantization for mri segmentation. Artif Intell Med 52(1):33–43CrossRefGoogle Scholar
  30. 30.
    Jeng JY, Mau TF, Leu SM (2000) Prediction of laser butt joint welding parameters using back propagation and learning vector quantization networks. J Mater Process Technol 99(1):207–218CrossRefGoogle Scholar
  31. 31.
    Jirayusakul A, Auwatanamongkol S (2007) A supervised growing neural gas algorithm for cluster analysis. Int J Hybrid Intell Syst 4(2):129–141MATHGoogle Scholar
  32. 32.
    Karayiannis NB (1997) A methodology for constructing fuzzy algorithms for learning vector quantization. IEEE Trans Neural Netw 8(3):505–518CrossRefGoogle Scholar
  33. 33.
    Karayiannis NB (1999) An axiomatic approach to soft learning vector quantization and clustering. IEEE Trans Neural Netw 10(5):1153–1165CrossRefGoogle Scholar
  34. 34.
    Karayiannis NB, Pai PI (1996) Fuzzy algorithms for learning vector quantization. IEEE Trans Neural Netw 7(5):1196–1211CrossRefGoogle Scholar
  35. 35.
    Karayiannis NB, Zervos N (2000) Entropy-constrained learning vector quantization algorithms and their application in image compression. J Electron Imaging 9(4):495–508CrossRefGoogle Scholar
  36. 36.
    Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16CrossRefGoogle Scholar
  37. 37.
    Kohonen T (1990) Improved versions of learning vector quantization. In: Neural networks, 1990. 1990 IJCNN international joint conference on, IEEE, pp 545–550Google Scholar
  38. 38.
    Kohonen T (1997) Self-organizing maps. Springer-Verlag New York, Inc., Secaucus, NJ, USAMATHCrossRefGoogle Scholar
  39. 39.
    Lehn-Schiøler T, Hegde A, Erdogmus D, Principe JC (2005) Vector quantization using information theoretic concepts. Nat Comput 4(1):39–51MathSciNetCrossRefGoogle Scholar
  40. 40.
    Lendasse A, Verleysen M, De Bodt E, Cottrell M, Grégoire P (1998) Forecasting time-series by kohonen classification. In: Proceedings of European symposium on artificial neural networks, pp 221–226Google Scholar
  41. 41.
    Lieberman MA, Patil RB (1997) Evaluation of learning vector quantization to classify cotton trash. Opt Eng 36(3):914–921CrossRefGoogle Scholar
  42. 42.
    Martinetz TM, Berkovich SG, Schulten KJ (1993) Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Trans Neural Netw 4(4):558–569CrossRefGoogle Scholar
  43. 43.
    Mitra P, Murthy C, Pal SK (2004) A probabilistic active support vector learning algorithm. IEEE Trans Pattern Anal Mach Intell 26(3):413–418CrossRefGoogle Scholar
  44. 44.
    Nanopoulos A, Alcock R, Manolopoulos Y (2001) Feature-based classification of time-series data. Int J Comput Res 49–61Google Scholar
  45. 45.
    Neural Networks Research Centre Helsinki University of Technology (2005) Bibliography on the self-organizing map (som) and learning vector quantization (lvq). http://liinwww.ira.uka.de/bibliography/Neural/SOM.LVQ.html
  46. 46.
    Nova D, Estévez PA (2013) Online visualization of prototypes and receptive fields produced by lvq algorithms. In: Estévez PA, Príncipe JC, Zegers P (eds) Advances in self-organizing maps. In: Advances in intelligent systems and computing, vol 198. Springer, Berlin, pp 173–182Google Scholar
  47. 47.
    Pal NR, Bezdek JC, Tsao EK (1993) Generalized clustering networks and kohonen’s self-organizing scheme. IEEE Trans Neural Netw 4(4):549–557CrossRefGoogle Scholar
  48. 48.
    Pękalska E, Duin RP (2005) The dissimilarity representation for pattern recognition: foundations and applications. 64, World Scientific, SingaporeGoogle Scholar
  49. 49.
    Pesu L, Helisto P, Ademovic E, Pesquet J, Saarinen A, Sovijärvi A (1998) Classification of respiratory sounds based on wavelet packet decomposition and learning vector quantization. Technol Health Care 6(1):65–74Google Scholar
  50. 50.
    Pradhan N, Sadasivan P, Arunodaya G (1996) Detection of seizure activity in eeg by an artificial neural network: a preliminary study. Comput Biomed Res 29(4):303–313CrossRefGoogle Scholar
  51. 51.
    Principe JC, Xu D, Fisher J (2000) Information theoretic learning. In: Haykin S (ed) Unsupervised adaptive filtering. Wiley, New York, NYGoogle Scholar
  52. 52.
    Qin AK, Suganthan P (2004) A novel kernel prototype-based learning algorithm. In: Pattern recognition, 2004. ICPR 2004. Proceedings of the 17th international conference on, vol 4, pp 621–624Google Scholar
  53. 53.
    Qin AK, Suganthan PN (2005) Initialization insensitive LVQ algorithm based on cost-function adaptation. Pattern Recognit 38(5):773–776MATHCrossRefGoogle Scholar
  54. 54.
    Qin AK, Suganthan P, Liang JJ (2004) A new generalized lvq algorithm via harmonic to minimum distance measure transition. In: 2004 IEEE international conference on systems, man and cybernetics, vol 5, pp 4821–4825Google Scholar
  55. 55.
    Salzberg SL (1997) On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min Knowl Discov 1(3):317–328CrossRefGoogle Scholar
  56. 56.
    Sato A, Yamada K (1996) Generalized learning vector quantization. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) Advances in neural information processing systems, vol 8. MIT Press, Cambridge, pp 423–429Google Scholar
  57. 57.
    Savio A, García-Sebastián M, Hernández C, Graña M, Villanúa J (2009) Classification results of artificial neural networks for alzheimer’s disease detection. Intelligent data engineering and automated learning—IDEAL 2009, pp 641–648Google Scholar
  58. 58.
    Schleif FM, Hammer B, Villmann T (2007) Margin-based active learning for LVQ networks. Neurocomputing 70(7–9):1215–1224CrossRefGoogle Scholar
  59. 59.
    Schleif FM, Villmann T, Hammer B, Schneider P (2011) Efficient kernelized prototype based classification. Int J Neural Syst 21(06):443CrossRefGoogle Scholar
  60. 60.
    Schneider P, Biehl M, Hammer B (2009) Adaptive relevance matrices in learning vector quantization. Neural Comput 21(12):3532–3561MATHMathSciNetCrossRefGoogle Scholar
  61. 61.
    Schneider P, Biehl M, Hammer B (2009) Distance learning in discriminative vector quantization. Neural Comput 21(10):2942–69MATHMathSciNetCrossRefGoogle Scholar
  62. 62.
    Scholkopf B, Mika S, Burges CJ, Knirsch P, Muller KR, Ratsch G, Smola AJ (1999) Input space versus feature space in kernel-based methods. IEEE Trans Neural Netw 10(5):1000–1017CrossRefGoogle Scholar
  63. 63.
    Seo S, Obermayer K (2003) Soft learning vector quantization. Neural Comput 15(7):1589–1604MATHCrossRefGoogle Scholar
  64. 64.
    Seo S, Bode M, Obermayer K (2003) Soft nearest prototype classification. IEEE Trans Neural Netw 14(2):390–8CrossRefGoogle Scholar
  65. 65.
    Strickert M, Bojer T (2001) Generalized relevance LVQ for time series. In: Artificial neural networks—ICANN’2001, pp 677–683Google Scholar
  66. 66.
    Torkkola K (2003) Feature extraction by non parametric mutual information maximization. J Mach Learn Res 3:1415–1438MATHMathSciNetGoogle Scholar
  67. 67.
    Torkkola K, Campbell WM (2000) Mutual information in learning feature transformations. In: Proceedings of the 17th international conference on machine learning, Morgan Kaufmann, pp 1015–1022Google Scholar
  68. 68.
    Tse P, Wang DD, Xu J (1995) Classification of image texture inherited with overlapped features using learning vector quantization. In: Proceedings of the second international conference on mechatronics and machine vision in practice. M/sup 2/VIP ‘95, City University Hong Kong, Hong Kong, pp 286–290Google Scholar
  69. 69.
    Villmann T, Haase S (2011) Divergence-based vector quantization. Neural Comput 23(5):1343–92MATHMathSciNetCrossRefGoogle Scholar
  70. 70.
    Villmann T, Hammer B, Schleif FM, Hermann W, Cottrell M (2008) Fuzzy classification using information theoretic learning vector quantization. Neurocomputing 71(16–18):3070–3076CrossRefGoogle Scholar
  71. 71.
    Williams C, Seeger M (2001) Using the nystrom method to speed up kernel machines. In: Leen TK, Dietterich TG, Tresp V (eds) Advances in neural information processing systems 13, MIT Press, pp 682–688Google Scholar
  72. 72.
    Xuan J, Adali T (1995) Learning tree-structured vector quantization for image compression. In: Proceedings of WCNN’95, world congress on neural networks, INNS, vol I, pp 756–759Google Scholar
  73. 73.
    Yang HT, Liao CC, Chou JH (2001) Fuzzy learning vector quantization networks for power transformer condition assessment. IEEE Trans Dielectr Electr Insul 8(1):143–149CrossRefGoogle Scholar
  74. 74.
    Zhang B, Hsu M, Dayal U (1999) K-harmonic means-a data clustering algorithm. Hewllet-Packard Research Laboratory Technical Report HPL-1999-124Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.Department of Electrical Engineering, Faculty of Physical and Mathematical SciencesUniversity of ChileSantiagoChile
  2. 2.Department of Electrical Engineering and Advanced Mining Technology Center, Faculty of Physical and Mathematical SciencesUniversity of ChileSantiagoChile

Personalised recommendations