Advertisement

Neural Computing and Applications

, Volume 21, Issue 6, pp 1205–1215 | Cite as

An incremental learning vector quantization algorithm for pattern classification

  • Ye Xu
  • Furao Shen
  • Jinxi Zhao
Original Article

Abstract

Prototype classifiers have been studied for many years. However, few methods can realize incremental learning. On the other hand, most prototype classifiers need users to predetermine the number of prototypes; an improper prototype number might undermine the classification performance. To deal with these issues, in the paper we propose an online supervised algorithm named Incremental Learning Vector Quantization (ILVQ) for classification tasks. The proposed method has three contributions. (1) By designing an insertion policy, ILVQ incrementally learns new prototypes, including both between-class incremental learning and within-class incremental learning. (2) By employing an adaptive threshold scheme, ILVQ automatically learns the number of prototypes needed for each class dynamically according to the distribution of training data. Therefore, unlike most current prototype classifiers, ILVQ needs no prior knowledge of the number of prototypes or their initial value. (3) A technique for removing useless prototypes is used to eliminate noise interrupted into the input data. Results of experiments show that the proposed ILVQ can accommodate the incremental data environment and provide good recognition performance and storage efficiency.

Keywords

Learning vector quantization Incremental learning Adaptive threshold Classification compression ratio 

Notes

Acknowledgments

This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (#BK2009080).

References

  1. 1.
    Blake CL, Merz CJ (1996) UCI repository of machine learning databases. University of California Department of Information, IrvineGoogle Scholar
  2. 2.
    Carpenter G, Grossberg S (1990) Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures. Neural Netw 3(3):129–152CrossRefGoogle Scholar
  3. 3.
    Cauwenberghs G, Poggio T (2002) Incremental and decremental support vector machine learning. In: NIPS02Google Scholar
  4. 4.
    Dasarathy BV (1991) Nearest neighbor NN norms: NN Pattern Classification Techniques. IEEE Computer Society PressGoogle Scholar
  5. 5.
    Devijver P, Kittler J (1982) Pattern recognition: a statistical approach. Prentince-Hall, NJzbMATHGoogle Scholar
  6. 6.
    Duda R, Hart D, Stork P (2001) Pattern classification, 2nd edn. Wiley Press, NYzbMATHGoogle Scholar
  7. 7.
    Eick CF, Zeidat N, Vilalta R (2004) Using representative-based clustering for nearest neighbor dataset editing. In: ICDM04, pp 375–378Google Scholar
  8. 8.
    Fritzke B (1995) A growing neural gas network learns topologies. In: NIPS95, pp 625–632Google Scholar
  9. 9.
    Gates G (2004) The reduced nearest neighbor rule. IEEE Trans Inf Theory 18(3):431–433CrossRefGoogle Scholar
  10. 10.
    Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning: data mining, inference, and prediction. SpringerGoogle Scholar
  11. 11.
    Haykin S (2004) A comprehensive foundation, 2nd edn. China Machine Press, ChinaGoogle Scholar
  12. 12.
    Kim SW, Oommen BJ (2005) On using prototype reduction schemes and classifier fusion strategies to optimize kernel-based nonlinear subspace methods. IEEE Trans PAMI 27(3):455–460CrossRefGoogle Scholar
  13. 13.
    Kohonen T (1990) Improved versions of learning vector quantization. In IJCNN90, pp 545–550Google Scholar
  14. 14.
    Lia B, Xuea X, Fan J (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans SMC Part C 31:497–508Google Scholar
  15. 15.
    Likas S, Vlassis N, Verbeek J (2005) The global k-means clustering algorithm. Pattern Recogn 36(2):451–461Google Scholar
  16. 16.
    Martinetz TM, Schulten K (1994) Topology representing networks. Neural Netw 7(3):507–522CrossRefGoogle Scholar
  17. 17.
    Mitra P, Murthy CA, Pal SK (2002) Density-based multiscale data condensation. IEEE Trans PAMI 24(6):734–747CrossRefGoogle Scholar
  18. 18.
    Mollineda RA, Ferri FJ, Vidal E (2002) A merge-based condensing strategy for multiple prototype classifiers. IEEE Trans SMC, Part B 32(5):662–668Google Scholar
  19. 19.
    Passerini A, Frasconi P (2002) From margins to probabilities in multiclass learning problems. In: ECAI02Google Scholar
  20. 20.
    Pekalska E, Duin RPW (2008) Beyond traditional kernels: classification in two dissimilarity-based representation spaces. IEEE Trans SMC, Part C 38(6):729–744Google Scholar
  21. 21.
    Polikar R, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans SMC Part C 31:497–508Google Scholar
  22. 22.
    Ren C-X, Dai D-Q (2010) Incremental learning of bidirectional principal components for face recognition. Pattern Recogn Lett 43:318–330zbMATHGoogle Scholar
  23. 23.
    Sato A, Yamada K (1995) Generalized learning vector quantization. In: NIPS95, pp 424–429Google Scholar
  24. 24.
    Shen F, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Networks 21(10):1537–1547CrossRefGoogle Scholar
  25. 25.
    Veenman CJ, Reinders MJT (2005) The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier. IEEE Trans PAMI 27(9):1417–1429CrossRefGoogle Scholar
  26. 26.
    Villegas M, Paredes R (2008) Simultaneous learning of a discriminative projection and prototypes for nearest-neighbor classification. In: CVPR08, pp 1–8Google Scholar
  27. 27.
    Wasan MT (1969) Stochastic approximation. Cambridge University Press, CambridgezbMATHGoogle Scholar
  28. 28.
    Wilson DL (1972) Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans SMC, Part B 2(5):408–420zbMATHGoogle Scholar
  29. 29.
    Wu M, Scholkopf B (2007) A local learning approach for clustering. In: NIPS07, pp 1529–1536Google Scholar
  30. 30.
    Zhang H, Berg AC, Maire M, Malik J (2006) Svm-knn: Discriminative nearest neighbor classification for visual category recognition. In: CVPR06, pp 2126–2134Google Scholar
  31. 31.
    Zhang H, Malik J (2003) Learning a discriminative classifier using shape context distances. In: CVPR03, pp 242–247Google Scholar
  32. 32.
    Zhang J, Gruenwald L (2006) Opening the black box of feature extraction: Incorporating visualization into high-dimensional data mining processes. In: ICDM06, pp 18–22Google Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations