Improving Accuracy of LVQ Algorithm by Instance Weighting

  • Marcin Blachnik
  • Włodzisław Duch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6354)

Abstract

Similarity-based methods belong to the most accurate data mining approaches. A large group of such methods is based on instance selection and optimization, with Learning Vector Quantization (LVQ) algorithm being a prominent example. Accuracy of LVQ highly depends on proper initialization of prototypes and the optimization mechanism. Prototype initialization based on context dependent clustering is introduced, and modification of the LVQ cost function that utilizes additional information about class-dependent distribution of training vectors. The new method is illustrated on 6 benchmark datasets, finding simple and accurate models of data in form of prototype-based rules.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aha, D.W. (ed.): Lazy learning. Kluwer Academic Publishers, Norwell (1997)MATHGoogle Scholar
  2. 2.
    Duch, W.: Similarity based methods: a general framework for classification, approximation and association. Control and Cybernetics 29, 937–968 (2000)MATHMathSciNetGoogle Scholar
  3. 3.
    Duch, W., Adamczak, R., Diercksen, G.: Classification, association and pattern completion using neural similarity based methods. Applied Mathematics and Computer Science 10, 101–120 (2000)Google Scholar
  4. 4.
    Pękalska, E., Paclik, P., Duin, R.: A generalized kernel approach to dissimilarity-based classification. Journal of Machine Learning Research 2, 175–211 (2001)CrossRefGoogle Scholar
  5. 5.
    Schölkopf, B., Smola, A.: Learning with Kernels. Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)Google Scholar
  6. 6.
    Sonnenburg, S., Raetsch, G., Schaefer, C., Schoelkopf, B.: Large scale multiple kernel learning. Journal of Machine Learning Research 7, 1531–1565 (2006)Google Scholar
  7. 7.
    Shakhnarovish, G., Darrell, T., Indyk, P. (eds.): Nearest-Neighbor Methods in Learning and Vision. MIT Press, Cambridge (2005)Google Scholar
  8. 8.
    Arya, S., Malamatos, T., Mount, D.: Space-time tradeoffs for approximate nearest neighbor searching. Journal of the ACM 57, 1–54 (2010)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Wilson, D.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans. Systems, Man and Cybernetics 2, 408–421 (1972)MATHCrossRefGoogle Scholar
  10. 10.
    Wilson, D., Martinez, T.: Reduction techniques for instance-based learning algorithms. Machine Learning 38, 257–286 (2000)MATHCrossRefGoogle Scholar
  11. 11.
    Brighton, H., Mellish, C.: Advances in instance selection for instance-based learning algorithms. Data Mining and Knowledge Discovery 6, 153–172 (2002)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Bhattacharya, B., Mukherjee, K., Toussaint, G.: Geometric decision rules for instance-based learning problems. In: Pal, S.K., Bandyopadhyay, S., Biswas, S. (eds.) PReMI 2005. LNCS, vol. 3776, pp. 60–69. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Jankowski, N., Grochowski, M.: Comparison of instance selection algorithms. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 598–603. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Kohonen, T.: Self-organizing maps, 3rd edn. Springer, Heidelberg (2000)Google Scholar
  15. 15.
    Biehl, M., Ghosh, A., Hammer, B.: Dynamics and generalization ability of lvq algorithms. J. Mach. Learn. Res. 8, 323–360 (2007)MathSciNetGoogle Scholar
  16. 16.
    Grochowski, M., Jankowski, N.: Comparison of instance selection algorithms. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 580–585. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Duch, W., Grudziński, K.: Prototype based rules - new way to understand the data. In: IEEE International Joint Conference on Neural Networks, pp. 1858–1863. IEEE Press, Washington (2001)Google Scholar
  18. 18.
    Blachnik, M., Duch, W., Wieczorek, T.: Selection of prototypes rules – context searching via clustering. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS (LNAI), vol. 4029, pp. 573–582. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Pedrycz, W.: Knowledge-Based Clustering: From Data to Information Granules. Wiley Interscience, Hoboken (2005)MATHCrossRefGoogle Scholar
  20. 20.
    Blachnik, M., Duch, W.: Prototype rules from SVM. Springer Studies in Computational Intelligence, vol. 80, pp. 163–184. Springer, Heidelberg (2008)Google Scholar
  21. 21.
    Asuncion, A., Newman, D.: UCI machine learning repository (2009), http://www.ics.uci.edu/~mlearn/MLRepository.html
  22. 22.
    Blachnik, M., Duch, W.: Building Localized Basis Function Networks Using Context Dependent Clustering. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008, Part I. LNCS, vol. 5163, pp. 953–962. Springer, Heidelberg (2008)Google Scholar
  23. 23.
    Haykin, S.: Neural Networks - A Comprehensive Foundation. Maxwell MacMillian Int., New York (1994)MATHGoogle Scholar
  24. 24.
    Wilson, D.R., Martinez, T.R.: Heterogeneous radial basis function networks. In: Proceedings of the International Conference on Neural Networks, vol. 2, pp. 1263–1276 (1996)Google Scholar
  25. 25.
    Spivey, M.: The continuity of mind. Oxford University Press, New York (2007)Google Scholar
  26. 26.
    Duch, W., Blachnik, M.: Fuzzy rule-based systems derived from similarity to prototypes. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 912–917. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Marcin Blachnik
    • 1
  • Włodzisław Duch
    • 2
  1. 1.Department of Management and InformaticsSilesian University of TechnologyKrasinskiego 8Poland
  2. 2.Department of InformaticsNicolaus Copernicus UniversityToruńPoland

Personalised recommendations