A Nearest Features Classifier Using a Self-organizing Map for Memory Base Evaluation

  • Christos Pateritsas
  • Andreas Stafylopatis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4132)


Memory base learning is one of main fields in the area of machine learning. We propose a new methodology for addressing the classification task that relies on the main idea of the k – nearest neighbors algorithm, which is the most important representative of this field. In the proposed approach, given an unclassified pattern, a set of neighboring patterns is found, but not necessarily using all input feature dimensions. Also, following the concept of the naïve Bayesian classifier, we adopt the hypothesis of the independence of input features in the outcome of the classification task. The two concepts are merged in an attempt to take advantage of their good performance features. In order to further improve the performance of our approach, we propose a novel weighting scheme of the memory base. Using the self-organizing maps model during the execution of the algorithm, dynamic weights of the memory base patterns are produced. Experimental results have shown superior performance of the proposed method in comparison with the aforementioned algorithms and their variations.


Classification Task Weighting Scheme Near Neighbor Data Pattern Evaluation Factor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aha, D.W., Kibler, D., Albert, M.K.: Instance-Based Learning Algorithms. Machine Learning 6, 37–66 (1991)Google Scholar
  2. 2.
    Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases. University of California, Irvine (1998),
  3. 3.
    Cost, S., Salzberg, S.: A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features. Machine Learning 10, 57–78 (1993)Google Scholar
  4. 4.
    Dasarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  5. 5.
    Demiroz, G., Guvenir, H.A.: Classification by Voting Feature Intervals. In: Proceedings of the 9th European Conference on Machine Learning, Prague (1997)Google Scholar
  6. 6.
    Fan, H., Ramamohanarao, K.: A Bayesian Approach to Use Emerging Patterns for Classification. In: Proceedings of the 14th Australasian Database Conference, Adelaide (2003)Google Scholar
  7. 7.
    Guvenir, H.A., Akkus¸, A.: Weighted K Nearest Neighbor Classification on Feature Projections. In: Proceedings of the 12-th International Symposium on Computer and Information Sciences, Antalya, Turkey (1997)Google Scholar
  8. 8.
    Hammerton, J., Erik, F.: Combining a self-organising map with memorybased learning. In: Conference on Computational Natural Language Learning (CoNLL), Toulouse, France, July 6-7, pp. 9–14 (2001)Google Scholar
  9. 9.
    Kohonen, T.: Self-Organizing Maps. In: Information Sciences, 2nd edn. Springer, Heidelberg (1997)Google Scholar
  10. 10.
    Kononenko, I.: Naive Bayesian classifier and continuous attributes. Informatica 16(1), 1–8 (1992)MathSciNetGoogle Scholar
  11. 11.
    Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Statistics 33, 1065–1076 (1962)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Pateritsas, C., Pertselakis, M., Stafylopatis, A.: A SOM-based classifier with enhanced structure learning. In: Proceedings of the IEEE International Conference on Systems, Man & Cybernetics, The Hague, Netherlands, October 10-13, pp. 4832–4837 (2004)Google Scholar
  13. 13.
    Pateritsas, C., Stafylopatis, A.: Independent Nearest Features Memory-Based Classifier. In: International Conference on Computational Intelligence for Modelling Control and Automation (CIMCA 2005), Vienna, Austria, November 28-30, vol. 2, pp. 781–786 (2005)Google Scholar
  14. 14.
    Rauber, A.: LabelSOM: On the labeling of self-organizing maps. In: Proceedings of International Joint Conference on Neural Networks, Washington, DC (1999)Google Scholar
  15. 15.
    Stanfill, C., Waltz, D.: Toward memory-based reasoning. Communications of the ACM 29(12), 1213–1228 (1986)CrossRefGoogle Scholar
  16. 16.
    Tomek, I.: An experiment with the edited nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics 6(6), 448–452 (1976)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Xin, T., Ozturk, P., Gu, M.: Dynamic feature weighting in nearest neighbor classifiers. In: Proceedings of 2004 International Conference on Machine Learning and Cybernetics, August 26-29, vol. 4, pp. 2406–2411 (2004)Google Scholar
  18. 18.
    Vesanto, J.: Using SOM in Data Mining. Licentiate’s thesis in the Helsinki University of Technology (2000)Google Scholar
  19. 19.
    Wettschereck, D., Aha, W.D.: Weighting Features. In: First International Conference on Case-Based Reasoning, Lisbon, Portugal, pp. 347–358. Springer, Heidelberg (1995)Google Scholar
  20. 20.
    Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6, 1–34 (1997)MATHMathSciNetGoogle Scholar
  21. 21.
    Wilson, D.R., Martinez, T.R.: Reduction Techniques for Instance-Based Learning Algorithm. In: Machine Learning, vol. 38, pp. 257–286. Kluwer Academic Publishers, Dordrecht (2000)Google Scholar
  22. 22.
    Wilson, D.L.: Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics 2(3), 408–421 (1972)MATHCrossRefGoogle Scholar
  23. 23.
    Yang, Y., Webb, G.I.: Proportional k-interval discretization for naive-Bayes classifiers. In: Proceedings of the 12th European Conference on Machine Learning, pp. 564–575 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Christos Pateritsas
    • 1
  • Andreas Stafylopatis
    • 1
  1. 1.School of Electrical and Computer EngineeringNational Technical University of AthensZografou, AthensGreece

Personalised recommendations