Sequential Reduction Algorithm for Nearest Neighbor Rule

  • Marcin Raniszewski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6375)


An effective training set reduction is one of the main problems in constructing fast 1-NN classifiers. A reduced set should be significantly smaller and ought to result in a similar fraction of correct classifications as a complete training set. In this paper a sequential reduction algorithm for nearest neighbor rule is described. The proposed method is based on heuristic idea of sequential adding and eliminating samples. The performance of the described algorithm is evaluated and compared with three other well-known reduction algorithms based on heuristic ideas, on four real datasets extracted from images.


Training Phase Reduction Level Handwritten Digit Neighbor Rule Heuristic Idea 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clarke, F., Ekeland, I., Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2007), Google Scholar
  2. 2.
    Bezdek, J.C., Kuncheva, L.I.: Nearest prototype classifier designs: an experimental study. International Journal of Intelligent Systems 16(12), 1445–1473 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Cerveron, V., Ferri, F.J.: Another move towards the minimum consistent subset: A tabu search approach to the condensed nearest neighbor rule. IEEE Trans. on Systems, Man and Cybernetics, Part B: Cybernetics 31(3), 408–413 (2001)CrossRefGoogle Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Inc., Chichester (2001)zbMATHGoogle Scholar
  5. 5.
    Hart, P.E.: The condensed nearest neighbor rule. IEEE Transactions on Information Theory IT-14(3), 515–516 (1968)Google Scholar
  6. 6.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proc. 14th Int. Joint Conf. Artificial Intelligence, pp. 338–345 (1995)Google Scholar
  7. 7.
    Kuncheva, L.I.: Editing for the k-nearest neighbors rule by a genetic algorithm. Pattern Recognition Letters 16, 809–814 (1995)CrossRefGoogle Scholar
  8. 8.
    Kuncheva, L.I.: Fitness functions in editing k-NN reference set by genetic algorithms. Pattern Recognition 30(6), 1041–1049 (1997)CrossRefGoogle Scholar
  9. 9.
    Kuncheva, L.I., Bezdek, J.C.: Nearest prototype classification: clustering, genetic algorithms, or random search? IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 28(1), 160–164 (1998)CrossRefGoogle Scholar
  10. 10.
    Skalak, D.B.: Prototype and feature selection by sampling and random mutation hill climbing algorithms. In: 11th International Conference on Machine Learning, New Brunswick, NJ, USA, pp. 293–301 (1994)Google Scholar
  11. 11.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 3rd edn. Academic Press/Elsevier, USA (2006)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Marcin Raniszewski
    • 1
  1. 1.Faculty of Physics and Applied InformaticsUniversity of ŁódźŁódźPoland

Personalised recommendations