The Sequential Reduction Algorithm for Nearest Neighbor Rule Based on Double Sorting

  • Marcin Raniszewski
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 57)


An effective and strong reduction of large training sets is very important for the Nearest Neighbour Rule usefulness. In this paper, the reduction algorithm based on double sorting of a reference set is presented. The samples are sorted with the use of the representative measure and Mutual Neighbourhood Value proposed by Gowda and Krishna. Then, the reduced set is built by sequential adding and removing samples according to double sort order. The results of proposed algorithm are compared with results of well-known reduction procedures on nine real and one artificial datasets.


Hill Climbing Representative Measure Neighbor Rule Pattern Recognition Letter Pima Indian Diabetes 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2007),
  2. 2.
    Cerveron, V., Ferri, F.J.: Another move towards the minimum consistent subset: A tabu search approach to the condensed nearest neighbor rule. IEEE Trans. on Systems, Man and Cybernetics, Part B: Cybernetics 31(3), 408–413 (2001)CrossRefGoogle Scholar
  3. 3.
    Dasarathy, B.V.: Minimal consistent set (MCS) identification for optimal nearest neighbor decision systems design. IEEE Transactions on Systems, Man, and Cybernetics 24(3), 511–517 (1994)CrossRefGoogle Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Inc., Chichester (2001)zbMATHGoogle Scholar
  5. 5.
    Fisher, R.A.: The use of multiple measures in taxonomic problems. Ann. Eugenics 7, 179–188 (1936)CrossRefGoogle Scholar
  6. 6.
    Gates, G.W.: The reduced nearest neighbor rule. IEEE Transactions on Information Theory IT 18(5), 431–433 (1972)CrossRefGoogle Scholar
  7. 7.
    Gowda, K.C., Krishna, G.: The condensed nearest neighbor rule using the concept of mutual nearest neighborhood. IEEE Transaction on Information Theory IT-25(4), 488–490 (1979)CrossRefGoogle Scholar
  8. 8.
    Hart, P.E.: The condensed nearest neighbor rule. IEEE Transactions on Information Theory IT-14(3), 515–516 (1968)CrossRefGoogle Scholar
  9. 9.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proc. 14th Int. Joint Conf. Artificial Intelligence, pp. 338–345 (1995)Google Scholar
  10. 10.
    Kuncheva, L.I.: Editing for the k-nearest neighbors rule by a genetic algorithm. Pattern Recognition Letters 16, 809–814 (1995)CrossRefGoogle Scholar
  11. 11.
    Kuncheva, L.I.: Fitness functions in editing k-NN reference set by genetic algorithms. Pattern Recognition 30(6), 1041–1049 (1997)CrossRefGoogle Scholar
  12. 12.
    Kuncheva, L.I., Bezdek, J.C.: Nearest prototype classification: clustering, genetic algorithms, or random search? IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 28(1), 160–164 (1998)CrossRefGoogle Scholar
  13. 13.
    Nakai, K., Kanehisa, M.: Expert System for Predicting Protein Localization Sites in Gram-Negative Bacteria. PROTEINS: Structure, Function, and Genetics 11, 95–110 (1991)CrossRefGoogle Scholar
  14. 14.
    Raniszewski, M.: Reference set reduction algorithms based on double sorting. In: Computer Recognition Systems 2: the 5th International Conference on Computer Recognition Systems CORES 2007, pp. 258–265. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Skalak, D.B.: Prototype and feature selection by sampling and random mutation hill climbing algorithms. In: 11th International Conference on Machine Learning, New Brunswick, NJ, USA, pp. 293–301 (1994)Google Scholar
  16. 16.
  17. 17.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 3rd edn. Academic Press, Elsevier (2006)Google Scholar
  18. 18.
    Tomek, I.: Two modifications of CNN. IEEE Transactions on Systems, Man and Cybernetics SMC-6(11), 769–772 (1976)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Toussaint, G.T.: A counter-example to Tomek’s consistency theorem for a condensed nearest neighbor decision rule. Pattern Recognition Letters 15(8), 797–801 (1994)CrossRefzbMATHGoogle Scholar
  20. 20.
    Wilson, D.R., Martinez, T.R.: Reduction techniques for instance-based learning algorithms. Machine Learning 38(3), 257–286 (2000)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Marcin Raniszewski
    • 1
  1. 1.Computer Engineering DepartmentTechnical University of LodzPoland

Personalised recommendations