A Modular Reduction Method for k-NN Algorithm with Self-recombination Learning

  • Hai Zhao
  • Bao-Liang Lu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3971)


A difficulty faced by existing reduction techniques for k-NN algorithm is to require loading the whole training data set. Therefore, these approaches often become inefficient when they are used for solving large-scale problems. To overcome this deficiency, we propose a new method for reducing samples for k-NN algorithm. The basic idea behind the proposed method is a self-recombination learning strategy, which is originally designed for combining classifiers to speed up response time by reducing the number of base classifiers to be checked and improve the generalization performance by rearranging the order of training samples. Experimental results on several benchmark problems indicate that the proposed method is valid and efficient.


Test Accuracy Near Neighbor Negative Class Positive Class Modular Reduction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dasarathy, B.V. (ed.): Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  2. 2.
    MacQueen, J.B.: Some Methods for Classification and Analysis of Multi Variate Observations. In: MacQueen, J.B. (ed.) Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)Google Scholar
  3. 3.
    Kohonen, T.: Self-Organizing Maps. Springer, Germany (1995)Google Scholar
  4. 4.
    Hart, P.E.: The Condensed Nearest Neighbor Rule. IEEE Trans. on Information Theory 14(5), 515–516 (1967)Google Scholar
  5. 5.
    Devijver, P.A., Kittler, J.: Pattern Recognition: A Statistical Approach. Prentice/Hall, London (1982)MATHGoogle Scholar
  6. 6.
    Wilson, D.R., Martinez, T.R.: Reduction Techniques for Instance-Based Learning Algorithms. Machine Learning 38(3), 257–286 (2000)MATHCrossRefGoogle Scholar
  7. 7.
    Zhao, H., Lu, B.L.: A Modular k-Nearest Neighbor Classification Method for Massively Parallel Text Categorization. In: Zhang, J., He, J.-H., Fu, Y. (eds.) CIS 2004. LNCS, vol. 3314, pp. 867–872. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Giacinto, G., Roli, F.: Automatic Design of Multiple Classifier Systems by Unsupervised Learning. In: Perner, P., Petrou, M. (eds.) MLDM 1999. LNCS (LNAI), vol. 1715, pp. 131–143. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  9. 9.
    Frosyniotis, D., Stafylopatis, A., Likas, A.: A Divide-and-conquer Method for Multi-net Classifiers. Pattern Anal. Applic. 6, 32–40 (2003)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Zhao, H.: A Study on Min-max Modular Classifier (in Chinese). Doctoral dissertation of Shanghai Jiao Tong University (2005)Google Scholar
  11. 11.
    Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Data-bases. Department of Information and Computer Science. University of California, Irvine (1998), Google Scholar
  12. 12.
    Ratsch, G., Onoda, T., Muller, K.: Soft Margins for AdaBoost. Machine Learning, 1–35 (2000)Google Scholar
  13. 13.
    Lu, B.L., Ito, M.: Task Decomposition and Module Combination Based on Class Relations: a Modular Neural Network for Pattern Classification. IEEE Transactions on Neural Networks 10, 1244–1256 (1999)CrossRefGoogle Scholar
  14. 14.
    Chawla, N.V., Hall, L.O., Bowyer, K.W., Kegelmeyer, K.P.: Learning Ensembles from Bites: A Scalable and Accurate Approach. Journal of Machine Learning Research 5, 421–451 (2004)MathSciNetGoogle Scholar
  15. 15.
    Yang, Y., Lu, B.L.: Structure Pruning Strategies for Min-Max Modular Network. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3496, pp. 646–651. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Wilson, D.L.: Asymptotic Properties of Nearest Neighbor Rules Using Edited Data. IEEE Transactions on Systems, Man, and Cybernetics 2-3, 408–421 (1972)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hai Zhao
    • 1
  • Bao-Liang Lu
    • 1
  1. 1.Department of Computer Science and EngineeringShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations