Fast Multistage Algorithm for K-NN Classifiers

  • I. Soraluze
  • C. Rodriguez
  • F. Boto
  • A. Cortes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2905)


In this paper we present a way to reduce the computational cost of k-NN classifiers without losing classification power. Hierarchical or multistage classifiers have been built with this purpose. These classifiers are designed putting incrementally trained classifiers into a hierarchy and using rejection techniques in all the levels of the hierarchy apart from the last. Results are presented for different benchmark data sets: some standard data sets taken from the UCI Repository and the Statlog Project, and NIST Special Databases (digits and upper-case and lower-case letters). In all the cases a computational cost reduction is obtained maintaining the recognition rate of the best individual classifier obtained.


Recognition Rate Near Neighbor Good Recognition Rate Multistage Classifier Handwritten Digit Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Dasarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  2. 2.
    Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, New York (1996)zbMATHGoogle Scholar
  3. 3.
    Giusti, N., Masulli, F., Sperduti, A.: Theoretical and Experimental Analysis of a Two-Stage System for Classification. IEEE Trans. PAMI 24(7), 893–905 (2002)Google Scholar
  4. 4.
    Hart, P.E.: The Condensed Nearest Neighbour Rule. IEEE Transactions on Information Theory 14, 515–516 (1968)CrossRefGoogle Scholar
  5. 5.
    Ho, T.K., Hull, J.J., Srihari, S.N.: Decision Combination in Multiple Classifier Systems. IEEE Trans. PAMI 16(1), 66–75 (1994)Google Scholar
  6. 6.
    Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning databases. Department of Information and Computer Science. University of California, Irvine (1994), Google Scholar
  7. 7.
    Paredes, R., Vidal, E.: A class-dependent weighted dissimilarity measure for nearest neighbor classification problems. Pattern Recognition Letters 21, 1027–1036 (2000)zbMATHCrossRefGoogle Scholar
  8. 8.
    Soraluze, I., Rodriguez, C., Boto, F., Perez, A.: Multidimensional Multistage K-NN classifiers for handwritten digit recognition. In: Proceedings of eighth IWFHR, Niagara-on-the lake (Ontario), Canada, pp. 19–23 (2002)Google Scholar
  9. 9.
    Soraluze, I., Rodriguez, C., Boto, F., Perez, A.: An Incremental and Hierarchical k-NN classifier for Handwritten characters. In: 16th International conference on Pattern Recognition Quebec, Canada (2002)Google Scholar
  10. 10.
    Vijaya Saradhi, V., Narasimha Murty, M.: Bootstrapping for efficient handwritten digit recognition. Pattern recognition 34, 1047–1056 (2001)zbMATHCrossRefGoogle Scholar
  11. 11.
    Murthy, S.K., Salzberg, S.: A system for the induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 133 (1994)Google Scholar
  12. 12.
    Wilkinson, R.A., Geist, J., Janet, S., Groter, P., Burges, R., Creecy, R., Hammond, B., Hull, J., Larsen, N., Vogl, T., Wilson, C.: First Census Optical Character Recognition System Conference. National Institute of Standards and Technology (NIST) (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • I. Soraluze
    • 1
  • C. Rodriguez
    • 1
  • F. Boto
    • 1
  • A. Cortes
    • 1
  1. 1.Computer Architecture and Technology DepartmentComputer Science Faculty, UPV/EHUSan SebastianSpain

Personalised recommendations