On-line gradient learning algorithms for K-nearest neighbor classifiers

  • Sergio Bermejo
  • Joan Cabestany
Plasticity Phenomena (Maturing, Learning & Memory)
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1606)


We present two online gradient learning algorithms to design condensed k-nearest neighbor (NN) classifiers. The goal of these learning procedures is to minimize a measure of performance closely related to the expected misclassification rate of the k-NN classifier. One possible implementation of the algorithm is given. Converge properties are analyzed and connections with other works are established. We compare these learning procedures with Kononen’s LVQ algorithms [7] and k-NN classification using the handwritten NIST databases [5]. Experimental results demonstrate the potential of the proposed learning algorithms.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Benveniste, M. Métivier and P. Priouret, “Adaptive Algorithms and Stochastic Approximations,” Springer-Verlag, 1990Google Scholar
  2. [2]
    L. Bottou “Online Learning and Stochastic Approximations”, Technical Report, AT&T Labs-Research, 1996Google Scholar
  3. [3]
    R. O. Duda and P. E. Hart. “Pattern Classification and Scene Analysis”, John Wiley Interscience, 1973Google Scholar
  4. [4]
    M. Garris et al. “NIST Form-Based Handprint Recognition System (Release 2.0)”, National Institute of Standards and Technology, 1997Google Scholar
  5. [5]
    Robert M. Gray and Richard A. Olshen. “Vector Quantization and Density estimation”, Technical Report, Department of Electrical Engineering, Stanford University, 1997Google Scholar
  6. [6]
    T. Kohonen, J. Hynninen, J. Kangas, J. Laaksonen and K. Torkkola, Kari. “LVQ_PAK. The Learning Vector Quantization Program Package. Version 3.1”, Laboratory of Computer and Information Science, Helsinki University of Tecchnology, April 7, 1995Google Scholar
  7. [7]
    T. Kohonen. “Self-organizing Maps”, 2nd Edition, Springer-Verlag, 1996.Google Scholar
  8. [8]
    D. Ripley “Pattern Recognition and Neural Networks”, Oxford University Press, 1996Google Scholar
  9. [9]
    Michael E. Tarter and Michael D. Lock, “Model-Free Curve Estimation”, Chapman & Hall, 1993Google Scholar
  10. [10]
    K. Urahama and Y. Furukawa. “Gradient Descent Learning of Nearest Neighbor Classifiers with Outlier Rejection”, Pattern Recognition, Vol. 28, No. 5, pp. 761–768, 1995CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Sergio Bermejo
    • 1
  • Joan Cabestany
    • 1
  1. 1.Department of Electronic EngineeringUniversitat Politècnica de Catalunya (UPC)BarcelonaSpain

Personalised recommendations