On-line gradient learning algorithms for K-nearest neighbor classifiers
We present two online gradient learning algorithms to design condensed k-nearest neighbor (NN) classifiers. The goal of these learning procedures is to minimize a measure of performance closely related to the expected misclassification rate of the k-NN classifier. One possible implementation of the algorithm is given. Converge properties are analyzed and connections with other works are established. We compare these learning procedures with Kononen’s LVQ algorithms  and k-NN classification using the handwritten NIST databases . Experimental results demonstrate the potential of the proposed learning algorithms.
Unable to display preview. Download preview PDF.
- A. Benveniste, M. Métivier and P. Priouret, “Adaptive Algorithms and Stochastic Approximations,” Springer-Verlag, 1990Google Scholar
- L. Bottou “Online Learning and Stochastic Approximations”, Technical Report, AT&T Labs-Research, 1996Google Scholar
- R. O. Duda and P. E. Hart. “Pattern Classification and Scene Analysis”, John Wiley Interscience, 1973Google Scholar
- M. Garris et al. “NIST Form-Based Handprint Recognition System (Release 2.0)”, National Institute of Standards and Technology, 1997Google Scholar
- Robert M. Gray and Richard A. Olshen. “Vector Quantization and Density estimation”, Technical Report, Department of Electrical Engineering, Stanford University, 1997Google Scholar
- T. Kohonen, J. Hynninen, J. Kangas, J. Laaksonen and K. Torkkola, Kari. “LVQ_PAK. The Learning Vector Quantization Program Package. Version 3.1”, Laboratory of Computer and Information Science, Helsinki University of Tecchnology, April 7, 1995Google Scholar
- T. Kohonen. “Self-organizing Maps”, 2nd Edition, Springer-Verlag, 1996.Google Scholar
- D. Ripley “Pattern Recognition and Neural Networks”, Oxford University Press, 1996Google Scholar
- Michael E. Tarter and Michael D. Lock, “Model-Free Curve Estimation”, Chapman & Hall, 1993Google Scholar