Pattern Recognition pp 48-85 | Cite as

# Nearest Neighbour Based Classifiers

## Abstract

One of the simplest decision procedures that can be used for classification is the nearest neighbour (NN) rule. It classifies a sample based on the category of its nearest neighbour. When large samples are involved, it can be shown that this rule has a probability of error which is less than twice the optimum error—hence there is less than twice the probability of error compared to any other decision rule. The nearest neighbour based classifiers use some or all the patterns available in the training set to classify a test pattern. These classifiers essentially involve finding the similarity between the test pattern and every pattern in the training set.

## Keywords

Leaf Node Class Label Test Pattern Neighbour Search Near Neighbour## Preview

Unable to display preview. Download preview PDF.

## Bibliography

- 1.Broder, A. J. Strategies for efficient incremental nearest neighbour search.
*Pattern Recognition*23(1/2): 171–178. 1990.CrossRefGoogle Scholar - 2.Chang, C. L. Finding prototypes for nearest neighbour classifiers.
*IEEE Trans. on Computers*C-23(11): 1179–1184. 1974.CrossRefGoogle Scholar - 3.Cover, T. M. and P. E. Hart. Nearest neighbor pattern classification
*IEEE Trans. on Information Theory*IT-13: 21–27. 1967.CrossRefGoogle Scholar - 4.Dasarathy, Belur V. Minimal consistent set (MCS) identification for optimal nearest neighbour decision system design.
*IEEE Trans. on Systems, Man and Cybernetics*24(3). 1994.Google Scholar - 5.Dejiver, P. A. and J. Kittler. On the edited nearest neighbour rule.
*Proceedings of the 5th International Conference on Pattern Recognition*. pp. 72–80. 1980.Google Scholar - 6.Dudani, S. A. The distance-weighted
*k*nearest neighbor rule.*IEEE Trans. on SMC*SMC-6(4): 325–327. 1976.Google Scholar - 7.Friedman, J. H., F. Baskett and L. J. Shustek. An algorithm for finding nearest neighbours.
*IEEE Trans on Computers*C-24(10): 1000–1006. 1975.CrossRefGoogle Scholar - 8.Fukunaga, K. and P. M. Narendra. A branch and bound algorithm for computing
*k*nearest neighbours.*IEEE Trans. on Computers*. pp. 750–753. 1975.Google Scholar - 9.Gates, G. W. The reduced nearest neighbour rule.
*IEEE Trans. on Information Theory*IT-18(3): 431–433. 1972.CrossRefGoogle Scholar - 10.Gowda, K.C. and G. Krishna. Edit and error correction using the concept of mutual nearest neighbourhood.
*International Conference on Cybernetics and Society*. pp. 222–226. 1979.Google Scholar - 11.Hart, P. E. The condensed nearest neighbor rule.
*IEEE Trans. on Information Theory*IT-14(3): 515–516. 1968.CrossRefGoogle Scholar - 12.Jozwik, A. A learning scheme for a fuzzy
*k*NN rule.*Pattern Recognition Letters*1(5/6): 287–289. 1983.CrossRefGoogle Scholar - 13.Kim, B. S. and S. B. Park. A fast nearest neighbour finding algorithm based on the ordered partition.
*IEEE Trans on PAMI*PAMI-8(6): 761–766. 1986.CrossRefGoogle Scholar - 14.Kuncheva, L. Editing for the
*k*nearest neighbours rule by a genetic algorithm.*Pattern Recognition Letters*16(8): 809–814. 1995.CrossRefGoogle Scholar - 15.Kuncheva, L. and L. C. Jain. Nearest neighbor classifier: Simultaneous editing and feature selection.
*Pattern Recognition Letters*20: 1149–1156. 1999.CrossRefGoogle Scholar - 16.Lai, Jim Z. C., Yi-Ching Liaw and Julie Liu. Fast
*k*nearest neighbour search based on projection and triangular inequality.*Pattern Recognition*40: 351–359. 2007.zbMATHCrossRefGoogle Scholar - 17.McNames, James. A fast nearest neighbour algorithm based on a principal axis search tree.
*IEEE Trans. on Pattern Analysis and Machine Intelligence*23(9): 964–976. 2001.CrossRefGoogle Scholar - 18.Miclet, L. and M.Dabouz. Approximative fast nearest neighbour recognition.
*Pattern Recognition Letters*1: 277–285. 1983.CrossRefGoogle Scholar - 19.Papadimitriou, C. H. and J. L. Bentley. A worst-case analysis of nearest neighbour searching by projection.
*Lecture Notes in Computer Science*85: 470–482. 1980.MathSciNetGoogle Scholar - 20.Patrick, E. A., and F. P. Fischer. A generalized
*k*nearest neighbor rule.*Information and Control*16: 128–152. 1970.MathSciNetzbMATHCrossRefGoogle Scholar - 21.Sanchez, J. S., F. Pla and F. J. Ferri. Prototype selection for the nearest neighbour rule through proximity graphs.
*Pattern Recognition Letters*18(6): 507–513. 1995.CrossRefGoogle Scholar - 22.Devi, V. Susheela and M. Narasimha Murty. An incremental prototype set building technique.
*Pattern Recognition*35: 505–513. 2002.zbMATHCrossRefGoogle Scholar - 23.Swonger, C.W. Sample set condensation for a condensed nearest neighbor decision rule for pattern recognition.
*Frontiers of Pattern Recognition*. 511– 519. 1972.Google Scholar - 24.Tomek, I. A generalization of the
*k*NNrule.*IEEE Trans. on SMC*SMC-6(2): 121–126. 1976.MathSciNetGoogle Scholar - 25.Tomek, I. An experiment with the edited nearest neighbour rule.
*IEEE Trans. on SMC*SMC-6(6): 448–452. 1976.MathSciNetGoogle Scholar - 26.Lam, Wai, Chi-Kin Keung and Charles X. Ling. Learning good prototypes for classification using filtering and abstraction of instances.
*Pattern Recognition*35: 1491–1506. 2002.zbMATHCrossRefGoogle Scholar - 27.Lam, Wai, Chi-Kin Keung and Danyu Liu. Discovering useful concept prototypes for classification based on filtering and abstraction.
*IEEE Trans PAMI*24(8): 1075–1090. 2002.CrossRefGoogle Scholar - 28.Wilson, D. L. Asymptotic properties of nearest neighbour rules using edited data.
*IEEE Trans. SMC*SMC-2(3): 408–421. 1972.Google Scholar - 29.Yunck, Thomas P. A technique to identify nearest neighbours.
*IEEE Trans. SMC*SMC-6(10): 678–683. 1976.MathSciNetGoogle Scholar - 30.Zhang, Bin and Sargur N. Srihari. Fast
*k*nearest neighbour classification using cluster-based trees.*IEEE Trans. on Pattern Analysis and Machine Intelligence*26(4): 525–528. 2004.CrossRefGoogle Scholar