Adaptive k-Nearest-Neighbor Classification Using a Dynamic Number of Nearest Neighbors

  • Stefanos Ougiaroglou
  • Alexandros Nanopoulos
  • Apostolos N. Papadopoulos
  • Yannis Manolopoulos
  • Tatjana Welzer-Druzovec
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4690)


Classification based on k-nearest neighbors (kNN classification) is one of the most widely used classification methods. The number k of nearest neighbors used for achieving a high accuracy in classification is given in advance and is highly dependent on the data set used. If the size of data set is large, the sequential or binary search of NNs is inapplicable due to the increased computational costs. Therefore, indexing schemes are frequently used to speed-up the classification process. If the required number of nearest neighbors is high, the use of an index may not be adequate to achieve high performance. In this paper, we demonstrate that the execution of the nearest neighbor search algorithm can be interrupted if some criteria are satisfied. This way, a decision can be made without the computation of all k nearest neighbors of a new object. Three different heuristics are studied towards enhancing the nearest neighbor algorithm with an early-break capability. These heuristics aim at: (i) reducing computation and I/O costs as much as possible, and (ii) maintaining classification accuracy at a high level. Experimental results based on real-life data sets illustrate the applicability of the proposed method in achieving better performance than existing methods.


kNN classification multidimensional data performance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aha, D.W.: Editorial. Artificial Intelligence Review (Special Issue on Lazy Learning) 11(1-5), 1–6 (1997)Google Scholar
  2. 2.
    Atkeson, C., Moore, A., Schaal, S.: Locally weighted learning. Artificial Intelligence Review 11(1-5), 11–73 (1997)CrossRefGoogle Scholar
  3. 3.
    Atkeson, C., Schaal, S.: Memory-based neural networks for robot learning. Neurocomputing 9, 243–269 (1995)CrossRefGoogle Scholar
  4. 4.
    Beckmann, N., Kriegel, H.-P., Schneider, R., Seeger, B.: The r*-tree: An efficient and robust access method for points and rectangles. In: Proceedings of the ACM SIGMOD Conference, pp. 590–601. ACM Press, New York (1990)Google Scholar
  5. 5.
    Boehm, C., Krebs, F.: The k-nearest neighbour join: Turbo charging the kdd process. Knowledge and Information Systems 6(6), 728–749 (2004)CrossRefGoogle Scholar
  6. 6.
    Cheung, K.L., Fu, A.: Enhanced nearest neighbour search on the r-tree. ACM SIGMOD Record 27(3), 16–21 (1998)CrossRefGoogle Scholar
  7. 7.
    Dasarathy, B.V.: Nearest Neighbor Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  8. 8.
    Frey, P.W., Slate, D.J.: Letter recognition using holland-style adaptive classifiers. Machine Learning 6(2), 161–182 (1991)Google Scholar
  9. 9.
    Guttman, A.: R-trees: A dynamic index structure for special searching. In: Proceedings of the ACM SIGMOD Conference, pp. 47–57. ACM Press, New York (1984)Google Scholar
  10. 10.
    Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaufmann, San Francisco (2000)Google Scholar
  11. 11.
    Hjaltason, G.R., Samet, H.: Distance browsing in spatial databases. ACM Transactions on Database Systems 24(2), 265–318 (1999)CrossRefGoogle Scholar
  12. 12.
    James, M.: Classification Algorithms. John Wiley & Sons, Chichester (1985)zbMATHGoogle Scholar
  13. 13.
    Rousopoulos, N., Kelley, S., Vincent, F.: Nearest neigbor queries. In: Proceedings of the ACM SIGMOD Conference, pp. 71–79. ACM Press, New York (1995)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Stefanos Ougiaroglou
    • 1
  • Alexandros Nanopoulos
    • 1
  • Apostolos N. Papadopoulos
    • 1
  • Yannis Manolopoulos
    • 1
  • Tatjana Welzer-Druzovec
    • 2
  1. 1.Department of Informatics, Aristotle University, Thessaloniki 54124Greece
  2. 2.Faculty of Electrical Eng. and Computer Science, University of MariborSlovenia

Personalised recommendations