Do We Need Whatever More Than k-NN?

  • Mirosław Kordos
  • Marcin Blachnik
  • Dawid Strzempa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6113)


Many sophisticated classification algorithms have been proposed. However, there is no clear methodology of comparing the results among different methods. According to our experiments on the popular datasets, k-NN with properly tuned parameters performs on average best. Tuning the parametres include the proper k, proper distance measure and proper weighing functions. k-NN has a zero training time and the test time can be significantly reduced by prior reference vector selection, which needs to be done only once or by applying advanced nearest neighbor search strategies (like KDtree algorithm). Thus we propose that instead of comparing new algorithms with an author’s choice of old ones (which may be especially selected in favour of his method), the new method would be rather compared first with properly tuned k-NN as a gold standard. And based on the comparison the author of the new method would have to aswer the question: ”Do we really need this method since we already have k-NN?”


Feature Ranking Wisconsin Breast Cancer Aviation Medicine Diabetes Dataset Popular Dataset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fix, E., Hodges, J.L.: Discriminatory analysis, nonparametric discrimination: Consistency properties. USAF School of Aviation Medicine, Randolph Field, Texas (1951)Google Scholar
  2. 2.
  3. 3.
    UCI Machine Learning Repository,
  4. 4.
    Strzempa, D.: Internet System for Data Classification (in Polish), MSc Thesis, The Silesian University of Technology, Katowice (2008),
  5. 5.
    Duda, R.O., et al.: Pattern Classification. Wisley, New York (2001)zbMATHGoogle Scholar
  6. 6.
    Breiman, L., et al.: Classification and Regression Trees. Wadsworth, CA (1984)Google Scholar
  7. 7.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  8. 8.
    Kohonen, T.: Statistical Pattern Recognition Revisited. Elsevier, Amsterdam (1990)Google Scholar
  9. 9.
    Schalkoff, R.: Pattern Recognition: Statistical, Structural and Neural Approaches. Wiley, Chichester (1992)Google Scholar
  10. 10.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Wiley, Chichester (1950)Google Scholar
  11. 11.
    Kordos, M.: Search-based Algorithms for Multilayer Perceptrons, PhD Thesis, The Silesian University of Technology, Gliwice (2005),
  12. 12.
    Grabczewski, K.: Application of SSV Criterion for generating classification rules, PhD Thesis, Nicholaus Copernicus University, Torun (2003) (in Polish)Google Scholar
  13. 13.
    Jankowski, N.: Ontogenic Neural Networks for Medical Data Classification, PhD Thesis, Nicholaus Copernicus University, Torun (1999) (in Polish)Google Scholar
  14. 14.
    Adamczak, R.: Neural networks application for experimental data classification, PhD Thesis, Nicholaus Copernicus University, Torun (1999, 2001) (in Polish)Google Scholar
  15. 15.
    Schlkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2001)Google Scholar
  16. 16.
    Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning, neural and statistical classification. Elis Horwood, London (1994)zbMATHGoogle Scholar
  17. 17.
    Ster, B., Dobnikar, A.: Neural Networks in medical diagnosis: Comparison with other methods. In: EANN 1996, pp. 427–430 (1996)Google Scholar
  18. 18.
    Zarndt, F.: A comprehensive case study: An examination of machine learning and connectionists algorithms, MSc Thesis, Department of Computer Science, Brigham Young University (1995)Google Scholar
  19. 19.
    Weiss, S.M., Kapouleas, I.: An empirical comparison of pattern Recognition, neural nets and machine learning classification methods. In: Reading in Machine Learning. Morgan Kauffman Publ., CA (1990)Google Scholar
  20. 20.
  21. 21.
  22. 22.
    Blachnik, M., Duch, W., Wieczorek, T.: Probabilistic distance measures for prototype-based rules. In: Proc. of the 12th Int. Conference on Neural Information Processing (ICONIP 2005), Taipei, Taiwan, pp. 445–450 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mirosław Kordos
    • 1
  • Marcin Blachnik
    • 2
  • Dawid Strzempa
    • 1
  1. 1.Department of Mathematics and Computer ScienceUniversity of Bielsko-BiałaWillowaPoland
  2. 2.Electrotechnology DepartmentSilesian University of TechnologyKrasinskiegoPoland

Personalised recommendations