The ‘kernel approach’ has attracted great attention with the development of support vector machine (SVM) and has been studied in a general way. It offers an alternative solution to increase the computational power of linear learning machines by mapping data into a high dimensional feature space. This ‘approach’ is extended to the well-known nearest-neighbor algorithm in this paper. It can be realized by substitution of a kernel distance metric for the original one in Hilbert space, and the corresponding algorithm is called kernel nearest-neighbor algorithm. Three data sets, an artificial data set, BUPA liver disorders database and USPS database, were used for testing. Kernel nearest-neighbor algorithm was compared with conventional nearest-neighbor algorithm and SVM Experiments show that kernel nearest-neighbor algorithm is more powerful than conventional nearest-neighbor algorithm, and it can compete with SVM.
Unable to display preview. Download preview PDF.
- 1.Duda, R. O. and Hart, P. E.: Pattern Classi¢cation and Scene Analysis, Wiley, New York, 1973.Google Scholar
- 5.Aizerman, M. A., Braverman, E.M. and Rozonoer, L. I.: The Robbince-Monroe process and the method of potential functions, Automat. Remote Contr. 28 (1965), 1882–1885.Google Scholar
- 8.Courant, R. and Hilbert, D.: Methods of Mathematical Physics, J. Wiley, New York, 1953.Google Scholar
- 9.Forsyth, R. S.: UCI Repository of machine learning databases, Irvine, CA: University of California, Department of Information and Computer Science, 1990.Google Scholar
- 10.LeCun, Y. et al.: Backpropagation applied to handwritten zip code recognition, Neural Comput. 1 (1989), 541–551.Google Scholar
- 11.Collobert, R. and Bengio, S.: Support Vector Machines for Large-Scale Regression Problems, IDIAP-RR–00–17, 2000.Google Scholar
- 12.Schölkopf, B., Burges, C. and Vapnik, V.: Extracting support data for a given task, In: U. M. Fayyad. and R. Uthurusamy (eds), Proc. 1st International Conference on Knowledge Discovery & Data Mining, Menlo Park, AAAI Press, 1995.Google Scholar