Advertisement

A Nearest Neighbor Method Using Bisectors

  • Mineichi Kudo
  • Hideyuki Imai
  • Akira Tanaka
  • Tetsuya Murai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)

Abstract

A novel algorithm for finding the nearest neighbor was proposed. According to the development of modern technology, the demand is increasing in large-scale datasets with a large number of samples and a large number of features. However, almost all sophisticated algorithms proposed so far are effective only in a small number of features, say, up to 10. This is because in a high-dimensional space many pairs of samples share a same distance. Then the naive algorithm outperforms the others. In this study, we considered to utilize a sequential information of distances obtained by the examined training samples. Indeed, a combinatorial information of examined samples was used as bisectors between possible pairs of them. With this algorithm, a query is processed in O(αβ nd) for n samples in a d-dimensional space and for α,β < 1, in expense of a preprocessing time and space in O(n 2). We examined the performance of the algorithm.

Keywords

Training Sample Distance Calculation Query Point Query Time Ball Test 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Information Theory IT-13(1), 21–27 (1967)zbMATHCrossRefGoogle Scholar
  2. 2.
    Hart, P.E.: The condensed nearest neighbor rule. IEEE Trans. Information Theory IT-14(3), 515–516 (1968)CrossRefGoogle Scholar
  3. 3.
    Gates, G.W.: The reduced nearest neighbor rule. IEEE Trans. Information Theory IT-18(3), 431–433 (1972)CrossRefGoogle Scholar
  4. 4.
    Dasarathy, B.V.: Minimal consistent set(MCS) identification for optimal neighbor decision systems design. IEEE Trans. Systems, Man, & Cybernetics SMC-24(3), 511–517 (1994)CrossRefGoogle Scholar
  5. 5.
    Fukunaga, K., Narendra, P.M.: A branch-and-bound algorithm for computing knearest neighbors. IEEE Trans. Computers C-24(7), 750–753 (1975)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Chang, C.-C., Wu, T.-C.: A hashing-oriented nearest neighbor searching scheme. Pattern Recognition Letters 14, 625–630 (1993)CrossRefGoogle Scholar
  7. 7.
    Fukunaga, K.: Nonparametric Density Estimation, in Introduction to Statistical Pattern Recognition, pp. 268–287. Academic Press, London (1990)Google Scholar
  8. 8.
    Murphy, P.M., Aha, D.W.: UCI Repository of Machine Learning Databases [Machine-Readable Data Repository]. University of California, Department of Information and Computer Science, Irvine, California (1991)Google Scholar
  9. 9.
    Maneewongvatana, S., Mount, D.M.: The Analysis of a Probabilistic Approach to Nearest Neighbor Searching. In: Dehne, F., Sack, J.-R., Tamassia, R. (eds.) WADS 2001. LNCS, vol. 2125, pp. 276–286. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  10. 10.
    Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching. Journal of the ACM 45, 891–923 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Kleinberg, J.: Two algorithms for nearest-neighbor search in high dimensions. In: Proc. 29th ACM Symposium on Theory of Computing, pp. 599–608 (1997)Google Scholar
  12. 12.
    Kudo, M., Masuyama, N., Shimbo, M.: Simple termination conditions for knearest neighbor method. Pattern Recognition Letters 24, 1213–1223 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Mineichi Kudo
    • 1
  • Hideyuki Imai
    • 1
  • Akira Tanaka
    • 1
  • Tetsuya Murai
    • 1
  1. 1.Division of Systems and Information Engineering, Graduate School of EngineeringHokkaido UniversitySapporoJapan

Personalised recommendations