Skip to main content
SpringerLink
Log in

Exploiting computer resources for fast nearest neighbor classification

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Modern computers provide excellent opportunities for performing fast computations. They are equipped with powerful microprocessors and large memories. However, programs are not necessarily able to exploit those computer resources effectively. In this paper, we present the way in which we have implemented a nearest neighbor classification. We show how performance can be improved by exploiting the ability of superscalar processors to issue multiple instructions per cycle and by using the memory hierarchy adequately. This is accomplished by the use of floating-point arithmetic which usually outperforms integer arithmetic, and block (tiled) algorithms which exploit the data locality of programs allowing for an efficient use of the data stored in the cache memory. Our results are validated with both an analytical model and empirical results. We show that regular codes could be performed faster than more complex irregular codes using standard data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. RISC stands for Reduced Instruction Set Computer

References

  1. Michie D, Spiegelhalter DJ, Taylor CC (1994) (eds) Machine learning, neural and statistical classification. Ellis Horwood, Chichester

  2. Shakhnarovich G, Indyk P, Darrell T (2006) (eds) Nearest neighbor methods in learning and vision. Theory and practice. MIT Press, New York

  3. Kevin B, Jonathan G, Raghu R, Uri S (1999) When is “nearest neighbour” meaningful? In: Catriel Beeri, Peter Buneman (eds) In: Proceedings of 7th international conference data theory, ICDT, number 1540 in lecture notes in computer science, LNCS. Springer, Heidelberg, 10–12 January 1999, pp 217–235

  4. Mahamud S, Hebert M (2003) Minimum risk distance measure for object recognition. In: International conference on computer vision, pp 242–248

  5. Pekalska E, Duin RPW, Gunter S, Bunke H (2004) On not making dissimilarities euclidean. In: Joint IAPR international workshops on statistical and structural pattern recognition, pp 1143–1151.

  6. Vassilis Athitsos, Jonathan Alon, Stan Sclaroff (2005) Efficient nearest neighbor classification using a cascade of approximate similarity measures. In: IEEE computer vision and pattern recognition (CVPR), IEEE Computer Society, pp I: 486–493.

  7. Kilian Q. Weinberger, John Blitzer, Lawrence K. Saul (2005) Distance metric learning for large margin nearest neighbor classification. In: Neural information processing systems, (NIPS)

  8. Navarro JJ, Juan A, Lang T (1994) MOB forms: a class of multilevel block algorithms for dense linear algebra computations. In: ACM International conference supercomputing, pp. 354–363

  9. Fu J, Huang TS (1984) VLSI for pattern recognition and image processing. Springer, Berlin

    Google Scholar 

  10. Bandyopadhyay S, Maulik U (2002) Efficient prototype reordering in nearest neighbor classification. Patt Recognit 35(12):2791–2799

    Article  MATH  Google Scholar 

  11. Dasarathy BV (1991) Nearest Neighbor (NN) norms: NN pattern classification techniques, IEEE Computer Society Press, Washington

  12. Grother PJ, Candela GT, Blue JL (1997) Fast implementation of nearest neighbor classifiers. Patt Recognit 30(3):459–465

    Article  Google Scholar 

  13. Van Der Heiden R, Groen FCA (1997) The box–cox metric for nearest neighbor classification improvement. Patt Recognit 30(2):273–279

    Article  Google Scholar 

  14. Kudo M, Masuyamaa N, Toyamaa J, Shimbob M (2003) Simple termination conditions for k-nearest neighbor method. Patt Recognit Lett 24(9–10):1203–1213

    Article  MATH  Google Scholar 

  15. Lee EW, Chae SI (1998) Fast design of reduced-complexity nearest-neighbor classifiers using triangular inequality. IEEE Trans Patt Anal Mach Intell 20(5):562–566

    Article  MathSciNet  Google Scholar 

  16. Cheng-Lin Liu, Sako H, Fujisawa H (2002) Performance evaluation of pattern classifiers for handwritten character recognition. Int J Doc Anal Recog 4:191–204

    Article  Google Scholar 

  17. Ricci F, Avesani P (1999) Data compression and local metrics for nearest neighbor classification. IEEE Trans Patt Anal Mach Intell 21(4):380–384

    Article  Google Scholar 

  18. Djouadi A, Bouktache E (1997) A fast algorithm for the nearest-neighbor classifier. IEEE Trans Patt Anal Mach Intell 19(3):277–282

    Article  Google Scholar 

  19. Harnamoto Y, Uchimura S, Tornita S (1997) A bootstrap technique for nearest neighbor classifier design. IEEE Trans Patt Anal Mach Intell 19(1):73–79

    Article  Google Scholar 

  20. Pekalska E, Harol A, Lai C, Duin RPW (2005) Pairwise selection of features and prototypes. In: International conference on computer recognition systems, Springer, Heidelberg, pp 271–278

  21. Decaestecker C (1997) Finding prototypes for nearest neighbor classification by means of gradient descent and deterministic annealing. Patt Recognit 30(2):281–288

    Article  Google Scholar 

  22. Chi Z, Wu J, Yan H (1995) Handwritten numeral recognition using self-organizing maps and fuzzy rules. Patt Recognit 28(1):59–66

    Article  Google Scholar 

  23. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  24. Baglietto P, Maresca M, Migliardi M (1996) Image processing on high-performance risc systems. In: Proc IEEE, 84(7):917–930

  25. Anderson E, Dongarra J (1992) LAPACK user’s guide. SIAM, Philadelphia

    Google Scholar 

  26. Hewlett Packard (1994) PA-RISC 1.1 Architecture and instruction set reference manual

  27. Digital Equip. Corp. (1994) DECchip 21064 and DECchip 21064A Alpha AXP microprocessors—hardware ref. manual

  28. SUN Microsystems (2003) UltraSPARC IIIi processor user’s manual, June

  29. Grother PJ, Candela GT (1993) Comparison of handprinted digit classifiers. Technical report NISTR 5209, National Institute of Standards and Technology (NIST)

  30. Lam M (1999) Software pipelining: an effective technique for VLIW machines. In: Proceedings of the SIGPLAN’88, pp 318–3281999

  31. John LH, David AP (1996) Computer architecture a quantitative approach, 2nd edn. Morgan Kaufmann, San Francisco

  32. Lam MS, Rothberg EE, Wolf ME (1991) The cache performance and optimizations of blocked algorithms. In: Proceedings of ASPLOS’91, pp 67–74

  33. Newman DJ, Hettich S, Blake CL, Merz CJ (1998) UCI repository of machine learning databases

  34. Chih-Chung C, Chih-Jen L (2001) LIBSVM: a library for support vector machines. Available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm

Download references

Acknowledgements

This work was supported by the Comissionat per a Universitats i Recerca of the Generalitat de Catalunya (BE94/Annex 3-642 and BE94-A1/110) and the Ministerio de Educación y Ciencia of Spain (TIN2004-07739-C02-01).

Author information

Authors and Affiliations

  1. Computer Architecture Department, Universitat Politècnica de Catalunya, Jordi Girona 1-3, Mòdul D6, 08034, Barcelona, Spain

    José R. Herrero & Juan J. Navarro

Authors
  1. José R. Herrero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan J. Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José R. Herrero.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Herrero, J.R., Navarro, J.J. Exploiting computer resources for fast nearest neighbor classification. Pattern Anal Applic 10, 265–275 (2007). https://doi.org/10.1007/s10044-007-0065-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-007-0065-y

Keywords

Search

Navigation