Skip to main content
SpringerLink
Log in

Effective naive Bayes nearest neighbor based image classification on GPU

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Non-parametric classifier, Naive Bayes nearest neighbor, is designed with no training phase, and its performance outperforms many well-trained learning-based image classifiers. Unfortunately, despite its high accuracy, it suffers from great computational pressure from distance computations in space of local feature. This paper explores accelerating strategies from perspectives of both algorithm design and software development. Our approach integrates space decomposition capability of Product quantization (PQ) and parallel accelerating capability of underlying computational platform, Graphics processing unit (GPU). PQ is exploited to compress the indexed local features and prune the search space. GPU is used to ease most of computational pressure by processing the tasks in parallel. To achieve good parallel efficiency, a new sequential classification process is first designed and decomposed into independent components with high parallelism. Effective parallelization techniques are then presented to make use of computational resources. Parallel heap array is built to accelerate the process of feature quantization. Distance table lookup is built to speed up the process of feature search. Comparative experiments on UIUC-Sport dataset demonstrate that our integrated solution outperforms other implementations significantly on Core-quad Intel Core i7 950 CPU and GPU of NVIDIA Geforce GTX460. Scalability experiment on 80 million tiny images database shows that our approach still performs well when large-scale image database is explored.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Behmo R, Marcombes P, Dalalyan A, Prinet V (2010) Towards optimal naive bayes nearest neighbor. In: 11th European conference on computer vision. Springer, Heraklion, pp 171–184

  2. Belloch JA, Gonzalez A, Martnez-Zaldvar FJ, Vidal AM (2011) Real-time massive convolution for audio applications on GPU. J Supercomput 58(3):449–457

    Article  Google Scholar 

  3. Boiman O, Shechtman E, Irani M (2008) In defense of nearest-neighbor based image classification. In: 21th IEEE international conference on computer vision and pattern recognition. IEEE, Anchorage, pp 1–8

  4. Bosch A, Zisserman A, Munoz X (2007) Representing shape with a spatial pyramid kernel. In: 6th ACM international conference on image and video retrieval. ACM, New York, pp 401–408

  5. Cornelis N, Van Gool L (2008) Fast scale invariant feature detection and matching on programmable graphics hardware. In: 21th IEEE international conference on computer vision and pattern recognition workshops. IEEE, Anchorage, pp 1–8

  6. Corporation N (2012) CUDA C programming guide. http://docs.nvidia.com/cuda/cuda-c-programming-guide/.

  7. Deng J, Berg A, Li K, Fei-Fei L (2010) What does classifying more than 10,000 image categories tell us? In: 11th European conference on computer vision. Springer, Heraklion, pp 71–84

  8. Diao M, Kim J (2009) Multimedia mining on manycore architectures: the case for GPUs. In: 5th International symposium on advances in visual computing. Springer, Las Vegas, pp 619–630

  9. Diao M, Nicopoulos C, Kim J (2011) Large-scale semantic concept detection on manycore platforms for multimedia mining. In: 25th IEEE international symposium on parallel and distributed processing. IEEE, Anchorage, pp 384–394

  10. Diao M, Nicopoulos C, Kim J (2012) MRF satellite image classification on GPU. In: 41th International conference on parallel processing workshops. IEEE, Pittsburgh, pp 149–156

  11. Enovar R, Risojevi V, Babi Z, Dobravec T, Buli P (2013) A GPU implementation of a structural-similarity-based aerial-image classification. J Supercomput 65(2):978–996

    Google Scholar 

  12. Fang J, Varbanescu A, Sips H (2011) A comprehensive performance comparison of cuda and opencl. In: 40th IEEE international conference on parallel processing. IEEE, Taipei City, pp 216–225

  13. Fatone L, Giacinti M, Mariani F, Recchioni M, Zirilli F (2012) Parallel option pricing on GPU: barrier options and realized variance options. J Supercomput 62(3):1480–1501

    Article  Google Scholar 

  14. Fei-Fei L, Fergus R, Perona P (2007) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. Comput Vis Image Underst 106(1):59–70

    Article  Google Scholar 

  15. Garcia V, Debreuve E, Barlaud M (2008) Fast k nearest neighbor search using GPU. In: 21th International conference on computer vision and pattern recognition workshops. IEEE, Anchorage, pp 1–6

  16. Garcia V, Debreuve F, Nielsen E, Barlaud M (2010) K-nearest neighbor search: fast GPU-based implementations and application to high-dimensional feature matching. In: 18th IEEE international conference on image processing. IEEE, Hong Kong, pp 3757–3760

  17. Intel (2009) Intel math Kernel library. Users Guide 314774–009US

  18. Jegou H, Douze M, Schmid C (2011) Product quantization for nearest neighbor search. IEEE Trans Pattern Anal Mach Intell 33(1):117–128

    Article  Google Scholar 

  19. Kato K, Hosino T (2012) Multi-GPU algorithm for k-nearest neighbor problem. Concurr Comp Pract E 24(1):45–53

    Article  Google Scholar 

  20. Li LJ, Fei-Fei L (2007) Evaluating bag-of-visual-words representations in scene classification. In: 11th IEEE international conference on computer vision. IEEE, Rio de Janeiro, pp 1–8

  21. Li LJ, Socher R, Fei-Fei L (2009) Towards total scene understanding: classification, annotation and segmentation in an automatic framework. In: 22th IEEE international conference on computer vision and pattern recognition. IEEE, Miami, pp 2036–2043

  22. Liu LK, Liu Q, Natsev A, Ross K, Smith J, Varbanescu A (2007) Digital media indexing on the cell processor. In: 8th IEEE international conference on multimedia and expo. IEEE, Beijing, pp 1866–1869

  23. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  24. McCann S, Lowe D (2012) Local naive bayes nearest neighbor for image classification. In: 25th IEEE international conference on computer vision and pattern recognition. IEEE, Providence, RI, pp 3650–3656

  25. Miao Q, Chen Y, Li J, Zhang Q, Zhang Y, Chen G (2009) Parallelization and optimization of a CBVIR system on multi-core architectures. In: 23th IEEE international symposium on parallel and distributed processing. IEEE, Rome, pp 1–8

  26. Noumsi A, Derrien S, Quinton P (2006) Acceleration of a content-based image-retrieval application on the RDISK cluster. In: 20th IEEE international symposium on parallel and distributed processing. IEEE, Rhodes Island, pp 25–29

  27. Owens JD, Luebke D, Govindaraju N, Harris M, Krger J, Lefohn A, Purcell TJ (2007) A survey of general-purpose computation on graphics hardware. Comput Graph Forum 26(1):80–113

    Article  Google Scholar 

  28. Schellmann M, Gorlatch S, Meilnder D, Ksters T, Schfers K, Wbbeling F, Burger M (2011) Parallel medical image reconstruction: from graphics processing units. J Supercomput 57(2):151–160

    Article  Google Scholar 

  29. Sinha SN, Michael Frahm J, Pollefeys M, Genc Y, (2006) GPU-based video feature tracking and matching. In: Technical report, in workshop on edge computing using new commodity architectures

  30. Terboven C, Deselaers T, Bischof C, Ney H (2006) Shared-memory parallelization for content-based image retrieval. In: 9th European conference on computer vision workshop on computation intensive methods for computer vision, Springer, Graz

  31. Thibault J, Senocak I (2012) Accelerating incompressible flow computations with a P threads-CUDA implementation on small-footprint multi-GPU platforms. J Supercomput 59(2):693–719

    Article  Google Scholar 

  32. Torralba A, Fergus R, Freeman W (2008) 80 million tiny images: a large data set for nonparametric object and scene recognition. IEEE Trans Pattern Anal Mach Intell 30(11):1958–1970

    Article  Google Scholar 

  33. Tsuchiyama R, Nakamura T, Iizuka T, Asahara A, Son J, Miki S (2012) The OpenCL programming book. Fixstars

  34. Tuytelaars T, Fritz M, Saenko K, Darrell T (2011) The NBNN kernel. In: 15th IEEE international conference on computer vision. IEEE, Barcelona, pp 1824–1831

  35. Van de Sande KEA, Gevers T, Snoek CGM (2011) Empowering visual categorization with the GPU. IEEE Trans Multimedia 13(1):60–70

    Article  Google Scholar 

  36. Varma M, Ray D (2007) Visual place categorization: problem, dataset, and algorithm. In: 11th IEEE international conference on computer vision. IEEE, Rio de Janeiro, pp 1–8

  37. Wang HL, Cheong LF (2006) Affective understanding in film. IEEE Trans Circ Syst Video Technol 16(6):689–704

    Article  Google Scholar 

  38. Wu J, Christensen HI, Rehg JM (2009) Visual place categorization: problem, dataset, and algorithm. In: 14th IEEE international conference on intelligent robots and systems. IEEE, St. Louis, pp 4763–4770

  39. Xiao J, Antonio S, Fu Y, Lu Y, Tian Q (2009) Refining image retrieval using one-class classification. In: 10th IEEE international conference on multimedia and expo. IEEE, New York, pp 314–317

Download references

Acknowledgments

The authors thank the anonymous reviewers for their constructive and helpful suggestions. This work is supported by National Natural Science Foundation of China (Grant No. 61133008) and the National High Technology Research and Development Program of China, under Grant 2012AA01A306.

Author information

Authors and Affiliations

  1. Services Computing Technology and System Lab, Cluster and Grid Computing Lab, School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

    Lei Zhu, Hai Jin, Ran Zheng & Xiaowen Feng

Authors
  1. Lei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ran Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowen Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai Jin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhu, L., Jin, H., Zheng, R. et al. Effective naive Bayes nearest neighbor based image classification on GPU. J Supercomput 68, 820–848 (2014). https://doi.org/10.1007/s11227-013-1068-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-1068-7

Keywords

Search

Navigation