k-NN Boosting Prototype Learning for Object Classification

  • Paolo Piro
  • Michel Barlaud
  • Richard Nock
  • Frank Nielsen
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 158)


Image classification is a challenging task in computer vision. For example fully understanding real-world images may involve both scene and object recognition. Many approaches have been proposed to extract meaningful descriptors from images and classifying them in a supervised learning framework. In this chapter, we revisit the classic k-nearest neighbors (k-NN) classification rule, which has shown to be very effective when dealing with local image descriptors. However, k-NN still features some major drawbacks, mainly due to the uniform voting among the nearest prototypes in the feature space. In this chapter, we propose a generalization of the classic k-NN rule in a supervised learning (boosting) framework. Namely, we redefine the voting rule as a strong classifier that linearly combines predictions from the k closest prototypes. In order to induce this classifier, we propose a novel learning algorithm, MLNN (Multiclass Leveraged Nearest Neighbors), which gives a simple procedure for performing prototype selection very efficiently. We tested our method first on object classification using 12 categories of objects, then on scene recognition as well, using 15 real-world categories. Experiments show significant improvement over classic k-NN in terms of classification performances.


Boosting k-NN classification Object recognition Scene categorization 


  1. 1.
    Bartlett P, Jordan M, McAuliffe JD (2006) Convexity, classification, and risk bounds. J Am Stat Assoc 101:138–156MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Bosch A, Zisserman A, Muñoz X (2008) Scene classification using a hybrid generative/discriminative approach. IEEE Trans Pattern Anal Mach Intell 30(4):712–727CrossRefGoogle Scholar
  3. 3.
    Brighton H, Mellish C (2002) Advances in instance selection for instance-based learning algorithms. Data Min Knowl Disc 6:153–172MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Cha SH (2008) Taxonomy of nominal type histogram distance measures. In: Proceedings of the American conference on applied mathematics, pp 325–330Google Scholar
  5. 5.
    Csurka G, Bray C, Dance C, Fan L (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision (ECCV), pp 1–22Google Scholar
  6. 6.
    Davis J, Kulis B, Jain P, Sra S, Dhillon I (2007) Information-theoretic metric learning. In: International conference on machine learning (ICML), pp 209–216Google Scholar
  7. 7.
    Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to Boosting. J Comput Syst Sci 55(1):119–139Google Scholar
  8. 8.
    Garcia V, Debreuve E, Barlaud M (2008) Fast k nearest neighbor search using gpu. In: CVPR workshop on computer vision on GPU (CVGPU), Anchorage, Alaska, USAGoogle Scholar
  9. 9.
    Hastie T, Tibshirani R (1996) Discriminant adaptive nearest neighbor classification. IEEE Trans Pattern Anal Mach Intell 18(6):(6):607–616Google Scholar
  10. 10.
    Jegou H, Douze M, Schmid C (2008) Hamming embedding and weak geometric consistency for large scale image search. In: European conference on computer vision (ECCV), vol I, pp 304–317Google Scholar
  11. 11.
    Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE Conference on computer vision& pattern recognition, pp 3304–3311.
  12. 12.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Computer vision and pattern recognition (CVPR), pp 2169–2178.
  13. 13.
    Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2):91–110. Google Scholar
  14. 14.
    Nock R, Nielsen F (2009) Bregman divergences and surrogates for learning. IEEE Trans. Pattern Anal. Mach. Intell. 31:2048–2059CrossRefGoogle Scholar
  15. 15.
    Perronnin F, Dance C (2007) Fisher kernels on visual vocabularies for image categorization. In: Computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  16. 16.
    Perronnin F, Sánchez J, Liu Y (2010) Large-scale image categorization with explicit data embedding. In: Computer vision and pattern recognition (CVPR), pp 2297–2304Google Scholar
  17. 17.
    Piro P, Nock R, Nielsen F, Barlaud M (2009) Boosting k-NN for categorization of natural scenes. ArXiv:1001.1221Google Scholar
  18. 18.
    Piro P, Nock R, Nielsen F, Barlaud M (2010) Leveraging \(k\)-NN for generic classification boosting. In: Proceedings of the 2010 IEEE international workshop on machine learning for signal Processing (MLSP)Google Scholar
  19. 19.
    Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated predictions. J Mach Learn 37:297–336MATHCrossRefGoogle Scholar
  20. 20.
    Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: Proceedings of the international conference on computer vision, vol 2, pp 1470–1477.
  21. 21.
    Sivic J, Zisserman A (2006) Video google: efficient visual search of videos. pp 127–144.
  22. 22.
    Zou H, Zhu J, Hastie T (2008) New multicategory boosting algorithms based on multicategory fisher-consistent losses. Ann Appl Stat 2(4):1290–1306MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Zhang H, Berg A, Maire M, Malik J (2006) SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: Computer vision and pattern recognition (CVPR’06), pp 2126–2136Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Paolo Piro
    • 1
  • Michel Barlaud
    • 1
  • Richard Nock
    • 2
  • Frank Nielsen
    • 3
  1. 1.CNRS/University of Nice-Sophia AntipolisSophia AntipolisFrance
  2. 2.CEREGMIA/University of Antilles-GuyaneMartiniqueFrance
  3. 3.LIX/Ecole PolytechniquePalaiseauFrance

Personalised recommendations