Keypoints and Codewords Selection for Efficient Bag-of-Features Representation

  • Veerapathirapillai Vinoharan
  • Amirthalingam Ramanan
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 886)


Bag-of-features (BoF) representation is one of the most popular image representations that is used in visual object classification, owing to its simplicity and good performance. However, the BoF representation always faces the difficulty of curse of dimensionality that leads to huge computational cost and increased storage requirement. To create a discriminative and compact BoF representation, it is desired to eliminate ambiguous features before the construction of visual codebook and to select the informative codewords from the constructed codebook. In this paper, we propose a two-staged approach to create a discriminative and compact BoF representation for object recognition. In the first step, we eliminate ambiguous patch-based descriptors using an entropy-based filtering approach to retain high-quality descriptors. In the subsequent step, we select the informative codewords based on statistical measures. We have tested the proposed technique on Xerox7, UIUC texture, PASCAL VOC 2007 and Caltech101 benchmark datasets. Testing results show that more training features and/or a high-dimensional codebook do not contribute significantly to increase the performance of classification but it increases the overall model complexity and computational cost. The proposed preprocessing step of descriptor selection increases the discriminative power of a codebook, whereas the post-processing step of codeword selection maintains the codebook to be more compact. The proposed framework would help to optimise BoF representation to be efficient with steady performance.


Keypoint selection Codebook Codeword selection Image representation Bag-of-features 


  1. 1.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  2. 2.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  3. 3.
    Csurka, C., Dance, R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV 2004, pp. 1–22 (2004)Google Scholar
  4. 4.
    Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using affine-invariant regions. Pattern Anal. Mach. Intell. 27(8), 1265–1278 (2005)CrossRefGoogle Scholar
  5. 5.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 524–531 (2005)Google Scholar
  6. 6.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. Int. J. Comput. Vis. Image Underst. 106(1), 59–70 (2007)CrossRefGoogle Scholar
  7. 7.
    van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Color descriptors for object category recognition. In: Proceedings of the Conference on Colour in Graphics, Imaging, and Vision, vol. 2008(1), pp. 378–381 (2008)Google Scholar
  8. 8.
    Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  9. 9.
    Ramanan, A., Niranjan, M.: A one-pass resource-allocating codebook for patch-based visual object recognition. In: Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing, pp. 35–40 (2010)Google Scholar
  10. 10.
    Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The binormal assumption on precision-recall curves. In: Proceedings of the International Conference on Pattern Recognition, pp. 4263–4266 (2010)Google Scholar
  11. 11.
    Ramanan, A., Niranjan, M.: A review of codebook models in patch-based visual object recognition. J. Signal Process. Syst. 68(3), 333–352 (2012)CrossRefGoogle Scholar
  12. 12.
    Wang, L., Zhou, L., Shen, C., Liu, L., Liu, H.: A hierarchical word-merging algorithm with class separability measure. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 417–435 (2014)CrossRefGoogle Scholar
  13. 13.
    Cui, J., Cui, M., Xiao, B., Li, G.: Compact and discriminative representation of bag-of-features. Neurocomputing 169, 55–67 (2015)CrossRefGoogle Scholar
  14. 14.
    Kirishanthy, T., Ramanan, A.: Creating compact and discriminative visual vocabularies using visual bits. In: Proceedings of the IEEE Digital Image Computing: Techniques and Applications, pp. 258–263 (2015)Google Scholar
  15. 15.
    Wang, C., Huang, K.: How to use bag-of-words model better for image classification. Image Vis. Comput. 38, 65–74 (2015)CrossRefGoogle Scholar
  16. 16.
    Lin, W.C., Tsai, C.F., Chen, Z.Y., Ke, S.W.: Keypoint selection for efficient bag-of-words feature generation and effective image classification. Inf. Sci. 329, 33–51 (2016)CrossRefGoogle Scholar
  17. 17.
    Peng, X., Wang, L., Wang, X., Qiao, Y.: Bag of visual words and fusion methods for action recognition: comprehensive study and good practice. Comput. Vis. Image Underst. 150, 109–125 (2016)CrossRefGoogle Scholar
  18. 18.
    Amato, G., Falchi, F., Gennaro, C.: On reducing the number of visual words in the bag-of-features representation. In: Computing Research Repository, pp. 657–662 (2016)Google Scholar
  19. 19.
    Vinoharan, V., Ramanan, A.: Are large scale training images or discriminative features important for codebook construction? In: Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods, vol. 1, pp. 193–198 (2016)Google Scholar
  20. 20.
    Nasirahmadi, A., Ashtiani, S.H.M.: Bag-of-feature model for sweet and bitter almond classification. Biosyst. Eng. 156, 51–60 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Veerapathirapillai Vinoharan
    • 1
  • Amirthalingam Ramanan
    • 2
  1. 1.Computer CentreUniversity of JaffnaJaffnaSri Lanka
  2. 2.Department of Computer ScienceUniversity of JaffnaJaffnaSri Lanka

Personalised recommendations