Image Retrieval Using Random Forest-Based Semantic Similarity Measures and SURF-Based Visual Words

  • Anindita Mukherjee
  • Jaya Sil
  • Ananda S. Chowdhury
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 703)


In this paper, we propose a novel image retrieval scheme using random forest-based semantic similarity measures and SURF-based bag of visual words. A patch-based representation for the images is carried out with SURF-based bag of visual words. A random forest, which is an ensemble of randomized decision trees, is applied next on a set of training images. The training images accumulate into different leaf nodes in each decision tree of the random forest as a result. During retrieval, a query image, represented using SURF-based bag of visual words, is passed through each decision tree. We define a query path and a semantic neighbor set for such query images in all the decision trees. Different measures of semantic image similarity are derived by exploring the characteristics of query paths and semantic neighbor sets. Experimental results on the publicly available COIL-100 image database clearly demonstrate the superior performance of the proposed content-based image retrieval (CBIR) method with these new measures over some of the similar existing approaches.


Semantic similarity measures Random forest Query path SURF Visual words 


  1. 1.
    Datta, R., Joshi, D., Li, J., Wang, James Z., Image retrieval: Ideas, influences, and trends of the new age, ACM Computing Surveys, 40(2), 1–60, (2008).Google Scholar
  2. 2.
    Sivic, J., Zisserman, A.: Video Google: Efficient Visual Search of Videos, In Toward Category-Level Object Recognition, 127–144, (2006).Google Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L.: Speeded-up robust features (SURF), Computer Vision and Image Understanding, 110(3), 346–359, (2008).Google Scholar
  4. 4.
    Lowe D. G.: Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, 60(2), 91–110, (2004).Google Scholar
  5. 5.
    Bouachir, W., Kardouchi, M., Belacel, N.: Improving Bag of Visual Words Image Retrieval: A Fuzzy Weighting Scheme for Efficient Indexation, Proc. SITIS, 215–220, (2009).Google Scholar
  6. 6.
    Mukherjee, A., Chakraborty, S., Sil, J., Chowdhury, A.S.: A Novel Visual Word Assignment Model for Content Based Image Retrieval, Proc. CVIP, Balasubramanian Raman et al. (eds.), Springer AISC, Vol. 459, 79–87, (2016).Google Scholar
  7. 7.
    Rahman, M.M., Bhattacharya, P., Kamel, M., Campilho A.: Probabilistic Similarity Measures in Image Databases with SVM Based Categorization and Relevance Feedback, Proc. ICIAR, Springer LNCS, Vol. 3656, 601–608, (2005).Google Scholar
  8. 8.
    Liu Y., Zhang D., Lu G., Ma W-Y.: A survey of content-based image retrieval with high-level semantics, Pattern Recognition, 40(1), 262–282, (2007).Google Scholar
  9. 9.
    Fu, H., Qiu G.: Fast Semantic Image Retrieval Based on Random Forest, Proc. ACM MM, 909–912, (2012).Google Scholar
  10. 10.
    Moosman, F., Triggs, B. and Jurie, F.: Fast Discriminative Visual Codebooks using Randomized Clustering Forests, Proc. NIPS, 985–992, (2006).Google Scholar
  11. 11.
    Dimitrovski, I., Kocev, D., Loskovska, S., Dzeroski, S.: Improving bag-of-visual-words image retrieval with predictive clustering trees, Information Science, 329(2), 851–865, (2016).Google Scholar
  12. 12.
    Nene, S. A., Nayar, S. K., Murase, H.: Columbia Object Image Library (COIL-100), Tech. Report, Department of Computer Science, Columbia University CUCS-006–96, (1996).Google Scholar
  13. 13.
    Breiman, L.: Random Forests, Machine Learning, 45, 5–32, (2001).Google Scholar
  14. 14.
    Sivic, J., Zisserman A.: Video Google: A Text Retrieval Approach to Object Matching in Videos, Proc. ICCV, 470–1477, (2003).Google Scholar
  15. 15.
    Newsam, S., Yang Y.: Comparing global and interest point descriptors for similarity retrieval in remote sensed imagery, Proc. ACM GIS, Article No. 9, (2007).Google Scholar
  16. 16.
    Kontschieder P., Rota Bulo S., Pelillo M.: Semantic Labeling and Object Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10), 2104–2116, (2014).Google Scholar
  17. 17.
    Wan J. et al.: Deep Learning for Content-Based Image Retrieval: A Comprehensive Study, Proc. ACM MM, 157–166, (2014).Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Anindita Mukherjee
    • 1
  • Jaya Sil
    • 2
  • Ananda S. Chowdhury
    • 3
  1. 1.Dream Institute of TechnologyKolkataIndia
  2. 2.IIEST ShibpurHowrahIndia
  3. 3.Jadavpur UniversityKolkataIndia

Personalised recommendations