Artificial Intelligence Review

, Volume 52, Issue 2, pp 901–925 | Cite as

Scene analysis and search using local features and support vector machine for effective content-based image retrieval

  • Uzma Sharif
  • Zahid MehmoodEmail author
  • Toqeer Mahmood
  • Muhammad Arshad Javid
  • Amjad Rehman
  • Tanzila Saba


Despite broad investigation in content-based image retrieval (CBIR), issue to lessen the semantic gap between high-level semantics and local attributes of the image is still an important issue. The local attributes of an image such as shape, color, and texture are not sufficient for effective CBIR. Visual similarity is a principal step in CBIR and in the baseline approach. In this article, we introduce a novel approach, which relies on the fusion of visual words of scale-invariant feature transform (SIFT) and binary robust invariant scalable keypoints (BRISK) descriptors based on the visual-bag-of-words approach. The two local feature descriptors are chosen as their fusion adds complementary improvement to CBIR. The SIFT descriptor is capable of detecting objects robustly under cluttering due to its invariance to scale, rotation, noise, and illumination variance. However, SIFT descriptor does not perform well at low illumination or poorly localized keypoints within an image. Due to this reason, the discriminative power of the SIFT descriptor is lost during the quantization process, which also reduces the performance of CBIR. However, the BRISK descriptor provides scale and rotation-invariant scale-space, high quality and adaptive performance in classification based applications. It also performs better at poorly localized keypoints along the edges of an object within an image as compared to the SIFT descriptor. The suggested approach based on the fusion of visual words achieves effective results on the Corel-1K, Corel-1.5K, Corel-5K, and Caltech-256 image repositories as equated to the feature fusion of both descriptors and latest CBIR approaches with the surplus assistances of scalability and fast indexing.


Visual retrieval Image matching Complementary features Clustering Image attributes 



All the sources are acknowledged properly, where applicable.

Authors’ contributions

All the authors contributed equally.


This work was partially supported by the Machine Learning Research Group; Prince Sultan University Riyadh; Saudi Arabia [RG-CCIS-2017-06-02]; The authors are grateful for this financial support.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Ali N, Bajwa KB, Sablatnig R, Mehmood Z (2016) Image retrieval by addition of spatial information based on histograms of triangular regions. Comput Electr Eng 54:539–550CrossRefGoogle Scholar
  2. Arampatzis A, Zagoris K, Chatzichristofis SA (2013) Dynamic two-stage image retrieval from large multimedia databases. Inf Process Manag 49(1):274–285CrossRefGoogle Scholar
  3. Arthur D, Vassilvitskii S (2007) k-means++: the advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms. Society for Industrial and Applied MathematicsGoogle Scholar
  4. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359CrossRefGoogle Scholar
  5. Chen Y, Wang JZ, Krovetz R (2005) CLUE: cluster-based retrieval of images by unsupervised learning. IEEE Trans Image Process 14(8):1187–1201CrossRefGoogle Scholar
  6. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5CrossRefGoogle Scholar
  7. Deb K, Padhye N (2014) Enhancing performance of particle swarm optimization through an algorithmic link with genetic algorithms. Comput Optim Appl 57(3):761–794MathSciNetzbMATHCrossRefGoogle Scholar
  8. Dinesh KR, Hariharan S (2014) Approaches and trends in content based image retrieval. In: Proceedings of 24th international conference in communication, network and computing, CNC. Elsevier, pp 473–477Google Scholar
  9. Dubey SR, Singh SK, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Electric Eng 46:288–302CrossRefGoogle Scholar
  10. ElAlami ME (2014) A new matching strategy for content based image retrieval system. Appl Soft Comput 14:407–418CrossRefGoogle Scholar
  11. Iakovidou C, Anagnostopoulos N, Kapoutsis AC, Boutalis Y, Chatzichristofis SA (2014) Searching images with MPEG-7 (& mpeg-7-like) powered localized descriptors: the SIMPLE answer to effective content based image retrieval. In: 12th International workshop on content-based multimedia indexing (CBMI). IEEEGoogle Scholar
  12. Krajník T, Cristóforis P, Nitsche M, Kusumam K, Duckett T (2015) Image features and seasons revisited. In: European conference on mobile robots (ECMR). IEEEGoogle Scholar
  13. Leutenegger S, Chli M, Siegwart RY (2011) BRISK: binary robust invariant scalable keypoints. In: IEEE international conference on computer vision (ICCV). IEEEGoogle Scholar
  14. Li J, Wang JZ (2003) Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans Pattern Anal Mach Intell 25(9):1075–1088CrossRefGoogle Scholar
  15. Liu G-H, Yang J-Y (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198CrossRefGoogle Scholar
  16. Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282zbMATHCrossRefGoogle Scholar
  17. Liu G-H, Zhang L, Hou Y-K, Li Z-Y, Yang J-Y (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389zbMATHCrossRefGoogle Scholar
  18. Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference. IEEEGoogle Scholar
  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110Google Scholar
  20. Mehmood Z, Anwar SM, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016(2016):1–12CrossRefGoogle Scholar
  21. Mehmood Z, Anwar SM, Altaf M (2018) A novel image retrieval based on rectangular spatial histograms of visual words. Kuwait J Sci 45(1):54–69MathSciNetGoogle Scholar
  22. Mehmood Z, Mahmood T, Javid MA (2018) Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine. Appl Intell 48(1):166–181CrossRefGoogle Scholar
  23. Mukherjee D, Wu QJ, Wang G (2015) A comparative experimental study of image feature detectors and descriptors. Mach Vis Appl 26(4):443–466CrossRefGoogle Scholar
  24. Padhye N, Bhardawaj P, Deb K (2013) Improving differential evolution through a unified approach. J Glob Optim 55(4):771–799MathSciNetzbMATHCrossRefGoogle Scholar
  25. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238CrossRefGoogle Scholar
  26. Raja NMK, Bhanu KS (2013) Content bases image search and retrieval using indexing by k means clustering technique. Int J Adv Res Comput Commun Eng 2(5):2181–2189Google Scholar
  27. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision (ICCV). IEEEGoogle Scholar
  28. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  29. Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of EODH and Color-SIFT: application to image retrieval based on codebook. Sig Process Image Commun 29(4):530–545CrossRefGoogle Scholar
  30. Tousch A-M, Herbin S, Audibert J-Y (2012) Semantic hierarchies for image annotation: a survey. Pattern Recogn 45(1):333–345CrossRefGoogle Scholar
  31. Tzelepi M, Tefas A (2018) Deep convolutional learning for content based image retrieval. Neurocomputing 275:2467–2478CrossRefGoogle Scholar
  32. Uricchio T, Ballan L, Seidenari L, Del Bimbo A (2017) Automatic image annotation via label transfer in the semantic space. Pattern Recogn 71:144–157CrossRefGoogle Scholar
  33. Yang X, Cai L (2013) Adaptive region matching for region-based image retrieval by constructing region importance index. IET Comput Vis 8(2):141–151CrossRefGoogle Scholar
  34. Yu J, Qin Z, Wan T, Zhang X (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364CrossRefGoogle Scholar
  35. Yuan X, Yu J, Qin Z, Wan T (2011) A SIFT-LBP image retrieval model based on bag of features. In: IEEE international conference on image processingGoogle Scholar
  36. Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by Gaussian mixture models. Neurocomputing 171:673–684CrossRefGoogle Scholar
  37. Zhang D, Islam MM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362CrossRefGoogle Scholar
  38. Zhong D, Defée I (2005) DCT histogram optimization for image database retrieval. Pattern Recogn Lett 26(14):2272–2281CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  • Uzma Sharif
    • 1
  • Zahid Mehmood
    • 1
    Email author
  • Toqeer Mahmood
    • 2
  • Muhammad Arshad Javid
    • 3
  • Amjad Rehman
    • 4
  • Tanzila Saba
    • 5
  1. 1.Department of Software EngineeringUniversity of Engineering and TechnologyTaxilaPakistan
  2. 2.Department of Computer ScienceUniversity of Engineering and TechnologyTaxilaPakistan
  3. 3.Department of Basic SciencesUniversity of Engineering and TechnologyTaxilaPakistan
  4. 4.College of Computer and Information SystemsAl-Yamamah UniversityRiyadhSaudi Arabia
  5. 5.College of Computer and Information SciencesPrince Sultan UniversityRiyadhSaudi Arabia

Personalised recommendations