Scene analysis and search using local features and support vector machine for effective content-based image retrieval

Abstract

Despite broad investigation in content-based image retrieval (CBIR), issue to lessen the semantic gap between high-level semantics and local attributes of the image is still an important issue. The local attributes of an image such as shape, color, and texture are not sufficient for effective CBIR. Visual similarity is a principal step in CBIR and in the baseline approach. In this article, we introduce a novel approach, which relies on the fusion of visual words of scale-invariant feature transform (SIFT) and binary robust invariant scalable keypoints (BRISK) descriptors based on the visual-bag-of-words approach. The two local feature descriptors are chosen as their fusion adds complementary improvement to CBIR. The SIFT descriptor is capable of detecting objects robustly under cluttering due to its invariance to scale, rotation, noise, and illumination variance. However, SIFT descriptor does not perform well at low illumination or poorly localized keypoints within an image. Due to this reason, the discriminative power of the SIFT descriptor is lost during the quantization process, which also reduces the performance of CBIR. However, the BRISK descriptor provides scale and rotation-invariant scale-space, high quality and adaptive performance in classification based applications. It also performs better at poorly localized keypoints along the edges of an object within an image as compared to the SIFT descriptor. The suggested approach based on the fusion of visual words achieves effective results on the Corel-1K, Corel-1.5K, Corel-5K, and Caltech-256 image repositories as equated to the feature fusion of both descriptors and latest CBIR approaches with the surplus assistances of scalability and fast indexing.

This is a preview of subscription content, access via your institution.

Fig. 1

(Reproduced with permission from Li and Wang 2003)

Fig. 2

(Reproduced with permission from Li and Wang 2003)

Fig. 3
Fig. 4

(Reproduced with permission from Li and Wang 2003)

Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

(Reproduced with permission from Li and Wang 2003)

Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

References

  1. Ali N, Bajwa KB, Sablatnig R, Mehmood Z (2016) Image retrieval by addition of spatial information based on histograms of triangular regions. Comput Electr Eng 54:539–550

    Article  Google Scholar 

  2. Arampatzis A, Zagoris K, Chatzichristofis SA (2013) Dynamic two-stage image retrieval from large multimedia databases. Inf Process Manag 49(1):274–285

    Article  Google Scholar 

  3. Arthur D, Vassilvitskii S (2007) k-means++: the advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms. Society for Industrial and Applied Mathematics

  4. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  5. Chen Y, Wang JZ, Krovetz R (2005) CLUE: cluster-based retrieval of images by unsupervised learning. IEEE Trans Image Process 14(8):1187–1201

    Article  Google Scholar 

  6. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5

    Article  Google Scholar 

  7. Deb K, Padhye N (2014) Enhancing performance of particle swarm optimization through an algorithmic link with genetic algorithms. Comput Optim Appl 57(3):761–794

    MathSciNet  MATH  Article  Google Scholar 

  8. Dinesh KR, Hariharan S (2014) Approaches and trends in content based image retrieval. In: Proceedings of 24th international conference in communication, network and computing, CNC. Elsevier, pp 473–477

  9. Dubey SR, Singh SK, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Electric Eng 46:288–302

    Article  Google Scholar 

  10. ElAlami ME (2014) A new matching strategy for content based image retrieval system. Appl Soft Comput 14:407–418

    Article  Google Scholar 

  11. Iakovidou C, Anagnostopoulos N, Kapoutsis AC, Boutalis Y, Chatzichristofis SA (2014) Searching images with MPEG-7 (& mpeg-7-like) powered localized descriptors: the SIMPLE answer to effective content based image retrieval. In: 12th International workshop on content-based multimedia indexing (CBMI). IEEE

  12. Krajník T, Cristóforis P, Nitsche M, Kusumam K, Duckett T (2015) Image features and seasons revisited. In: European conference on mobile robots (ECMR). IEEE

  13. Leutenegger S, Chli M, Siegwart RY (2011) BRISK: binary robust invariant scalable keypoints. In: IEEE international conference on computer vision (ICCV). IEEE

  14. Li J, Wang JZ (2003) Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans Pattern Anal Mach Intell 25(9):1075–1088

    Article  Google Scholar 

  15. Liu G-H, Yang J-Y (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198

    Article  Google Scholar 

  16. Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    MATH  Article  Google Scholar 

  17. Liu G-H, Zhang L, Hou Y-K, Li Z-Y, Yang J-Y (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389

    MATH  Article  Google Scholar 

  18. Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference. IEEE

  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  20. Mehmood Z, Anwar SM, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016(2016):1–12

    Article  Google Scholar 

  21. Mehmood Z, Anwar SM, Altaf M (2018) A novel image retrieval based on rectangular spatial histograms of visual words. Kuwait J Sci 45(1):54–69

    MathSciNet  Google Scholar 

  22. Mehmood Z, Mahmood T, Javid MA (2018) Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine. Appl Intell 48(1):166–181

    Article  Google Scholar 

  23. Mukherjee D, Wu QJ, Wang G (2015) A comparative experimental study of image feature detectors and descriptors. Mach Vis Appl 26(4):443–466

    Article  Google Scholar 

  24. Padhye N, Bhardawaj P, Deb K (2013) Improving differential evolution through a unified approach. J Glob Optim 55(4):771–799

    MathSciNet  MATH  Article  Google Scholar 

  25. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  26. Raja NMK, Bhanu KS (2013) Content bases image search and retrieval using indexing by k means clustering technique. Int J Adv Res Comput Commun Eng 2(5):2181–2189

    Google Scholar 

  27. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision (ICCV). IEEE

  28. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Google Scholar 

  29. Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of EODH and Color-SIFT: application to image retrieval based on codebook. Sig Process Image Commun 29(4):530–545

    Article  Google Scholar 

  30. Tousch A-M, Herbin S, Audibert J-Y (2012) Semantic hierarchies for image annotation: a survey. Pattern Recogn 45(1):333–345

    Article  Google Scholar 

  31. Tzelepi M, Tefas A (2018) Deep convolutional learning for content based image retrieval. Neurocomputing 275:2467–2478

    Article  Google Scholar 

  32. Uricchio T, Ballan L, Seidenari L, Del Bimbo A (2017) Automatic image annotation via label transfer in the semantic space. Pattern Recogn 71:144–157

    Article  Google Scholar 

  33. Yang X, Cai L (2013) Adaptive region matching for region-based image retrieval by constructing region importance index. IET Comput Vis 8(2):141–151

    Article  Google Scholar 

  34. Yu J, Qin Z, Wan T, Zhang X (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364

    Article  Google Scholar 

  35. Yuan X, Yu J, Qin Z, Wan T (2011) A SIFT-LBP image retrieval model based on bag of features. In: IEEE international conference on image processing

  36. Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by Gaussian mixture models. Neurocomputing 171:673–684

    Article  Google Scholar 

  37. Zhang D, Islam MM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362

    Article  Google Scholar 

  38. Zhong D, Defée I (2005) DCT histogram optimization for image database retrieval. Pattern Recogn Lett 26(14):2272–2281

    Article  Google Scholar 

Download references

Acknowledgements

All the sources are acknowledged properly, where applicable.

Funding

This work was partially supported by the Machine Learning Research Group; Prince Sultan University Riyadh; Saudi Arabia [RG-CCIS-2017-06-02]; The authors are grateful for this financial support.

Author information

Affiliations

Authors

Contributions

All the authors contributed equally.

Corresponding author

Correspondence to Zahid Mehmood.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sharif, U., Mehmood, Z., Mahmood, T. et al. Scene analysis and search using local features and support vector machine for effective content-based image retrieval. Artif Intell Rev 52, 901–925 (2019). https://doi.org/10.1007/s10462-018-9636-0

Download citation

Keywords

  • Visual retrieval
  • Image matching
  • Complementary features
  • Clustering
  • Image attributes