Skip to main content
SpringerLink
Log in

Image retrieval using underlying importance feature histogram

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep features can exhibit superior retrieval performance than low-level features. However, low-level features (e.g. colour and orientation) can be extracted by generally imitating the human visual perceptual system. Combining human-like low-level and deep features can harmoniously yield more discriminative representations. However, it remains challenging. To address this problem, a new representation method for image retrieval, namely the underlying importance feature histogram (UIFH), is presented in this study. Its main highlights are: (1) This new method extracts low-level features by simulating the human visual perception mechanism, such as opponent colour and orientation selectivity mechanisms. (2) Inspired by the salience evaluation mechanism, the new method can harmoniously evaluate the underlying importance information between deep and low-level features. (3) Assisting the various important information can facilitate the UIFH. It can substantially improve the discriminative power of representation. Comprehensive experiments on seven benchmark datasets demonstrated that the proposed UIFH method outperforms some recent state-of-the-art methods based on pre-trained models. The proposed UIFH method is suitable for the retrieval scenes where images have various colours and prominent orientations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The datasets generated and/or analysed during the current study are available from the corresponding author upon reasonable request.

References

  1. Tyagi V (2017) Content-based image retrieval: ideas, influences, and current trends. Springer, Singapore

    Book  Google Scholar 

  2. Smeulders AWM, Worring M, Santini S et al (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  3. Chen W, Liu Y, Wang W et al (2023) Deep learning for instance retrieval: a survey. IEEE Trans Pattern Anal Mach Intell 45(6):7270–7292

    Article  Google Scholar 

  4. Zhu Y, Nachtrab G, Keyes PC et al (2018) Dynamic salience processing in paraventricular thalamus gates associative learning. Science 362(6413):423–429

    Article  Google Scholar 

  5. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR, pp 1–14

  6. Gegenfurtner KR (2003) Cortical mechanisms of colour vision. Nat Rev Neurosci 4:563–572

    Article  Google Scholar 

  7. Shapley R, Hawken M (2011) Color in the cortex-single-and double-opponent cells. Vis Res 51:701–717

    Article  Google Scholar 

  8. Shapley R, Hawken M (2002) Neural mechanisms for color perception in the primary visual cortex. Curr Opin Neurobiol 12:426–432

    Article  Google Scholar 

  9. Marĉelja S (1980) Mathematical description of the responses of simple cortical cells. J Opt Soc Am 70(11):1297–1300

    Article  MathSciNet  Google Scholar 

  10. Daugman JG (1980) Two-dimensional spectral analysis of cortical receptive field profiles. Vis Res 20(10):847–856

    Article  Google Scholar 

  11. Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimised by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169

    Article  Google Scholar 

  12. Jain AK, Farrokhnia F (1991) Unsupervised texture segmentation using Gabor filters. Pattern Recognit 24:1167–1186

    Article  Google Scholar 

  13. Kruizinga P, Petkov N (1999) Nonlinear operator for oriented texture. IEEE Trans Image Process 8(10):1395–1407

    Article  Google Scholar 

  14. Liu GH, Yang JY (2021) Deep-seated features histogram: a novel image retrieval method. Pattern Recognit 116:107926

    Article  Google Scholar 

  15. Liu GH, Yang JY (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16

    Article  MathSciNet  Google Scholar 

  16. Liu GH, Yang JY, Li ZY (2015) Content-based image retrieval using computational visual attention model. Pattern Recognit 48(8):2554–2566

    Article  Google Scholar 

  17. Yuan BH, Liu GH (2020) Image retrieval based on gradient-structures histogram. Neural Comput Appl 32(15):11717–11727

    Article  Google Scholar 

  18. Singh C, Walia E, Kaur KP (2017) Color texture description with novel local binary patterns for effective image retrieval. Pattern Recognit 76:50–68

    Article  Google Scholar 

  19. Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032

    Article  MathSciNet  Google Scholar 

  20. Saikia S, Fernández-Robles L, Alegre E et al (2021) Image retrieval based on texture using latent space representation of discrete Fourier transformed maps. Neural Comput Appl 33:13301–13316

    Article  Google Scholar 

  21. Liu GH, Yang JY (2013) Content-based image retrieval using color difference histogram. Pattern Recognit 46(1):188–198

    Article  Google Scholar 

  22. Clement M, Kurtz C, Wendling L (2018) Learning spatial relations and shapes for structural object description and scene recognition. Pattern Recognit 84:197–210

    Article  Google Scholar 

  23. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  24. Bay H, Tuytelaars T, Gool LV (2006) SURF: speeded up robust features. In: ECCV, pp 404–417

  25. Sivic J, Zisserman A (2003) Video Google: a text retrieval approach to object matching in videos. In: ICCV, pp 1470–1477

  26. Perronnin F, Sanchez J, Mensink T (2010) Improving the Fisher kernel for large-scale image classification. In: ECCV, pp 143–156

  27. Jégou H, Perronnin F et al (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716

    Article  Google Scholar 

  28. Jégou H, Zisserman A (2014) Triangulation embedding and democratic aggregation for image search. In: CVPR, pp 3310–3317

  29. Husain SS, Bober M (2017) Improving large-scale image retrieval through robust aggregation of local descriptors. IEEE Trans Pattern Anal Mach Intell 39(9):1783–1796

    Article  Google Scholar 

  30. Babenko A, Slesarev A, Chigorin A, Lempitsky V (2013) Neural codes for image retrieval. In: ECCV, pp 584–599

  31. Babenko A, Lempitsky V (2015) Aggregating local deep features for image retrieval. In: ICCV, pp 1269–1277

  32. Tolias G, Sicre R, Jégou H (2015) Particular object retrieval with integral max-pooling of CNN activations. In: ICLR, pp 1–15

  33. Kalantidis Y, Mellina C, Osindero S (2016) Cross-dimensional weighting for aggregated deep convolutional features. In: ECCV, pp 685–701

  34. Xu J, Wang C et al (2019) Unsupervised semantic-based aggregation of deep convolutional features. IEEE Trans Image Process 28(2):601–611

    Article  MathSciNet  Google Scholar 

  35. Lu F, Liu GH (2022) Image retrieval using contrastive weight aggregation histograms. Digit Signal Process 123:103457

    Article  Google Scholar 

  36. Li J, Bo Y, Yang W et al (2021) Subspace-based multi-view fusion for instance-level image retrieval. Visual Comput 37:619–633

    Article  Google Scholar 

  37. Liu GH, Yang JY (2023) Exploiting deep textures for image retrieval. Int J Mach Learn Cyb 14:483–493

    Article  Google Scholar 

  38. Lu Z, Liu GH, Lu F, Zhang B (2023) Image retrieval using dual-weighted deep feature descriptor. Int J Mach Learn Cyb 14:643–653

    Article  Google Scholar 

  39. Lu F, Liu GH (2023) Image retrieval using object semantic aggregation histogram. Cogn Comput 15:1736–1747

    Article  Google Scholar 

  40. Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2018) NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1437–1451

    Article  Google Scholar 

  41. Radenovic F, Tolias G, Chum O (2018) Fine-tuning CNN image retrieval with no human annotation. IEEE Trans Pattern Anal Mach Intell 41(7):1655–1668

    Article  Google Scholar 

  42. Husain SS, Ong EJ, Bober M (2021) ACTNET: end-to-end learning of feature activations and multi-stream aggregation for effective instance image retrieval. Int J Comput Vision 129:1432–1450

    Article  Google Scholar 

  43. Bai C, Li H, Zhang J et al (2021) Unsupervised adversarial instance-level image retrieval. IEEE Trans Multimed 23:2199–2207

    Article  Google Scholar 

  44. El-Nouby A, Neverova N, Laptev I, Jégou H (2021) Training vision transformers for image retrieval. https://doi.org/10.48550/arXiv.2102.05644

  45. Van de Sande KEA, Gevers T, Snoek CGM (2010) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell 32(9):1582–1596

    Article  Google Scholar 

  46. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: NeurIPS, pp 1097–1105

  47. He K, Zhang X, et al (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778

  48. Ding X, Zhang X, et al (2021) RepVGG: making VGG-style ConvNets great again. In: CVPR, pp 13728–13737

  49. Molenberghs G, Lesaffre E (1997) Non-Linear Integral Equations to Construct Bivariate Densities with Given Marginals and Dependence Function. Stat Sinica 7:713–738

    Google Scholar 

  50. UFLDL Tutorial, PCA Whitening. http://ufldl.stanford.edu/tutorial/unsupervised/PCAWhitening/. Accessed 20 Oct 2023

  51. Philbin J, Chum O, et al (2007) Object retrieval with large vocabularies and fast spatial matching. In: CVPR, pp 1–8

  52. Philbin J, Chum O, et al (2008) Lost in quantisation: improving particular object retrieval in large scale image databases. In: CVPR, pp 1–8

  53. Jégou H, Douze M, Schmid C (2008) Hamming embedding and weak geometry consistency for large scale image search. In: ECCV, pp 304–317

  54. Radenovic F, Iscen A, Tolias G, Avrithis Y, Chun O (2018) Revisiting Oxford and Paris: large-scale image retrieval benchmarking. In: CVPR, pp 5706–5715

  55. Forcén JI, Pagola M, Barrenechea E, Bustince H (2020) Co-occurrence of deep convolutional features for image search. Image Vis Comput 97:103909

    Article  Google Scholar 

  56. Gkelios S, Boutalis Y, Chatzichristofis SA (2021) Investigating the vision transformer model for image retrieval tasks. https://doi.org/10.48550/arXiv.2101.03771

  57. Chum O, Philbin J, Sivic J, Isard M, Zisserman A (2007) Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV, pp 1–8

  58. Liu GH, Li ZY, Yang JY, Zhang D (2024) Exploiting sublimated deep features for image retrieval. Pattern Recognit 147:110076

    Article  Google Scholar 

Download references

Acknowledgements

This study is supported by National Natural Science Foundation of China (grant no. 62266008), the Foundation of Guangxi Normal University (grant no. 2021JC007) and the Foundation of Development Research Centre of Guangxi in Humanities and Social Sciences (grant no. ZXZJ202201). Here, I am very grateful to Dr. Fen Lu (My graduated doctor), because the first author (Qiao-Ping He) has other research tasks, the revised manuscript was completed by Dr. Fen Lu.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, China

    Qiao-Ping He & Guang-Hai Liu

Authors
  1. Qiao-Ping He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guang-Hai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

QPH helped in conceptualisation, software, validation, writing—original draft, resources and data curation. GHL helped in methodology, writing—review & editing, supervision, revision, funding acquisition and formal analysis.

Corresponding author

Correspondence to Guang-Hai Liu.

Ethics declarations

Conflict of interest

Authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work submitted for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, QP., Liu, GH. Image retrieval using underlying importance feature histogram. Neural Comput & Applic (2024). https://doi.org/10.1007/s00521-024-09735-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00521-024-09735-6

Keywords

Search

Navigation