Skip to main content

A hybrid late fusion-genetic algorithm approach for enhancing CBIR performance

Multimedia Tools and Applications volume 79, pages 20281–20298 (2020)Cite this article

Abstract

Accurate discrimination of images features is a main success factor towards efficient content-based image retrieval systems. These features can be extracted using local and/or global descriptors. Researchers efforts showed that, hybrid descriptors reported superior results compared to methods that use single descriptor, where hybridization certainly complements benefits from different perspectives. Genetic Algorithm (GA) is a heuristic computational intelligence approach that can be used to achieve the optimal satisfactory user image retrieval requests. In this paper, a new hybrid efficient and effective evolutionary retrieval approach (CBIR-GAF) based on late fusion of four global descriptors is proposed. Each descriptor produces a list of retrieved similar images to user query image and if these lists are merged correctly by late fusion, the results are improved. Thus, GA occurs to assign different weights to each retrieved image while merging, and then it optimizes these weights with a suitable fitness function to select optimum heterogeneous retrieved images. The proposed approach is evaluated on two benchmark datasets (Inria Holidays and Oxford5k), and reported a promising results where it enhanced the average accuracy in comparison of literature techniques.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. http://www.robots.ox.ac.uk/~vgg/data/oxbuildings/

  2. https://lear.inrialpes.fr/~jegou/data.php

References

  1. Al-Sahaf H, Al-Sahaf A, Xue B, Johnston M, Zhang M (2017) Automatically evolving rotation-invariant texture image descriptors by genetic programming. IEEE Trans Evol Comput 21(1):83–101

    Google Scholar 

  2. Ashraf R, Ahmed M, Jabbar S, Khalid S, Ahmad A, Din S, Jeon G (2018) Content Based Image Retrieval by using Color Descriptor and Discrete wavelet transform. J Med Syst 42(3):1–12

    Google Scholar 

  3. Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2016) NetVLAD CNN architecture for weakly supervised place recognition. In CVPR

  4. Babenko A, Lempitsky VS (2015) Aggregating deep convolutional features for image retrieval. CoRR:4321–4330

  5. Babenko A, Slesarev A, Chigorin A, Lempitsky VS (2014) Neural Codes for Image Retrieval. In: Computer Vision - ECCV 2014 - 13th European Conference. Proceedings, Part I, Zurich, pp 584–599

  6. Barley A, Town C (2014) Combinations of feature descriptors for texture image classification. J Data Anal Inf Process 2(3):67–76

    Google Scholar 

  7. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up Robust Features (surf). Comput Vis Image Underst 110(3):346–359

    Google Scholar 

  8. Belattar K, Mostefai S, Draa A (2018) A hybrid ga-lda scheme for feature selection in content-based image retrieval. Int J Appl Metaheuristic Comput 9(2):48–71

    Google Scholar 

  9. Bober M (2001) Mpeg-7 visual shape descriptors. IEEE Trans Circ Syst Video Technol 11(6):716–719

    Google Scholar 

  10. Boparai NK, Chhabra A (2015) A hybrid approach for improving content based image retrieval systems. In: 2015 1st International Conference on Next Generation Computing Technologies (NGCT), pp 944–949

  11. Brown M, Hua G, Winder S (2011) Discriminative learning of local image descriptors. IEEE Trans Pattern Anal Mach Intell 33(1):43–57

    Google Scholar 

  12. Chatzichristofis SA, Boutalis YS (2008) Cedd: Color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. (ICVS’08) international conference on Computer Vision Systems:312–322

  13. Cieplinski L (2001) Mpeg-7 Color Descriptors and their Applications. In: Proceedings of the 9th International Conference on Computer Analysis of Images and Patterns, CAIP ’01, pp 11–20

  14. Ciocca G, Corchs S, Gasparini F (2016) Genetic programming approach to evaluate complexity of texture images. Electron Imaging J 25(6):061408

    Google Scholar 

  15. De K, Masilamani V (2018) No-reference Image Quality Measure for Images with Multiple Distortions using Random Forests for Multi Method Fusion. Image Anal Stereol 37(2):105–117

    MATH  Google Scholar 

  16. Douze M, Jegou H, Sandhawalia H, Amsaleg L, Schmid C (2009) Evaluation of GIST Descriptors for Web-scale Image Search Inproceedings of the 8th ACM International Conference on Image and Video Retrieval, CIVR 19:1–8

  17. El-Naser GA, Mahmoud AM, El-Horbaty El-SM. (2014) A compartive study of Meta-Heuristic algorithms for solving quadratic assignment problem. Int J Adv Comput Sci Appl (IJACSA) 5(1):1–6

    Google Scholar 

  18. Gali R, Dewal ML, Anand RS (2012) Genetic algorithm for content based image retrieval. In: 2012 fourth international conference on computational intelligence, Communication Systems and Networks, pp 243–247

  19. Gao Z, Xuan H-Z, Zhang H, Wan Sh, Kwang K, Choo R (2019) Adaptive Fusion and Category-Level Dictionary Learning Model for Multi-View Human Action Recognition This article has been accepted for publication in a future issue of IEEE INTERNET OF THINGS JOURNAL

  20. Gehler P, Nowozin S (2009) On Feature Combination for Multiclass object classification. In: 2009 IEEE 12th International Conference on Computer Vision, pp 221–228

  21. Gopal N, Bhooshan RS (2015) Content Based Image Retrieval using Enhanced Surf. In: Fifth National Conference on Computer Vision, Pattern recognition, Image Processing and Graphics (NCVPRIPG), pp 1–4

  22. Jain AK, Nandakumar K, Ross A (2005) Score normalization in multimodal biometric systems. Pattern Recogn 38(12):2270–2285

    Google Scholar 

  23. Jégou H, Zisserman A (2014) Triangulation Embedding and Democratic Aggregation for Image Search. In: 2014 IEEE conference on computer vision and pattern recognition, CVPR, pp 3310–3317,

  24. Karamti H, Tmar M, Visani M, Urruty T, Gargouri F (2018) Vector space model adaptation and pseudo relevance feedback for Content-Based image retrieval. Multimed Tools Appl 77(5):5475–5501

    Google Scholar 

  25. Kasutani E, Yamada A (2001) The MPEG-7 color layout descriptor: a compact image feature description for High-Speed Image/Video segment retrieval. In: ICIP, (1), pp 674–677

  26. Kurchaniya D, Johari P (2017) Analysis of different similarity measures in image retrieval based on texture and shape. Int Res J Eng Technol (IRJET) 4:2395–0056

    Google Scholar 

  27. Li L, Chen J, Fieguth PW, Zhao G, Chellappa R, Pietikäinen M (2018) A survey of recent advances in texture representation. The Computing Research Repository (CoRR):1801–10324

  28. Liu G-H, Yang J-Y (2013) Content-based Image Retrieval using Color Difference Histogram. Pattern Recogn 46(1):188–198

    Google Scholar 

  29. Lowe DG (2004) Distinctive image features from Scale-Invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  30. Madhavi KV, Tamilkodi R, sudha KJ (2016) An innovative method for retrieving relevant images by getting the top-ranked images first using interactive genetic algorithm. Proc Comput Sci 79:254–261

    Google Scholar 

  31. Mehmood Z, Anwar S, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016:1–12

    Google Scholar 

  32. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

    Google Scholar 

  33. Mingqiang Y, Idiyo KK, Joseph R (2008) A Survey of Shape Feature Extraction Techniques. Pattern Recognition Peng-Yeng Yin (Ed. ), pp 43–90

  34. Mukherjee A (2016) Content Based Image Retrieval using GLCM. Int J Innov Res Comput Commun Eng 4:20142–20149

    Google Scholar 

  35. Nouman A, Bashir BK, Robert S, Zeshan CSA, Muhammad R, Adnan HH (2016) Iqbal a novel image retrieval based on visual words integration of sift and surf. PLOS One 11(6):1–20

    Google Scholar 

  36. Oliva Aude, Torralba A (2001) Modeling the shape of the scene a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    MATH  Google Scholar 

  37. Ortiz-Jaramillo B, Benítez-Restrepo H, García-Álvarez JC, Castellanos-Domínguez CG (2010) Region of Interest Extraction based on Multi-resolution Analysis for Infrared Nondestructive Testing. In: 10th Quantitative Infrared Thermography Conference, QIRT

  38. Park DK, Jeon YS, Won CS (2000) Efficient use of local edge histogram descriptor. In: Proceedings of the ACM, Multimedia 2000 Workshops, pp 51–54

  39. Paulin M, Douze M, Harchaoui Z, Mairal J, Perronnin F, Schmid C (2015) Local convolutional features with unsupervised training for image retrieval. In: IEEE international conference on computer vision, ICCV, pp 91–99

  40. Qin D, Gammeter S, Bossard L, Quack T, van Gool L (2011) Hello neighbor: Accurate Object Retrieval with K-reciprocal Nearest Neighbors. In: CVPR 2011, pp 777–784

  41. Radenovic F, Jegou H, Chum O (2015) Multiple Measurements and Joint Dimensionality Reduction for Large Scale Image Search with Short Vectors. In: Computer Vision and Pattern Recognition

  42. Radenovic F, Tolias G, Chum O (2016) CNN image retrieval ‘learns from BoW: Unsupervised fine-tuning with hard examples Pattern Recognition. In: ECCV

  43. Radenovic F, Tolias G, Chum O (2017) Fine-tuning CNN ‘ image retrieval with no human annotation. In: arXiv

  44. Radenovic F, Iscen A, Tolias G, Avrithis Y, Chum O (2018) Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking IEEE Conference on Computer Vision and Pattern Recognition (CVPR):5706–5715

  45. Rashedi E, Nezamabadi-pour H, Saryazdi S (2013) A simultaneous feature adaptation and feature selection method for Content-Based image retrieval systems. Knowl-Based Syst 39:85–94

    Google Scholar 

  46. Razavian AS, Sullivan J, Maki A, Carlsson S (2015) A baseline for visual instance retrieval with deep convolution networks. In: Proceedings of ICLR

  47. Razavian AS, Sullivan J, Carlsson S, Maki A (2016) Visual instance retrieval with deep convolutional networks. JITE Trans MTA

  48. Salem A.-B.M., Mahmoud AM (2003) A Hybrid Genetic Algorithm-Decision Tree Classifier. Proc. of 3rd Int. Conference on New Trends in Intelligent Information Processing and Web Mining, Zakopane, Poland:221–232

  49. Salembier P, Sikora T (2002) Introduction to MPEG-7: multimedia content description interface. Wiley, New York

  50. Sharma P (2017) Improved Shape Matching and Retrieval using Robust Histograms of Spatially Distributed Points and Angular Radial Transform. Optik 145:346–364

    Google Scholar 

  51. Stathopoulos S, Kalamboukis T (2015) Applying latent semantic analysis to large-scale medical image databases. Comput Med Imaging Graph 39:27–34

    Google Scholar 

  52. Teran L, Mordohai P (2014) 3d Interest Point Detection Via Discriminative Learning. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision – ECCV, pp 159–173

  53. Tolias G, Sicre R, Jegou H (2016) Particular object retrieval with integral maxpooling of CNN activations. In: ICLR

  54. Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348

    Google Scholar 

  55. Wanb Z, Gao DY, Wang SH, Zhang H, Wanga YL (2019) Cognitive-inspired class-statistic matching with triple-constrain for camera free 3D object retrieval. Fut Gener Comput Syst J 94:641–653

    Google Scholar 

  56. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Google Scholar 

  57. Wasson PV (2017) An Efficient Content Based Image Retrieval Based on Speeded up Robust Features (surf) with Optimization Technique. In: 2nd IEEE International Conference on Recent Trends in Electronics, Information Communication Technology (RTEICT), pp 730–735

  58. Wei C-H, Li Y, Chau W-Y, Li C-T (2009) Trademark Image Retrieval using Synthetic Features for Describing Global Shape and Interior Structure. Pattern Recogn 42(3):386–394

    MATH  Google Scholar 

  59. Wu H, He L (2015) Combining Visual and Textual Features for Medical Image Modality Classification with lp-norm Multiple Kernel Learning. Neurocomputing 147:387–394

    Google Scholar 

  60. Wu H, Liu B, Su W, Chen Z, Zhang W, Ren X, Sun J (2017) Optimum Pipeline for Visual Terrain Classification using Improved Bag of Visual Words and Fusion Methods. J. Sensors 2017:1–25

    Google Scholar 

  61. Xiang-Yang W, Lin-Lin L, Li Y-W, Hong-Ying Y (2017) Image retrieval based on exponent moments descriptor and localized angular phase histogram. Multimed Tools Appl 76(6):7633–7659

    Google Scholar 

  62. Yu J, Qin Z, Wan T, Xi Z (2013) Feature Integration Analysis of Bag-of-Features Model for Image Retrieval. Neurocomputing 120:355–364

    Google Scholar 

  63. Zeng S, Huang R, Wang H, Kang Z (2016) Image Retrieval using Spatiograms of Colors Quantized by Gaussian Mixture Models. Neurocomputing 171:673–684

    Google Scholar 

  64. Zou Y, Chen W, Xie L, Wu X (2014) Comparison of different approaches to visual terrain classification for outdoor mobile robots. Pattern Recogn Lett 38:54–62

    Google Scholar 

Download references

Acknowledgements

This research was funded by the Deanship of Scientific Research at Princess Nourah bint Abdulrahman University through the Fast-track Research Funding Program.

Author information

Authors and Affiliations

  1. Computer Sciences Department, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, PO Box 84428, Riyadh, Saudi Arabia

    Abeer M. Mahmoud, Hanen Karamti & Myriam Hadjouni

  2. Computer Science Department, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt

    Abeer M. Mahmoud

  3. MIRACL Laboratory, ISIMS, University of Sfax, B.P. 242, 3021 Sakiet Ezzit, Sfax, Tunisia

    Hanen Karamti

  4. Higher Institute of Computer Science and Telecom (ISITCom)University of Sousse, Sousse, Tunisia

    Myriam Hadjouni

Authors
  1. Abeer M. Mahmoud
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanen Karamti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Myriam Hadjouni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abeer M. Mahmoud.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mahmoud, A.M., Karamti, H. & Hadjouni, M. A hybrid late fusion-genetic algorithm approach for enhancing CBIR performance. Multimed Tools Appl 79, 20281–20298 (2020). https://doi.org/10.1007/s11042-020-08825-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08825-6

Keywords

  • Genetic algorithm
  • CBIR
  • Late fusion
  • Global descriptors