Multimedia Tools and Applications

, Volume 78, Issue 5, pp 6163–6190 | Cite as

Image retrieval based on effective feature extraction and diffusion process

  • Juxiang Zhou
  • Xiaodong LiuEmail author
  • Wanquan Liu
  • Jianhou Gan


Feature extraction and its matching are two critical tasks in image retrieval. This paper presents a new methodology for content-based image retrieval by integrating three features, and then optimizing feature metric by diffusion process. To boost the discriminative power, the color histogram, local directional pattern, and dense SIFT features based on bag of features (BoF) are selected. Then diffusion process is applied to seek a global optimization for image matching based on fused multi-features. The diffusion process can capture the intrinsic manifold structure on a dataset, and thus enhance the overall retrieval performance significantly. Finally, a new search strategy is explored to make the diffusion process work even better when the number of retrieval images is small. In order to validate our proposed approach, four benchmark databases are used, and the results of experiments show that the proposed approach outperforms all other existing approaches.


Image retrieval Bag of features Diffusion processes 



We thank the anonymous reviewers and associate editor for their valuable comments that are invaluable in improving the quality of this paper. This work is supported by some grants from NSFC projects (Nos. 61673082, 61462097, 61602321, 61562093), and Application Infrastructure Projects of Science and Technology Plan in Yunnan Province (No.2016FD022).


  1. 1.
    Ahmad J, Sajjad M, Rho S, Baik SW (2016) Multi-scale local structure patterns histogram for describing visual contents in social image retrieval systems. Multimed Tools Appl 75(20):12669–12692CrossRefGoogle Scholar
  2. 2.
    Belalia A, Belloulata K, Kpalma K (2016) Region-based image retrieval in the compressed domain using shape-adaptive dct. Multimed Tools Appl 75(17):10175–10199CrossRefGoogle Scholar
  3. 3.
    Boureau YL, Bach F, LeCun Y, Ponce J (2010) Learning mid-level features for recognition. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 2559–2566. IEEEGoogle Scholar
  4. 4.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, pp 886–893Google Scholar
  5. 5.
    Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surveys (Csur) 40(2):5CrossRefGoogle Scholar
  6. 6.
    de Ves E, Benavent X, Coma I, Ayala G (2016) A novel dynamic multi-model relevance feedback procedure for content-based image retrieval. Neurocomputing 208:99–107CrossRefGoogle Scholar
  7. 7.
    Delaitre V, Laptev I, Sivic J (2010) Recognizing human actions in still images: a study of bag-of-features and part-based representations. In: BMVC 2010-21st British machine vision conferenceGoogle Scholar
  8. 8.
    Deng C, Ji R, Liu W, Tao D, Gao X (2014) Visual reranking through weakly supervised multi-graph learning. In: IEEE international conference on computer vision, pp 2600–2607Google Scholar
  9. 9.
    Deng C, Ji R, Tao D, Gao X, Li X (2014) Weakly supervised multi-graph learning for robust image reranking. IEEE Trans Multimed 16(3):785–795CrossRefGoogle Scholar
  10. 10.
    Donoser M, Bischof H (2013) Diffusion processes for retrieval revisited. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1320–1327Google Scholar
  11. 11.
    Dsouza D, Yampolskiy RV (2014) Natural vs artificial face classification using uniform local directional patterns and wavelet uniform local directional patterns. In: Computer vision and pattern recognition workshops, pp 27–33Google Scholar
  12. 12.
    ElAlami ME (2011) A novel image retrieval model based on the most relevant features. Knowl-Based Syst 24(1):23–32CrossRefGoogle Scholar
  13. 13.
    ElAlami ME (2014) A new matching strategy for content based image retrieval system. Appl Soft Comput 14:407–418CrossRefGoogle Scholar
  14. 14.
    Fei-Fei L, Perona P (2005) A bayesian hierarchical model for learning natural scene categories. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2, pp 524–531. IEEEGoogle Scholar
  15. 15.
    Giveki D, Soltanshahi MA, Montazer GA (2017) A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern. Optik-Int J Light Electron Opt 131:242–254CrossRefGoogle Scholar
  16. 16.
    He Z, You X, Tang YY (2008) Writer identification using global wavelet-based features. Neurocomputing 71(10):1832–1841CrossRefGoogle Scholar
  17. 17.
    Huang W, Gao Y, Chan KL (2010) A review of region-based image retrieval. J Signal Process Syst 59(2):143–161CrossRefGoogle Scholar
  18. 18.
    Jabid T, Kabir MH, Chae O (2010) Gender classification using local directional pattern (ldp). In: 2010 20th international conference on pattern recognition (ICPR), pp 2162–2165. IEEEGoogle Scholar
  19. 19.
    Jabid T, Kabir MH, Chae O (2010) Local directional pattern (ldp)–a robust image descriptor for object recognition. In: 2010 7th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 482–487. IEEEGoogle Scholar
  20. 20.
    Jabid T, Kabir MH, Chae O (2010) Local directional pattern (ldp) for face recognition. In: 2010 digest of technical papers international conference on consumer electronics (ICCE), pp 329–330. IEEEGoogle Scholar
  21. 21.
    Jégou H, Douze M, Schmid C (2010) Improving bag-of-features for large scale image search. Int J Comput Vis 87(3):316–336CrossRefGoogle Scholar
  22. 22.
    Jing F, Li M, Zhang HJ, Zhang B (2004) An efficient and effective region-based image retrieval framework. IEEE Trans Image Process 13(5):699–709CrossRefGoogle Scholar
  23. 23.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 2169–2178. IEEEGoogle Scholar
  24. 24.
    Liu GH (2015) Content-based image retrieval based on visual attention and the conditional probability. In: International conference on chemical, material, and food engineering, Atlantis press, pp 838–842Google Scholar
  25. 25.
    Liu GH (2016) Content-based image retrieval based on cauchy density function histogram. In: 2016 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), pp 506–510. IEEEGoogle Scholar
  26. 26.
    Liu GH, Yang JY (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198CrossRefGoogle Scholar
  27. 27.
    Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282CrossRefGoogle Scholar
  28. 28.
    Liu Y, Zhang D, Lu G (2008) Region-based image retrieval with high-level semantics using decision tree learning. Pattern Recogn 41(8):2554–2570CrossRefGoogle Scholar
  29. 29.
    Liu GH, Zhang L, Hou YK, Li ZY, Yang JY (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389CrossRefGoogle Scholar
  30. 30.
    Liu GH, Li ZY, Zhang L, Xu Y (2011) Image retrieval based on micro-structure descriptor. Pattern Recogn 44(9):2123–2133CrossRefGoogle Scholar
  31. 31.
    Liu GH, Yang JY, Li Z (2015) Content-based image retrieval using computational visual attention model. Pattern Recog 48(8):2554–2566CrossRefGoogle Scholar
  32. 32.
    Liu X, Huang L, Deng C, Lang B, Tao D (2016) Query-adaptive hash code ranking for large-scale multi-view visual search. IEEE Trans Image Process Public IEEE Signal Process Soc 25(10):4514– 4524MathSciNetCrossRefGoogle Scholar
  33. 33.
    Lowe DG, Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  34. 34.
    Lux M, Marques O (2013) Visual information retrieval using java and lire. Synthesis Lectures on Information Concepts, Retrieval, and Services 5(1):1–112CrossRefGoogle Scholar
  35. 35.
    O’Hara S, Draper BA (2011) Introduction to the bag of features paradigm for image classification and retrieval. Computer ScienceGoogle Scholar
  36. 36.
    Ojala T, Pietik IM (2000) Topi: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Springer, BerlinGoogle Scholar
  37. 37.
    Online: Corel5k,corel10k and ghim10k databases.
  38. 38.
  39. 39.
    Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. Comput Vis–ECCV 2010:143–156Google Scholar
  40. 40.
    Rao MB, Rao BP, Govardhan A (2011) Ctdcirs: content based image retrieval system based on dominant color and texture features. Int J Comput Appl 18(6):40–46Google Scholar
  41. 41.
    Shrivastava N, Tyagi V (2015) An efficient technique for retrieval of color images in large databases. Comput Electr Eng 46:314–327CrossRefGoogle Scholar
  42. 42.
    Singhai N, Shandilya SK (2010) A survey on: content based image retrieval systems. Int J Comput Appl 4(2):22–26Google Scholar
  43. 43.
    Smeulders AW, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380CrossRefGoogle Scholar
  44. 44.
    Subrahmanyam M, Wu QJ, Maheshwari R, Balasubramanian R (2013) Modified color motif co-occurrence matrix for image indexing and retrieval. Comput Electr Eng 39(3):762–774CrossRefGoogle Scholar
  45. 45.
    Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32CrossRefGoogle Scholar
  46. 46.
    Vadivel A, Sural S, Majumdar AK (2007) An integrated color and intensity co-occurrence matrix. Pattern Recogn Lett 28(8):974–983CrossRefGoogle Scholar
  47. 47.
    Verma M, Raman B, Murala S (2015) Local extrema co-occurrence pattern for color and texture image retrieval. Neurocomputing 165:255–269CrossRefGoogle Scholar
  48. 48.
    Vimina ER, Jacob KP (2013) A sub-block based image retrieval using modified integrated region matching. Int J Comput Sci Issues 10(1):686–692Google Scholar
  49. 49.
    Vipparthi SK, Nagar S (2014) Expert image retrieval system using directional local motif xor patterns. Expert Syst Appl 41(17):8016–8026CrossRefGoogle Scholar
  50. 50.
    Vipparthi SK, Murala S, Nagar SK (2015) Dual directional multi-motif xor patterns: A new feature descriptor for image indexing and retrieval. Optik-Int J Light Electron Opt 126(15):1467–1473CrossRefGoogle Scholar
  51. 51.
    Wang X, Wang Z (2013) A novel method for image retrieval based on structure elements descriptor. J Vis Commun Image Represent 24(1):63–74CrossRefGoogle Scholar
  52. 52.
    Wengert C, Douze M, Jégou H (2011) Bag-of-colors for improved image search. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1437–1440. ACMGoogle Scholar
  53. 53.
    Yang F, Matei B, Davis LS (2015) Re-ranking by multi-feature fusion with diffusion for image retrieval. In: Applications of computer vision, pp 572–579Google Scholar
  54. 54.
    Yang X, Koknar-Tezel S, Latecki LJ (2009) Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 357–364. IEEEGoogle Scholar
  55. 55.
    Yang X, Prasad L, Latecki LJ (2013) Affinity learning with diffusion on tensor product graph. IEEE Trans Pattern Anal Mach Intell 35(1):28–38CrossRefGoogle Scholar
  56. 56.
    Youssef SM (2012) Ictedct-cbir: Integrating curvelet transform with enhanced dominant colors extraction and texture analysis for efficient content-based image retrieval. Comput Electr Eng 38(5):1358–1376CrossRefGoogle Scholar
  57. 57.
    Yu J, Qin Z, Wan T, Zhang X (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364CrossRefGoogle Scholar
  58. 58.
    Zhou JX, Liu X, Xu TW, Gan JH, Liu WQ (2018) A new fusion approach for content based image retrieval with color histogram and local directional pattern. Int J Mach Learn Cybern 9(4):677–689CrossRefGoogle Scholar
  59. 59.
    Zhou L, Zhou Z, Hu D (2013) Scene classification using a multi-resolution bag-of-features model. Pattern Recogn 46(1):424–433CrossRefGoogle Scholar
  60. 60.
    Zhou Y, Zeng FZ, Zhao HM, Murray P, Ren J (2016) Hierarchical visual perception and two-dimensional compressive sensing for effective content-based color image retrieval. Cogn Comput 8(5):877–889CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Juxiang Zhou
    • 1
    • 3
  • Xiaodong Liu
    • 1
    Email author
  • Wanquan Liu
    • 2
  • Jianhou Gan
    • 3
  1. 1.School of Control Science and EngineeringDalian University of TechnologyDalianChina
  2. 2.Department of ComputingCurtin UniversityPerthAustralia
  3. 3.Key Laboratory of Education Informatization for Nationalities, Ministry of EducationYunnan Normal UniversityKunmingChina

Personalised recommendations