Skip to main content
SpringerLink
Log in

Combining re-ranking and rank aggregation methods for image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents novel approaches for combining re-ranking and rank aggregation methods aiming at improving the effectiveness of Content-Based Image Retrieval (CBIR) systems. Given a query image as input, CBIR systems retrieve the most similar images in a collection by taking into account image visual properties. In this scenario, accurately ranking collection images is of great relevance. Aiming at improving the effectiveness of CBIR systems, re-ranking and rank aggregation algorithms have been proposed. However, different re-ranking and rank aggregation approaches, applied to different image descriptors, may produce different and complementary image rankings. In this paper, we present four novel approaches for combining these rankings aiming at obtaining more effective results. Several experiments were conducted involving shape, color, and texture descriptors. The proposed approaches are also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate that our approaches can improve significantly the effectiveness of image retrieval systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://research.rutgers.edu/∼shaoting/image_search.html (As of October 2015). Images not present in the provided rankings had their distance defined as a constant n s =200.

References

  1. Arica N, Vural FTY (2003) BAS: a perceptual shape descriptor based on the beam angle statistics. Pattern Recogn Lett 24(9–10):1627–1639

    Article  MATH  Google Scholar 

  2. Baeza-Yates RA, Ribeiro-Neto B (1999) Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston

    Google Scholar 

  3. Bai X, Wang B, Wang X, Liu W, Tu Z (2010) Co-transduction for shape retrieval. In: European conference on computer vision (ECCV’2010), vol 3, pp 328–341

  4. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(4):509–522

    Article  Google Scholar 

  5. Brodatz P (1966) Textures: A Photographic Album for Artists and Designers. Dover

  6. Carrillo M, Villatoro-Tello E, López-López A, Eliasmith C, Montes-Y-Gómez M, Villaseñor Pineda L (2009) Representing context information for document retrieval. In: 8th international conference on flexible query answering systems (FQAS’09), pp 239–250

  7. Clinchant S, Ah-Pine J, Csurka G (2011) Semantic combination of textual and visual information in multimedia retrieval. In: ACM international conference on multimedia retrieval (ICMR’11), pp 44:1–44:8

  8. Coppersmith D, Fleischer LK, Rurda A (2010) Ordering by weighted number of wins gives a good ranking for weighted tournaments. ACM Transactions Algorithms 6(3):55:1–55:13

    Article  MathSciNet  MATH  Google Scholar 

  9. Cormack GV, Clarke CLA, Buettcher S (2009) Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: ACM SIGIR conference on research and development in information retrieval, pp 758–759

  10. Croft WB (2002) Combining approaches to information retrieval. In: Croft WB, Croft WB (eds) Advances in information retrieval, The information retrieval, vol 7, pp 1–36. Springer, US

  11. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60. doi:10.1145/1348246.1348248

    Article  Google Scholar 

  12. Deselaers T, Keysers D, Ney H (2008) Features for image retrieval: an experimental comparison. Inf Retr 11(2):77–107

    Article  Google Scholar 

  13. Fagin R, Kumar R, Mahdian M, Sivakumar D, Vee E (2004) Comparing and aggregating rankings with ties. In: 23th ACM SIGMOD symposium on principles of database systems (PODS’04), pp 47–58

  14. Fagin R, Kumar R, Sivakumar D (2003) Comparing top k lists. In: ACM-SIAM symposium on discrete algorithms (SODA’03), pp 28–36

  15. Faria FF, Veloso A, Almeida HM, Valle E, da S. Torres R, Gonçalves MA,Meira Jr.WM(2010) Learning to rank for content-based image retrieval. In:Multimedia information retrieval (MIR’2010), pp 285–294. doi:10.1145/1743384.1743434

  16. Fox EA, Shaw JA (1994) Combination of multiple searches. In: The Second Text REtrieval Conference (TREC-2), NIST Special Publication, vol 500–215, pp 243–252. NIST

  17. Gopalan R, Turaga P, Chellappa R (2010) Articulation-invariant representation of non-planar shapes. In: 11th European conference on computer vision (ECCV’2010), vol 3, pp 286–299

  18. Hoi SC, Liu W, Chang SF (2010) Semi-supervised distance metric learning for collaborative image retrieval and clustering. ACM Transactions on Multimedia Computing and Communication Applications 6(3):18:1–18:26

    Article  Google Scholar 

  19. Huang CB, Liu Q (2007) An orientation independent texture descriptor for image retrieval. In: International conference on communications, circuits and systems (ICCCAS 2007), pp 772–776

  20. Huang J, Kumar SR, Mitra M, Zhu WJ, Zabih R (1997) Image indexing using color correlograms. In: IEEE Conference on computer vision and pattern recognition (CVPR’97), pp 762–768

  21. Jegou H, Schmid C, Harzallah H, Verbeek J (2010) Accurate image search using the contextual dissimilarity measure. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1):2–11

    Article  Google Scholar 

  22. Kontschieder P, Donoser M, Bischof H (2009) Beyond pairwise shape similarity analysis. In: Asian conference on computer vision, pp 655–666

  23. Kovalev V, Volmer S (1998) Color co-occurence descriptors for querying-by-example. In: International conference on multimedia modeling, p 32

  24. Krapac J, Allan M, Verbeek J, Jurie F (2010) Improving web image search results using query-relative classifiers. In: IEEE Conference on computer vision and pattern recognition (CVPR’2010), pp 1094–1101

  25. Latecki LJ, Lakmper R, Eckhardt U (2000) Shape descriptors for non-rigid shapes with a single closed contour. In: IEEE Conference on computer vision and pattern recognition (CVPR’2000), pp 424–429

  26. Lewis J, Ossowski S, Hicks J, Errami M, Garner HR (2006) Text similarity: an alternative way to search medline. Bioinformatics 22(18):2298–2304

    Article  Google Scholar 

  27. Ling H, Jacobs DW (2007) Shape classification using the inner-distance. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(2):286–299. doi:10.1109/TPAMI.2007.41

    Article  Google Scholar 

  28. Ling H, Yang X, Latecki LJ (2010) Balancing deformability and discriminability for shape matching. In: European conference on computer vision (ECCV’2010), vol 3, pp 411–424

  29. Liu YT, Liu TY, Qin T, Ma ZM, Li H (2007) Supervised rank aggregation. In: International Conference on World Wide Web (WWW’2007), pp 481–490

  30. Nistér D, Stewénius H (2006) Scalable recognition with a vocabulary tree. In: IEEE Conference on computer vision and pattern recognition (CVPR’2006), vol 2, pp 2161–2168

  31. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987

    Article  MATH  Google Scholar 

  32. Park G, Baek Y, Lee HK (2005) Re-ranking algorithm using post-retrieval clustering for content-based image retrieval. Inf Process Manag 41(2):177–194

    Article  MATH  Google Scholar 

  33. Pedronette DCG, Almeida J, da S, Torres R (2014) A scalable re-ranking method for content-based image retrieval. Inf Sci 265(1):91–104

    Article  MathSciNet  MATH  Google Scholar 

  34. Pedronette DCG, da S, Torres R (2010) Exploiting contextual information for image re-ranking. In: Iberoamerican congress on pattern recognition (CIARP’2010), pp 541–548

  35. Pedronette DCG, da S, Torres R (2010) Shape retrieval using contour features and distance optmization. In: International joint conference on computer vision, imaging and computer graphics theory and applications (VISAPP’2010), vol 1, pp 197–202

  36. Pedronette DCG, da S, Torres R (2011) Exploiting clustering approaches for image re-ranking. J Vis Lang Comput 22(6):453–466

    Article  Google Scholar 

  37. Pedronette DCG, da S, Torres R (2011) Exploiting contextual information for rank aggregation. In: International conference on image processing (ICIP’2011), pp 97–100

  38. Pedronette DCG, da S, Torres R (2011) Exploiting contextual spaces for image re-ranking and rank aggregation. In: ACM international conference on multimedia retrieval (ICMR’11), pp 13:1–13:8

  39. Pedronette DCG, da S, Torres R (2012) Combining re-ranking and rank aggregation methods. In: Iberoamerican congress on pattern recognition (CIARP’2012), pp 170–178

  40. Pedronette DCG, da S, Torres R (2012) Exploiting contextual information for image re-ranking and rank aggregation. International Journal of Multimedia Information Retrieval 1(2):115–128

    Article  Google Scholar 

  41. Pedronette DCG, da S, Torres R (2013) Image re-ranking and rank aggregation based on similarity of ranked lists. Pattern Recogn 46(8):2350–2360

    Article  Google Scholar 

  42. Pedronette DCG, da S, Torres R, Borin E, Breternitz M (2013) Rl-sim algorithm acceleration on GPUs. In: International symposium on computer architecture and high performance computing (SBAC 2013), pp 176–183

  43. Perronnin F, Liu Y, Renders JM (2009) A family of contextual measures of similarity between distributions with application to image retrieval. In: IEEE conference on computer vision and pattern recognition (CVPR’2009), pp 2358–2365

  44. Qin D, Gammeter S, Bossard L, Quack T, van Gool L (2011) Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In: IEEE conference on computer vision and pattern recognition (CVPR’2011), pp 777–784

  45. Renda ME, Straccia U (2003) Web metasearch: Rank vs. score based rank aggregation methods. In: Proceedings of the 2003 ACM symposium on applied computing, SAC ’03, pp 841–846

  46. Robertson SE, Walker S, Jones S, Hancock-Beaulieu M, Gatford M (1994) Okapi at trec-3. In: Text REtrieval Conference, pp 109–126

  47. da S, Torres R, Falcão AX (2006) Content-Based Image Retrieval: Theory and Applications. Revista de Informática Teórica e Aplicada 13(2):161–185

    Google Scholar 

  48. da S, Torres R, Falcão AX (2007) Contour Salience Descriptors for Effective Image Retrieval and Analysis. Image Vis Comput 25(1):3–13

    Article  Google Scholar 

  49. Schwander O, Nielsen F (2010) Reranking with contextual dissimilarity measures from representational bregmanl k-means. In: International joint conference on computer vision, imaging and computer graphics theory and applications (VISAPP’2010), vol 1, pp 118–122

  50. Stehling RO, Nascimento MA, Falcão AX (2002) A compact and efficient image retrieval approach based on border/interior pixel classification. In: ACM conference on information and knowledge management (CIKM’2002), pp 102–109

  51. Swain MJ, Ballard DH (1991) Color indexing. International Journal on Computer Vision 7(1):11–32

    Article  Google Scholar 

  52. Tao B, Dickinson BW (2000) Texture recognition and image retrieval using gradient indexing. Journal of Visual Comunication and Image Representation 11(3):327–342

    Article  Google Scholar 

  53. Thollard F, Qunot G (2013) Content-based re-ranking of text-based image search results. In: European conference on IR research (ECIR’2013), vol 7814, pp 618–629

  54. Tu Z, Yuille AL (2004) Shape matching and recognition - using generative models and informative features. In: European conference on computer vision (ECCV’2004), pp 195–209

  55. Voravuthikunchai W, Crėmilleux B, Jurie F (2014) Image re-ranking based on statistics of frequent patterns. In: International conference on multimedia retrieval, ICMR’14, p 129

  56. Wang B, Jiang J, Wang W, Zhou ZH, Tu Z (2012) Unsupervised metric fusion by cross diffusion. In: IEEE conference on computer vision and pattern recognition (CVPR’2012), pp 3013–3020

  57. Wang X, Yang M, Cour T, Zhu S, Yu K, Han T (2011) Contextual weighting for vocabulary tree based image retrieval. In: IEEE international conference on computer vision (ICCV’2011), pp 209–216

  58. van (2006) de Weijer, J., Schmid, C.: Coloring local feature extraction. In: European conference on computer vision (ECCV’2006), vol Part II, pp 334–348

  59. Williams A, Yoon P (2007) Content-based image retrieval using joint correlograms. Multimedia Tools and Applications 34(2):239–248

    Article  Google Scholar 

  60. Wu P, Manjunanth BS, Newsam SD, Shin HD (1999) A texture descriptor for image retrieval and browsing. In: IEEE workshop on Content-Based access of image and video libraries (CBAIVL’99), pp 3–7

  61. Yang X, Bai X, Latecki LJ, Tu Z (2008) Improving shape retrieval by learning graph transduction. In: European conference on computer vision (ECCV’2008), vol 4, pp 788–801

  62. Yang X, Koknar-Tezel S, Latecki LJ (2009) Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval. In: IEEE conference on computer vision and pattern recognition (CVPR’2009), pp 357–364

  63. Yang X, Latecki LJ (2011) Affinity learning on a tensor product graph with applications to shape and image retrieval. In: IEEE conference on computer vision and pattern recognition (CVPR’2011), pp 2369–2376

  64. Young HP (1974) An axiomatization of borda’s rule. J Econ Theory 9(1):43–52

    Article  Google Scholar 

  65. Zhang S, Yang M, Cour T, Yu K, Metaxas DN (2012) Query specific fusion for image retrieval. In: ECCV, pp 660–673

Download references

Acknowledgments

The authors are grateful to São Paulo Research Foundation - FAPESP (grant 2013/08645-0), CNPq (grants 306580/2012-8 and 484254/2012-0), CAPES, AMD, and Microsoft Research for the financial support. Authors are also grateful to Rodrigo T. Calumby for his support in the experiments involving multimodal searches.

Author information

Authors and Affiliations

  1. Department of Statistics, Applied Mathematics and Computing, São Paulo State University (UNESP), Rio Claro, Brazil

    Daniel Carlos Guimarães Pedronette

  2. Recod Lab - Institute of Computing, University of Campinas (UNICAMP), Campinas, Brazil

    Ricardo da S. Torres

Authors
  1. Daniel Carlos Guimarães Pedronette
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ricardo da S. Torres
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Carlos Guimarães Pedronette.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pedronette, D.C.G., Torres, R.d.S. Combining re-ranking and rank aggregation methods for image retrieval. Multimed Tools Appl 75, 9121–9144 (2016). https://doi.org/10.1007/s11042-015-3044-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3044-0

Keywords

Search

Navigation