Skip to main content
SpringerLink
Log in

Hypergraph learning with collaborative representation for image search reranking

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

Image search reranking has received considerable attention in recent years. It aims at refining the text-based image search results by boosting the rank of relevant images. Hypergraph has been widely used for relevance estimation, where textual results are taken as vertices and the hypergraph ranking is performed to learn their relevance scores. Rather than using the K-nearest neighbor method, recent works have adopted the sparse representation to effectively construct an informative hypergraph. The sparse representation is insensitive to noise and can capture the real neighborhood structure. However, it suffers from a heavy computational cost. Motivated by this observation, in this paper, we leveraged the ridge regression for hypergraph construction. By imposing an \(\ell _2\)-regularizer on the size of their regression coefficients, the ridge regression enforces the training samples to collaborate to represent one query. The so-called collaborative representation exhibits more discriminative power and robustness while being computationally efficient. Thereafter, based on the obtained collaborative representation vectors, we measured the pairwise similarities among samples and generated hyperedges. Extensive experiments on the public MediaEval benchmarks demonstrated the effectiveness and superiority of our method over the state-of-the-art reranking methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Boteanu B, Constantin M-G, Ionescu B (2016) LAPI @ 2016 retrieving diverse social images task: a pseudo-relevance feedback diversification perspective. In: MediaEval 2016 workshop, Hilversum, Netherlands, 20–21 October 2016

  2. Boteanu B, Mironică I, Ionescu B (2017) Pseudo-relevance feedback diversification of social image retrieval results. Multimed Tools Appl 76(9):11889–11916

    Article  Google Scholar 

  3. Bouhlel N, Feki G, Ammar AB, Amar CB (2017) A hypergraph-based reranking model for retrieving diverse social images. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 10424 LNCS. Springer, Cham, pp 279–291

  4. Bouhlel N, Ksibi A, Ammar AB, Amar CB (2016) Semantic-aware framework for mobile image Search. In: International conference on intelligent systems design and applications, ISDA, vol 2016. IEEE, pp 479–484

  5. Brin S, Page L (2012) The anatomy of a large-scale hypertextual web search engine. In: Computer networks, vol 56. Elsevier Science Publishers B. V., Amsterdam, pp 3825–3833

  6. Cai J, Zha ZJ, Wang M, Zhang S, Tian Q (2015) An attribute-assisted reranking model for web image search. IEEE Trans Image Process 24(1):261–272

    Article  MathSciNet  Google Scholar 

  7. Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with \(\ell _1\)-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  Google Scholar 

  8. Cheng XQ, Du P, Guo J, Zhu X, Chen Y (2013) Ranking on data manifold with sink points. IEEE Trans Knowl Data Eng 25(1):177–191

    Article  Google Scholar 

  9. Dang-Nguyen D, Piras L, Giacinto G, Boato G, De Natale FGB (2015) A hybrid approach for retrieving diverse social images of landmarks. In: 2015 IEEE international conference on multimedia and expo (ICME), pp 1–6

  10. Feki G, Ammar AB, Amar CB (2014) Adaptive semantic construction for diversity-based image retrieval. In: Proceedings of the international conference on knowledge discovery and information retrieval (KDIR 2014), pp 444–449

  11. Feki G, Fakhfakh R, Ammar AB, Amar CB (2015) Knowledge structures: which one to use for the query disambiguation? In: 2015 15th international conference on intelligent systems design and applications (ISDA), pp 499–504

  12. Feki G, Fakhfakh R, Ammar AB, Ben Amar C (2016) Query disambiguation: user-centric approach. J Inf Assur Secur 11:144–156

    Google Scholar 

  13. Feki G, Fakhfakh R, Bouhlel N, Ammar AB, Amar CB (2016) REGIM @ 2016 retrieving diverse social images task. In: Working notes proceedings of the MediaEval 2016 workshop, Hilversum, The Netherlands, 20–21 Oct 2016

  14. Feki G, Ksibi A, Ammar AB, Amar CB (2013) Improving image search effectiveness by integrating contextual information. In: 2013 11th international workshop on content-based multimedia indexing (CBMI), pp 149–154

  15. Ferreira C, Calumby R, Do I, Araujo C, Dourado I, Munoz J, Penatti O, Li L, Almeida J, Torres R (2016) Recod @ MediaEval 2016: diverse social images retrieval. In: MediaEval 2016 workshop, Hilversum, Netherlands, 20–21 October 2016

  16. Guedri B, Zaied M, Amar CB (2011) Indexing and images retrieval by content. In: 2011 International conference on high performance computing simulation, pp 369–375

  17. Hong C, Zhu J (2013) Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval. Neurocomputing 101:94–103

    Article  Google Scholar 

  18. Hsu WH, Kennedy LS, Chang SF (2006) Video search reranking via information bottleneck principle. In: Proceedings of the 14th annual ACM international conference on multimedia, MM 2006, MM’06. ACM, New York, pp 35–44

  19. Huang Y, Liu Q, Zhang S, Metaxas DN (2010) Image retrieval via probabilistic hypergraph ranking. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 3376–3383

  20. Ionescu B, Popescu A, Lupu M, GÎnsca AL, Boteanu B, Müller H (2015) Div150Cred: a social image retrieval result diversification with user tagging credibility dataset. In: Proceedings of the 6th ACM multimedia systems conference, MMSys 2015, MMSys’15. ACM, New York, pp 207–212

  21. Ionescu B, Popescu A, Radu AL, Müller H (2016) Result diversification in social image retrieval: a benchmarking framework. Multimed Tools Appl 75(2):1301–1331

    Article  Google Scholar 

  22. Jing P, Su Y, Xu C, Zhang L (2018) HyperSSR: a hypergraph based semi-supervised ranking method for visual search reranking. Neurocomputing 274:50–57

    Article  Google Scholar 

  23. Jing Y, Baluja S (2008) VisualRank: applying pagerank to large-scale image search. IEEE Trans Pattern Anal Mach Intell 30(11):1877–1890

    Article  Google Scholar 

  24. Ksibi A, Feki G, Ammar AB, Amar CB (2013) Effective diversification for ambiguous queries in social image retrieval. In: Wilson R, Hancock E, Bors A, Smith W (eds) Computer analysis of images and patterns. Springer, Berlin, pp 571–578

    Chapter  Google Scholar 

  25. Liu Y, Shao J, Xiao J, Wu F, Zhuang Y (2013) Hypergraph spectral hashing for image retrieval with heterogeneous social contexts. Neurocomputing 119:49–58

    Article  Google Scholar 

  26. Mei T, Rui Y, Li S, Tian Q (2014) Multimedia search reranking: a literature survey. ACM Comput Surv 46(3):38:1–38:38

    Article  Google Scholar 

  27. Mejdoub M, Fonteles L, Ben Amar C, Antonini M (2009) Embedded lattices tree: an efficient indexing scheme for content based retrieval on image databases. J Vis Commun Image Represent 20(2):145–156

    Article  Google Scholar 

  28. Mejdoub M, Fonteles L, Amar CB, Antonini M (2008) Fast indexing method for image retrieval using tree-structured lattices. In: 2008 international workshop on content-based multimedia indexing, pp 365–372

  29. Palotti JRM, Rekabsaz N, Lupu M, Hanbury A (2014) TUW @ retrieving diverse social images task 2014. In: Working notes proceedings of the MediaEval 2014 workshop, Barcelona, Catalunya, Spain, 16–17 Oct 2014

  30. Spampinato C, Palazzo S (2014) Perceive lab@unict at mediaeval 2014 diverse images: random forests for diversity-based clustering. In: MediaEval

  31. Spyromitros-Xioufis E, Papadopoulos S, Ginsca AL, Popescu A, Kompatsiaris Y, Vlahavas I (2015) Improving diversity in image search via supervised relevance scoring. In: ICMR 2015—proceedings of the 2015 ACM international conference on multimedia retrieval, ICMR’15. ACM, New York, pp 323–330

  32. Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris Y, Vlahavas I (2014) Socialsensor: finding diverse images at mediaeval 2014. In: MediaEval 2014 multimedia benchmark workshop, Barcelona, Catalunya, Spain, 16–17 October 2014

  33. Tian X, Yang L, Wang J, Wu X, Hua XS (2011) Bayesian visual reranking. IEEE Trans Multimed 13(4):639–652

    Article  Google Scholar 

  34. Tollari S (2016) Upmc at MediaEval 2016 retrieving diverse social images task. In: CEUR workshop proceedings, vol 1739

  35. Wang M, Liu X, Wu X (2015) Visual classification by \(\ell _1\)-hypergraph modeling. IEEE Trans Knowl Data Eng 27:2564–2574

    Article  Google Scholar 

  36. Wang X, Qiu S, Liu K, Tang X (2014) Web image re-ranking using query-specific semantic signatures. IEEE Trans Pattern Anal Mach Intell 36(4):810–823

    Article  Google Scholar 

  37. Wang Y, Lin X, Wu L, Zhang W (2015) Effective multi-query expansions: robust landmark retrieval. In: MM 2015—proceedings of the 2015 ACM multimedia conference, MM’15. ACM, New York, pp 79–88

  38. Yan R, Hauptmann A, Jin R (2003) Multimedia search with pseudo-relevance feedback. In: Proceedings of the 2nd international conference on image and video retrieval, CIVR’03. Springer, Berlin, pp 238–247

  39. Zheng L, Yang Y, Tian Q (2018) Sift meets cnn: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell 40(5):1224–1244

    Article  Google Scholar 

  40. Zhou D, Huang J, Schölkopf B (2006) Learning with hypergraphs: clustering, classification, and embedding. In: Advances in neural information processing systems (NIPS), vol 19. MIT Press, p 2006

  41. Zhou W, Li H, Lu Y, Tian Q (2013) SIFT match verification by geometric coding for large-scale partial-duplicate web image search. ACM Trans Multimed Comput Commun Appl 9(1):4:1–4:18

    Article  Google Scholar 

  42. Zhou W, Lu Y, Li H, Song Y, Tian Q (2010) Spatial coding for large scale partial-duplicate web image search. In: MM’10—proceedings of the ACM multimedia 2010 international conference, MM’10. ACM, New York, pp 511–520

  43. Zhu L, Shen J, Jin H, Zheng R, Xie L (2015) Content-based visual landmark search via multimodal hypergraph learning. IEEE Trans Cybern 45(12):2756–2769

    Article  Google Scholar 

  44. Zhu X, Goldberg AB, Gael JV, Andrzejewski D (2007) Improving diversity in ranking using absorbing random walks. In: HLT-NAACL

Download references

Acknowledgements

The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the Grant Agreement Number LR11ES48.

Author information

Authors and Affiliations

  1. REGIM: Research Groups in Intelligent Machines, National Engineering School of Sfax (ENIS), University of Sfax, BP 1173, 3038, Sfax, Tunisia

    Noura Bouhlel, Ghada Feki, Anis Ben Ammar & Chokri Ben Amar

Authors
  1. Noura Bouhlel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ghada Feki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anis Ben Ammar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chokri Ben Amar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noura Bouhlel.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouhlel, N., Feki, G., Ben Ammar, A. et al. Hypergraph learning with collaborative representation for image search reranking. Int J Multimed Info Retr 9, 205–214 (2020). https://doi.org/10.1007/s13735-019-00191-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-019-00191-w

Keywords

Search

Navigation